Information processing method, information processing apparatus, and computer program

Abstract

An information processing method, an information processing apparatus, and a computer program that extract features of an intermediate layer that reflects subtle features of input data and perform class discrimination of discriminated data are provided. The information processing method includes: (a) a process of preparing, for each of a plurality of classes, a known feature spectrum group obtained when a plurality of teaching data is input to a machine learning model of a vector neural network type; and (b) a process of performing class discrimination processing of discriminated data using the machine learning model and the known feature spectrum group. Step (b) includes: (b1) a process of calculating a feature spectrum from input of the discriminated data to the machine learning model; (b2) a process of calculating, for each of the plurality of classes, a class-based similarity of the feature spectrum to the known feature spectrum group; and (b3) a process of discriminating a class of the discriminated data from the class-based similarity.

Description

Technical Field

[0001] This disclosure relates to methods, information processing apparatus, and computer programs for using machine learning models to determine the category of classified data. Background Technology

[0002] Patent documents 1 and 2 disclose a model called a capsule network as a vector neural network-type machine learning model utilizing vector neurons. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons, called capsules, as network nodes. Vector neural network-type machine learning models, such as capsule networks, can be used for class discrimination of input data.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: U.S. Patent No. 5,210,798

[0006] Patent document 2: International Publication No. 2019 / 083553.

[0007] The inventors of this disclosure have discovered that in the prior art, subtle features of the input data are omitted during propagation and cannot be correctly reflected in the category discrimination results of the output layer, sometimes resulting in incorrect category discrimination results. This disclosure provides a technique for extracting intermediate layer features that reflect the subtle features of the input data and performing category discrimination on the data. Summary of the Invention

[0008] According to a first aspect of this disclosure, a method is provided for determining the category of discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The method includes: (a) preparing a known set of feature spectra obtained from the output of a specific layer of the multiple vector neuron layers when multiple teaching data are input to the machine learning model for each of a plurality of categories; and (b) performing category discrimination processing on the discrimination data using the machine learning model and the known set of feature spectra. Step (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating a category-wise similarity between the feature spectrum and the known set of feature spectra for each of the plurality of categories; and (b3) determining the category of the discrimination data based on the category-wise similarity.

[0009] According to a second aspect of this disclosure, an information processing apparatus is provided that uses a vector neural network-type machine learning model having multiple vector neuron layers to perform category discrimination processing for classifying discrimination data. The information processing apparatus includes: a memory storing the machine learning model; and a processor performing operations using the machine learning model. The processor is configured to perform the following processes: (a) when multiple teaching data are input to the machine learning model, for each of the multiple categories, reading from the memory a known set of feature spectra obtained from the output of a specific layer of the multiple vector neuron layers; and (b) using the machine learning model and the known set of feature spectra to perform category discrimination processing on the discrimination data. Process (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known set of feature spectra for each of the multiple categories; and (b3) classifying the discrimination data based on the similarity.

[0010] According to a third aspect of this disclosure, a computer program is provided that causes a processor to perform category discrimination processing for classifying discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The computer program causes the processor to perform the following processes: (a) for each of a plurality of categories, reading from memory a known set of feature spectra obtained from the output of a specific layer among the plurality of vector neuron layers when multiple teaching data are input to the machine learning model; and (b) using the machine learning model and the known set of feature spectra to perform category discrimination processing on the discrimination data. Process (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known set of feature spectra for each of the plurality of categories; and (b3) determining the category of the discrimination data based on the similarity. Attached Figure Description

[0011] Figure 1 This is a block diagram of the category discrimination system in the implementation method.

[0012] Figure 2 This is a block diagram of an information processing device.

[0013] Figure 3 This is an explanatory diagram showing the structure of a machine learning model.

[0014] Figure 4This is an illustrative diagram representing other structures of a machine learning model.

[0015] Figure 5 This is a flowchart illustrating the preparation process for a machine learning model.

[0016] Figure 6 This is an explanatory diagram showing the characteristic spectrum.

[0017] Figure 7 This is an explanatory diagram illustrating the use of teaching data to generate a known characteristic spectral set.

[0018] Figure 8 This is an explanatory diagram showing the structure of a known characteristic spectral group.

[0019] Figure 9 This is a flowchart illustrating the processing sequence of the media identification / printing process.

[0020] Figure 10 This is an illustration of how to calculate the category-based similarity of the data to be classified.

[0021] Figure 11 This is an illustration of the first method for calculating similarity by category.

[0022] Figure 12 This is an illustration of the second method for calculating similarity by category.

[0023] Figure 13 This is an illustration of the third method for calculating similarity by category.

[0024] Figure 14 This is an illustration of the first decision method that uses multiple specific layers to determine the category.

[0025] Figure 15 This is an illustration of a second method for determining categories that uses multiple specific layers.

[0026] Symbol Explanation

[0027] 10. Printer; 20. Information processing device; 30. Spectrometer; 110. Processor; 112. Printing processing unit; 114. Category discrimination processing unit; 120. Memory; 130. Interface circuit; 150. Display unit; 200. Machine learning model; 210. Convolutional layer; 220. Main vector neuron layer; 230. First convolutional vector neuron layer; 240. Second convolutional vector neuron layer; 250. Classification vector neuron layer; 310. Similarity calculation unit; 320. Category determination unit. Detailed Implementation

[0028] A. System structure and processing sequence:

[0029] Figure 1 This is a block diagram illustrating a category discrimination system in one embodiment. The category discrimination system is a printing system having a printer 10, an information processing device 20, and a spectrometer 30. The spectrometer 30 is capable of performing spectroscopic measurements on the printing medium PM used in the printer 10 in its unprinted state and obtaining its spectroscopic reflectance. In this disclosure, the spectroscopic reflectance is also referred to as "spectral data." The spectrometer 30 includes, for example, a wavelength-variable interference spectroscopic filter and a monochrome image sensor. The spectroscopic data obtained by the spectrometer 30 is used as discrimination data input into a machine learning model described later. The information processing device 20 uses the machine learning model to perform category discrimination processing on the spectroscopic data and determines which of a plurality of categories the printing medium PM corresponds to. "Category of printing medium PM" refers to the type of printing medium PM. The information processing device 20 controls the printer 10 to perform printing under appropriate printing conditions corresponding to the type of printing medium PM. It should be noted that the category discrimination system disclosed herein can be implemented as a system other than a printing system. For example, it can be implemented as a system that uses one-dimensional data other than discrimination images and spectroscopic data, spectroscopic images, time series data, etc., as discrimination data for category discrimination.

[0030] Figure 2 This is a block diagram illustrating the functions of the information processing device 20. The information processing device 20 includes a processor 110, a memory 120, an interface circuit 130, an input device 140 connected to the interface circuit 130, and a display unit 150. A spectrometer 30 and a printer 10 are also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has the function of performing the processing described in detail below, but also has the function of displaying the data obtained through the processing and the data generated during the processing on the display unit 150.

[0031] The processor 110 functions as a printing processing unit 112 that performs printing processes using the printer 10, and also functions as a category discrimination processing unit 114 that performs category discrimination processing of the spectroscopic data of the printing medium PM. The category discrimination processing unit 114 includes a similarity calculation unit 310 and a category determination unit 320. The printing processing unit 112 and the category discrimination processing unit 114 are implemented by the processor 110 executing a computer program stored in the memory 120. However, these parts 112 and 114 can also be implemented by hardware circuitry. The term "processor" in this specification also includes such hardware circuitry. Furthermore, the processor performing the category discrimination processing can also be a processor included in a remote computer connected to the information processing device 20 via a network.

[0032] The memory 120 stores a machine learning model 200, training data TD, a known feature spectrum set KSp, and a printing setting table PST. The machine learning model 200 is used for processing by the category discrimination processing unit 114. The structure and operation of the machine learning model 200 will be described later. The training data TD is a collection of labeled data used in the learning of the machine learning model 200. In this embodiment, the training data TD is a collection of spectroscopic data. The known feature spectrum set KSp is a collection of feature spectra obtained when the training data TD is input into the learned machine learning model 200. Feature spectra will be described later. The printing setting table PST is a table that records printing settings suitable for various printing media.

[0033] Figure 3 This is an explanatory diagram illustrating the structure of the machine learning model 200. Starting from the input data IM, the machine learning model 200 sequentially includes a convolutional layer 210, a principal vector neuron layer 220, a first convolutional vector neuron layer 230, a second convolutional vector neuron layer 240, and a classification vector neuron layer 250. Among these five layers 210-250, the convolutional layer 210 is the lowest layer, and the classification vector neuron layer 250 is the highest layer. In the following description, layers 210-250 will be referred to as "Conv layer 210," "PrimeVN layer 220," "ConvVN1 layer 230," "ConvVN2 layer 240," and "ClassVN layer 250," respectively.

[0034] In this embodiment, since the input data IM is spectroscopic data, it is arranged in one dimension. For example, the input data IM is data from spectroscopic data in the range of 380nm to 730nm, from which 36 representative values ​​are extracted every 10nm.

[0035] exist Figure 3 The example uses two convolutional vector neuron layers, 230 and 240, but the number of convolutional vector neuron layers is arbitrary, and they can be omitted. However, it is preferable to use more than one convolutional vector neuron layer.

[0036] Figure 3 The structure of each layer 210-250 can be described as follows.

[0037] Description of the structure of machine learning model 200

[0038] • Conv layer 210: Conv[32, 6, 2]

[0039] • PrimeVN layer 220: PrimeVN[26, 1, 1]

[0040] • ConvVN1 layer 230: ConvVN1[20, 5, 2]

[0041] • ConvVN2 layer 240: ConvVN2[16, 4, 1]

[0042] • ClassVN layer 250: ClassVN[n1, 3, 1]

[0043] • Vector dimension VD: VD = 16

[0044] In the descriptions of these layers 210–250, the string before the parentheses is the layer name, and the numbers inside the parentheses are, in order, the number of channels, the surface size of the kernel, and the stride. For example, the layer name for Conv layer 210 is "Conv", the number of channels is 32, the surface size of the kernel is 1×6, and the stride is 2. Figure 3 The descriptions are shown below each layer. The shaded rectangles depicted in each layer represent the kernel surface size used when calculating the output vector of the adjacent upper layer. In this embodiment, since the input data IM is arranged in one dimension, the kernel surface size is also one dimension. It should be noted that the values ​​of the parameters used in the descriptions of layers 210 to 250 are illustrative and can be changed arbitrarily.

[0045] Conv layer 210 is a layer composed of scalar neurons. The other four layers 220-250 are layers composed of vector neurons. A vector neuron is a neuron that takes vectors as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. Hereafter, the term "node" will be used as a higher-level concept than scalar neurons and vector neurons.

[0046] exist Figure 3 Regarding Conv layer 210, a first axis x and a second axis y are shown to define the planar coordinates of the node arrangement, and a third axis z represents the depth. Additionally, the dimensions of Conv layer 210 in the x, y, and z directions are shown to be 1, 16, and 32, respectively. The dimensions in the x and y directions are referred to as "resolution." In this embodiment, the resolution in the x direction is always 1. The dimension in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes representing the position of each node in other layers. However, in… Figure 3 In the layers other than Conv layer 210, the diagrams of these axes x, y, and z are omitted.

[0047] As is well known, the resolution W1 in the y-direction after convolution is given by the following formula.

[0048] W1=Ceil{(W0-Wk+1) / S} (1)

[0049] Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is the function that performs the carry operation on X.

[0050] Figure 3 The resolutions shown for each layer are examples where the resolution in the y-direction of the input data IM is set to 36. The actual resolution of each layer will vary depending on the size of the input data IM.

[0051] ClassVN layer 250 has n1 channels. Figure 3 In the example, n1 = 3. Typically, n1 is an integer greater than 2, representing the number of known categories that can be identified using machine learning model 200. The three channels of ClassVN layer 250 output decision values ​​(category 1 to category 3) for the three known categories. Generally, the category with the highest value among these decision values ​​(category 1 to category 3) is used as the category classification result for the input data IM. Alternatively, if the highest value among the decision values ​​(category 1 to category 3) is less than a predetermined threshold, the category of the input data IM can be determined as unknown.

[0052] In this disclosure, as described later, instead of using the decision values ​​of ClassVN layer 250 (which is the output layer) for Class 1 to Class 3, the classification is determined by the class similarity calculated from the output of a specific vector neuron layer.

[0053] exist Figure 3 The diagram also depicts partial regions Rn within layers 210, 220, 230, 240, and 250. The suffix "n" in partial region Rn indicates the layer. For example, partial region R210 represents a partial region within Conv layer 210. A "partial region Rn" refers to a region within each layer, defined by a planar position (x, y) determined by the position of the first axis x and the second axis y, encompassing multiple channels along the third axis z. Partial region Rn has dimensions of "width" × "height" × "depth" corresponding to the first axis x, second axis y, and third axis z. In this embodiment, the number of nodes contained in a "partial region Rn" is "1 × 1 × depth," i.e., "1 × 1 × number of channels."

[0054] like Figure 3As shown, the feature spectrum Sp_ConvVN1 (described later) is calculated from the output of ConvVN1 layer 230 and input into the similarity calculation unit 310. Similarly, feature spectra Sp_ConvVN2 and Sp_ClassVN are calculated from the outputs of ConvVN2 layer 240 and ClassVN layer 250, respectively, and input into the similarity calculation unit 310. Using these feature spectra Sp_ConvVN1, Sp_ConvVN, Sp_ClassVN, and a pre-made known feature spectrum group KSp, the similarity by category (described later) Sclass_ConvVN1, Sclass_ConvVN2, and Sclass_ClassVN are calculated. The category determination unit 320 uses at least a portion of these similarity by category Sclass_ConvVN1, Sclass_ConvVN2, and Sclass_ClassVN to generate a discrimination result RD. The discrimination result RD includes a discrimination category D_class and a similarity value S_value corresponding to the discrimination category D_class.

[0055] In this disclosure, the vector neuron layer used for similarity calculation is also referred to as a "specific layer." As a specific layer, any number of vector neuron layers, more than one, can be used. It should be noted that the structure of the feature spectrum, the method for calculating similarity using the feature spectrum, and the method for determining the category will be described later.

[0056] Figure 4 This is an illustrative diagram representing other structures of the machine learning model 200. The machine learning model 200 uses input data arranged in a one-dimensional arrangement. Figure 3 The difference between the machine learning model 200 and the model 200 is that the input data IM is arranged in two dimensions. Figure 4 The structure of each layer 210-250 can be described as follows.

[0057] Description of the structure of each layer

[0058] • Conv layer 210: Conv[32, 5, 2]

[0059] • PrimeVN layer 220: PrimeVN[16, 1, 1]

[0060] • ConvVN1 layer 230: ConvVN1[12, 3, 2]

[0061] • ConvVN2 layer 240: ConvVN2[6, 3, 1]

[0062] • ClassVN layer 250: ClassVN[n1, 4, 1]

[0063] • Vector dimension VD: VD = 16

[0064] Figure 4 The machine learning model 200 shown can, for example, be used in a class discrimination system to classify images. However, in the following description, the model used... Figure 3 The machine learning model 200 is shown.

[0065] Figure 5 This is a flowchart illustrating the processing sequence of the preparation steps for a machine learning model. This preparation step, for example, is performed by the manufacturer of printer 10.

[0066] exist Figure 5 In step S110, the category discrimination processing unit 114 uses multiple teaching data TDs to perform learning of the machine learning model 200. Each teaching data TD is pre-labeled. In this embodiment, it is assumed that any one of labels 1 to 3 is assigned to each teaching data TD. These labels correspond to the three categories 1 to 3 of the machine learning model 200. In this disclosure, "label" and "category" have the same meaning.

[0067] When the learning process using multiple teaching data TDs is complete, the learned machine learning model 200 is stored in memory 120. Figure 5 In step S120, multiple teaching data TDs are input again into the learned machine learning model 200 to generate a known feature spectrum set KSp. The known feature spectrum set KSp is the set of feature spectra described below.

[0068] Figure 6 This is an explanatory diagram showing the feature spectrum Sp obtained by inputting arbitrary input data into the learned machine learning model 200. Here, the feature spectrum Sp obtained from the output of ConvVN1 layer 230 will be explained. Figure 6 The horizontal axis represents the position of vector elements of the output vectors of multiple nodes contained in a partial region R230 of the ConvVN1 layer 230. The position of each vector element is represented by a combination of the element number ND and the channel number NC of the output vector in each node. In this embodiment, since the vector dimension is 16, the element numbers ND of the output vectors are 16 from 0 to 15. Furthermore, since the number of channels in the ConvVN1 layer 230 is 20, the channel numbers NC are 20 from 0 to 19. In other words, the characteristic spectrum Sp is obtained by arranging the multiple element values ​​of the output vectors of each vector neuron contained in a partial region R230 across multiple channels along the third axis z.

[0069] Figure 6 The vertical axis represents the characteristic value C at each spectral position. VIn this example, the eigenvalue C V The values ​​V of each element of the output vector. ND It should be noted that, as an eigenvalue C... V The values ​​V of each element in the output vector can be used. ND The value obtained by multiplying by the normalization coefficient (described later), or by using the normalization coefficient directly. In the latter case, the eigenvalue C contained in the characteristic spectrum Sp. V The number of channels is equal to the number of nodes, which is 20. It should be noted that the normalization coefficient is a value equivalent to the vector length of the output vector of that node.

[0070] The number of feature spectra Sp obtained from the output of ConvVN1 layer 230 for a given input data is equal to the number of planar positions (x, y) of ConvVN1 layer 230, i.e., the number of partial regions R230, and therefore is 6. Similarly, for a given input data, three feature spectra Sp are obtained from the output of ConvVN2 layer 240, and one feature spectrum Sp is obtained from the output of ClassVN layer 250.

[0071] When the similarity calculation unit 310 inputs the teaching data TD back into the machine learning model 200 after it has been learned, it calculates... Figure 6 The characteristic spectrum Sp shown is recorded in memory 120 as a known characteristic spectrum group KSp.

[0072] Figure 7 This diagram illustrates the use of teaching data TD to create a known feature spectrum group KSp. In this example, by inputting the teaching data TD labeled 1 to 3 into the learned machine learning model 200, the outputs of three vector neuron layers—ConvVN1 layer 230, ConvVN2 layer 240, and ClassVN layer 250—result in feature spectra KSp_ConvVN1, KSp_ConvVN2, and KSp_ClassVN corresponding to each label or class. These feature spectra KSp_ConvVN1, KSp_ConvVN2, and KSp_ClassVN are stored in memory 120 as the known feature spectrum group KSp.

[0073] Figure 8 This is an explanatory diagram representing the structure of a known characteristic spectral group KSp. In this example, the known characteristic spectral group KSp_ConvVN1 obtained from the output of ConvVN1 layer 230 is shown. The known characteristic spectral group KSp_ConvVN2 obtained from the output of ConvVN2 layer 240 also has the same structure as the known characteristic spectral group KSp_ConvVN1 obtained from the output of ClassVN layer 250, but... Figure 8The illustration is omitted. It should be noted that, as a known feature spectral set KSp, it is sufficient to register a known feature spectral set obtained from the output of at least one vector neuron layer.

[0074] The known characteristic spectral set KSp_ConvVN1 includes the following parameters: parameter i representing the order of labels or categories, parameter j representing the order of specific layers, parameter k representing the order of partial regions Rn, parameter q representing the data number, and the known characteristic spectral set KSp. The known characteristic spectral set KSp and... Figure 6 The characteristic spectra Sp are the same.

[0075] The parameter i for the category takes a value of 1 to 3, the same as the label. The parameter j for the specific layer takes a value of 1 to 3 indicating which of the three specific layers 230, 240, and 250 it is. The parameter k for the partial region Rn takes a value indicating which of the multiple partial regions Rn contained in each specific layer it is, that is, which of the planar positions (x, y). Regarding ConvVN1 layer 230, since there are 6 partial regions R230, k = 1 to 6. The parameter q for the data number indicates the number of the teaching data with the same label attached, taking a value of 1 to max1 for category 1, a value of 1 to max2 for category 2, and a value of 1 to max3 for category 3.

[0076] It should be noted that the multiple teaching data TDs used in step S120 do not need to be the same as the multiple teaching data TDs used in step S110. However, in step S120, if some or all of the multiple teaching data TDs used in step S110 are used, it has the advantage that no new teaching data needs to be prepared.

[0077] Figure 9 This is a flowchart illustrating the processing sequence of the media identification / printing process using a learned machine learning model. This media identification / printing process is performed, for example, by a user using printer 10.

[0078] In step S210, the user instructs the category determination processing unit 114 whether category determination processing is required for the target printing medium. Even when the user knows the type of the target printing medium, they can still instruct that category determination processing is needed for confirmation. If category determination processing is not required, the process proceeds to step S260, where the user selects printing settings suitable for the target printing medium. In step S270, the printing processing unit 112 causes the printer 10 to perform printing using the target printing medium. On the other hand, if the type of the target printing medium is unknown and category determination processing is required, the process proceeds to step S220.

[0079] In step S220, the category discrimination processing unit 114 obtains spectroscopic data by having the spectrometer 30 perform a spectroscopic measurement of the target printing medium. This spectroscopic data is used as discrimination data input into the machine learning model 200.

[0080] In step S230, the category discrimination processing unit 114 inputs discrimination data into the learned machine learning model 200 and calculates the feature spectrum Sp. In step S240, the similarity calculation unit 310 calculates the category similarity using the feature spectrum Sp obtained from the input discrimination data and the registered known feature spectrum group KSp.

[0081] Figure 10 This is an explanatory diagram illustrating the process of calculating the category-based similarity of the discriminative data. When the discriminative data is input into the machine learning model 200, the category discrimination processing unit 114 calculates the feature spectra Sp_ConvVN1, Sp_ConvVN2, and Sp_ClassVN from the outputs of the ConvVN1 layer 230, ConvVN2 layer 240, and ClassVN layer 250, respectively. The similarity calculation unit 310 uses the feature spectrum Sp_ConvVN1 obtained from the output of the ConvVN1 layer 230 and the known feature spectrum group KSp_ConvVN1 to calculate the category-based similarity Sclass_ConvVN1. The specific calculation method for the category-based similarity will be described later. Similarly, the category-based similarities Sclass_ConvVN2 and Sclass_ClassVN are calculated for the ConvVN2 layer 240 and the ClassVN layer 250.

[0082] While it is not necessary to use three separate vector neuron layers 230, 240, and 250 to generate all the category-specific similarities Sclass_ConvVN1, Sclass_ConvVN2, and Sclass_ClassVN, it is preferable to use more than one of these vector neuron layers to calculate the category-specific similarities. As mentioned above, in this disclosure, the vector neuron layer used for similarity calculation is referred to as a "specific layer".

[0083] In step S250, the category determination unit 320 determines the category of the target printing medium, i.e., the type of the target printing medium, based on the category similarity obtained in step S240. In step S260, the printing processing unit 112 selects printing settings according to the type of the target printing medium, referring to the printing setting table PST. In step S270, the printing processing unit 112 performs printing according to the printing settings. Figure 9Even when the type of the target printing medium is unknown, the machine learning model 200 can be used to determine the type of the target printing medium, thus enabling printing to be performed using printing settings suitable for that type.

[0084] B. Similarity calculation method:

[0085] As mentioned above Figure 10 The method for calculating similarity by category shown can, for example, employ any of the following three methods.

[0086] (1) The first calculation method M1 is used to calculate the similarity by category without considering the correspondence between the feature spectrum Sp and a portion of the known feature spectrum group KSp.

[0087] (2) The second operation method M2 is used to calculate the category-specific similarity between the characteristic spectrum Sp and the corresponding partial region Rn of the known characteristic spectrum group KSp.

[0088] (3) The third calculation method M3 is used to calculate the similarity by category without considering any partial region Rn.

[0089] The following sections will explain how to calculate the category-specific similarity Sclass_ConvVN1 from the output of ConvVN1 layer 230, based on the three calculation methods M1, M2, and M3.

[0090] Figure 11 This is an explanatory diagram of the first operation method M1, representing the similarity by category. In the first operation method M1, firstly, based on the output of the ConvVN1 layer 230 (which is a specific layer), a local similarity S(i, j, k) representing the similarity for each category i is calculated for each partial region k. Then, based on these local similarities S(i, j, k), the similarity for each category i is calculated... Figure 11 The right side shows any one of the three category-based similarity values ​​Sclass(i,j). The category-based similarity Sclass(i,j) is compared with... Figure 3 as well as Figure 10 The similarity by category shown is the same for Sclass_ConvVN1.

[0091] In the first operation method M1, the local similarity S(i,j,k) is calculated using the following formula.

[0092] S(i,j,k)=max[G{Sp(j,k),KSp(i,j,k=all,q=all)}] (2)

[0093] Here,

[0094] i is a parameter representing the category.

[0095] j is a parameter representing a specific layer.

[0096] k is a parameter representing a partial region Rn.

[0097] q is a parameter representing the data number.

[0098] G{a, b} is a function that calculates the similarity between a and b.

[0099] Sp(j,k) is a characteristic spectrum obtained from the output of a specific region k of a specific layer j based on the discrimination data.

[0100] KSp(i,j,k=all,q=all) is Figure 8 The known characteristic spectra of all data numbers q in all partial regions k of a specific layer j associated with category i in the known characteristic spectral group KSp are shown.

[0101] max[X] is a logical operation that retrieves the maximum value among the values ​​of X.

[0102] It should be noted that in the function G{a, b} for calculating similarity, 'a' is an OR set, 'b' is a set, and the function returns multiple values. For example, the function G{a, b} can use a formula to calculate cosine similarity or a formula to calculate similarity corresponding to distance.

[0103] Figure 11 The three types of category-specific similarity Sclass(i,j) shown on the right are obtained by taking the maximum, average, or minimum value of the local similarity S(i,j,k) with respect to multiple partial regions k for each category i. Which operation—maximum, average, or minimum—is used depends on the intended use of the category discrimination process. For example, when the purpose is to identify objects using natural images, it is preferable to take the maximum value of the local similarity S(i,j,k) for each category i to obtain the category-specific similarity Sclass(i,j). Conversely, when the purpose is to identify the type of printed media, or to determine whether an industrial product is good or bad using an image, it is preferable to obtain the category-specific similarity Sclass(i,j) by taking the minimum value of the local similarity S(i,j,k) for each category i. Furthermore, it is also considered that it is preferable to obtain the category-specific similarity Sclass(i,j) by taking the average value of the local similarity S(i,j,k) for each category i. The user can pre-determine which of these three operations to use through experimentation or experience.

[0104] exist Figure 11In the example, the final discrimination result RD_ConvVN1 is further determined based on the category similarity Sclass(i,j). The discrimination result RD_ConvVN1 can be represented in the form of a discrimination category D_class and a similarity value S_value corresponding to the discrimination category D_class. The similarity value S_value is obtained by taking the maximum of the similarity values ​​for the three categories 1 to 3 in the category similarity Sclass(i,j). The discrimination category D_class is the category with the highest similarity value in the category similarity Sclass(i,j).

[0105] As described above, in the first method M1 for calculating similarity by category,

[0106] (1) Based on the discrimination data, calculate the local similarity S(i,j,k). This local similarity S(i,j,k) is the similarity between the feature spectrum Sp obtained from the output of a specific region k of a specific layer j and all known feature spectrum groups KSp associated with that specific layer j and each category i.

[0107] (2) The similarity Sclass(i,j) by category is obtained by taking the maximum, average, or minimum value of the local similarity S(i,j,k) with respect to multiple partial regions k for each category i.

[0108] (3) For each category i, calculate the maximum value of the category-based similarity Sclass(i,j) as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp.

[0109] (4) Determine the category D_class that corresponds to the maximum similarity value S_value among multiple categories i.

[0110] According to the first operation method M1, the similarity Sclass(i,j) and the discrimination result by category can be obtained through relatively simple operations and order.

[0111] Figure 12 This is an explanatory diagram of the second operation method M2, which represents the similarity by category. In the second operation method M2, the local similarity S(i,j,k) is calculated using the following formula instead of equation (2) above.

[0112] S(i,j,k)=max[G{Sp(j,k),KSp(i,j,k,q=all)}] (3)

[0113] Here,

[0114] KSp(i,j,k,q=all) is Figure 8The known characteristic spectra of all data number q in a specific partial region k of a specific layer j associated with category i are shown in the known characteristic spectral group KSp.

[0115] In the first operation method M1 described above, the known characteristic spectral set KSp(i,j,k=all,q=all) is used for all partial regions k of a specific layer j. In contrast, in the second operation method M2, the known characteristic spectral set KSp(i,j,k,q=all) is used only for partial regions k that are identical to the characteristic spectrum Sp(j,k). The other methods in the second operation method M2 are the same as those in the first operation method M1.

[0116] In the second similarity calculation method M2 based on category,

[0117] (1) Based on the discrimination data, calculate the local similarity S(i,j,k). This local similarity S(i,j,k) is the similarity between the feature spectrum Sp obtained from the output of a specific region k of a specific layer j and all known feature spectrum groups KSp associated with the specific region k of the specific layer j and each category i.

[0118] (2) The similarity Sclass(i,j) by category is obtained by taking the maximum, average, or minimum value of the local similarity S(i,j,k) with respect to multiple partial regions k for each category i.

[0119] (3) For each category i, calculate the maximum value of the category-based similarity Sclass(i,j) as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp.

[0120] (4) Determine the category D_class that corresponds to the maximum similarity value S_value among multiple categories i.

[0121] According to the second operation method M2, the similarity Sclass(i,j) and the discrimination result by category can also be obtained through relatively simple operations and order.

[0122] Figure 13 This is an explanatory diagram of the third operation method M3, which represents the similarity by category. In the third operation method M3, the local similarity S(i,j,k) is not calculated; instead, the similarity by category Sclass(i,j) is calculated from the output of the ConvVN1 layer 230, which is a specific layer.

[0123] The following formula is used to calculate the category-specific similarity Sclass(i,j) obtained by the third operation method M3.

[0124] Sclass(i,j)=max[G{Sp(j,k=all),KSp(i,j,k=all,q=all)}](4)

[0125] Here,

[0126] Sp(j, k = all) is a characteristic spectrum obtained from the output of all partial regions k of a specific layer j based on the discrimination data.

[0127] The method for calculating the final judgment result based on the category similarity Sclass(i,j) is the same as the first operation method M1 and the second operation method M2 mentioned above.

[0128] As mentioned above, in the third similarity calculation method M3 by category,

[0129] (1) Based on the discrimination data, calculate the category similarity Sclass(i,j) for each category. The category similarity Sclass(i,j) is the similarity between all feature spectra Sp obtained from the output of a specific layer j and all known feature spectrum groups KSp associated with that specific layer j and each category i.

[0130] (2) For each category i, calculate the maximum value of multiple category-based similarities Sclass(i,j) as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp.

[0131] (3) The category corresponding to the maximum similarity value S_value among multiple categories i is determined as the discriminant category D_class.

[0132] According to the third operation method M3, the similarity Sclass(i,j) and the discrimination result by category can be obtained through simpler operations and order.

[0133] The three computational methods M1 to M3 described above all determine the classification based on the similarity of categories for each specific layer i. As mentioned above, in this embodiment, it is possible to... Figure 3 One or more of the multiple vector neuron layers 230, 240, and 250 shown are used as specific layers to compute category-specific similarity, which determines the category of the data. When using multiple specific layers, the following category determination method can be employed, for example.

[0134] Figure 14This is an explanatory diagram illustrating the first decision method MM1, which uses multiple specific layers to determine the classification. Here, for three specific layers—ConvVN1 layer 230, ConvVN2 layer 240, and ClassVN layer 250—the classification results RD_ConvVN1, RD_ConvVN2, and RD_ClassVN are calculated respectively. Among the multiple classification results RD_ConvVN1, RD_ConvVN2, and RD_ClassVN representing the classification class D_class for the multiple specific layers, the class that appears most frequently is adopted as the final classification class D_class. Figure 14 In the example, since category 3 appears most frequently, it is adopted or derived as the final discriminant category D_class. According to this first decision method MM1, category discrimination can be performed with higher precision using multiple specific layers. It should be noted that in the case of multiple most frequent categories, these multiple categories can also be presented to the user as all discriminant categories.

[0135] Figure 15 This is an illustration of MM2, a second decision method that uses multiple specific layers to determine the classification category. In MM2, the classification category is determined using the layer that represents the most statistically significant decision result among multiple specific layers. Figure 15 In the example, ConvVN1 layer 230 and ConvVN2 layer 240 are used as specific layers. First, for each partial region k of ConvVN1 layer 230, the process of determining the category with the maximum local similarity S(i, j, k) is performed, and the category parameter value i of that category is assigned to each partial region k. It should be noted that the category parameter value i is a value representing the order among multiple categories. The category parameter value i is, for example, a consecutive integer. In this embodiment, the category parameter value i is the same as the category i. Similarly, for ConvVN2 layer 240, the process of determining the category with the maximum local similarity S(i, j, k) and assigning the category parameter value i of that category to each partial region k is performed.

[0136] Furthermore, in the second decision method MM2, for each region k, if there is no difference in the local similarity S between categories—that is, if the error or variance of multiple categories of a certain region k with respect to the local similarity S is within a threshold—then a category parameter value may not be assigned to that region k. When calculating the variance of the category parameter values, the region k without assigned category parameter values ​​is excluded. Therefore, the variance can be calculated only in the feature portion, thus enabling higher accuracy in category discrimination.

[0137] In the second decision method MM2, the variance is further calculated within each specific layer for the distribution of the class parameter values ​​i across multiple partial regions k. This variance is the value of the statistical variance of the class parameter values ​​i. Figure 15 In the example, the variance of ConvVN1 layer 230 is 0.14, and the variance of ConvVN2 layer 240 is 0.22. Among these specific layers 230 and 240, since a greater deviation in the expected distribution of the class parameter value i would result in a clearer judgment, the class discrimination result for the specific layer with the lower variance is adopted. In other words, the class-specific similarity obtained for the specific layer with the smallest variance among multiple specific layers is used to determine the class of the data. According to this second decision method MM2, class discrimination can be performed with higher accuracy using multiple specific layers.

[0138] It should be noted that methods other than those mentioned above can also be used for category discrimination. For example, the result of the layer that is closest to the output layer among multiple specific layers can be prioritized as the discrimination result.

[0139] As described above, in this embodiment, the feature spectrum Sp is calculated based on the output of a specific layer of the machine learning model 200 according to the input of the discrimination data. The similarity Sclass(i,j) of the feature spectrum Sp with the known feature spectrum group KSp for each of the multiple categories is calculated. The category of the discrimination data is determined based on the similarity Sclass(i,j), thus improving the accuracy of category discrimination.

[0140] C. Methods for calculating the output vectors of each layer in a machine learning model:

[0141] Figure 3 The calculation method for the output of each layer in the machine learning model 200 shown is as follows. Figure 4 The machine learning model 200 shown is the same except for the values ​​of each parameter.

[0142] Each node in PrimeVN layer 220 treats the scalar outputs of the 1×1×32 nodes in Conv layer 210 as a 32-dimensional vector, multiplies this vector by a transformation matrix, and obtains the vector output of that node. This transformation matrix consists of elements of a kernel with a surface size of 1×1, and is updated through learning by machine learning model 200. It should be noted that the processing of Conv layer 210 and PrimeVN layer 220 can also be combined to form a single main vector neuron layer.

[0143] When the PrimeVN layer 220 is referred to as the "lower layer L" and the ConvVN1 layer 230 adjacent to it is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined by the following formula.

[0144] Mathematical Formula 1

[0145]

[0146] u j =∑ i v ij (E2)

[0147] a j =F(||u j ||) (E3)

[0148]

[0149] Here,

[0150] M L i It is the output vector of the i-th node in the lower layer L.

[0151] M L+1 j It is the output vector of the j-th node in the upper layer L+1.

[0152] v ij It is the output vector M L+1 The prediction vector of j,

[0153] W L ij It is used to determine the output vector M of the lower layer L. L i To calculate the prediction vector v ij The prediction matrix,

[0154] u j It is the prediction vector v ij The sum, that is, the sum vector as a linear combination,

[0155] a j By using the sum vector u j norm |u j The activation values ​​of the normalized coefficients obtained by normalization.

[0156] F(X) is a normalization function that normalizes X.

[0157] As a normalization function, F(X) can be expressed, for example, by the following equation (E3a) or (E3b).

[0158] Mathematical formula 2

[0159]

[0160]

[0161] Here,

[0162] k is the ordinal number of all nodes in the upper layer L+1.

[0163] β is an adjustment parameter that can be any positive coefficient, for example, β = 1.

[0164] In equation (E3a) above, for all nodes in the upper layer L+1, the sum vector u is obtained by applying the Softmax function. j norm |u j By normalizing, we can obtain the activation value a. j On the other hand, in equation (E3b), by summing the vector u... j norm |u j |Divide by the norm of all nodes in the upper layer L+1|u j The sum of these values ​​yields the activation value a. j It should be noted that, as the normalization function F(X), other functions besides those in equations (E3a) and (E3b) can also be used.

[0165] The ordinal number i in the above equation (E2) is used to determine the output vector M of the j-th node in the upper layer L+1. L+1 j The ordinal numbers of the nodes used, assigned to the lower-level layer L for convenience, are values ​​from 1 to n. Additionally, the integer n is used to determine the output vector M of the j-th node in the upper-level layer L+1. L+1 j The number of nodes in the lower-level layer L used is given by the following formula.

[0166] n = Nk × Nc (E5)

[0167] Here, Nk is the surface size of the kernel, and Nc is the number of channels in the PrimeVN layer 220, which is the lower-level layer. Figure 3 In the example, Nk = 5, Nc = 26, so n = 130.

[0168] To determine the output vector of ConvVN1 layer 230, a kernel is used with a surface size of 1×5 and a depth of 1×5×26=130 elements, which are the prediction matrix W. L ij Furthermore, to generate the output vectors of the 20 channels of ConvVN1 layer 230, 20 sets of this kernel are required. Therefore, the prediction matrix W of the kernel used to calculate the output vectors of ConvVN1 layer 230 is... L ij The number is 130 × 20 = 2600. These prediction matrices WL ij It is updated through learning from machine learning model 200.

[0169] From equations (E1) to (E4) above, it can be seen that the output vector M of each node in the upper layer L+1 is... L+1 j The following calculations will yield the result.

[0170] (a) Convert the output vectors M of each node in the lower layer L L i Multiply by the prediction matrix W L ij To obtain the prediction vector v ij ,

[0171] (b) Calculate the prediction vector v obtained from each node of the lower layer L. ij The sum, that is, the sum vector u as a linear associativity. j ,

[0172] (c) By adjusting the sum vector u j norm |u j |The activation value 'a' is obtained by normalizing the input and outputting the normalization coefficient. j ,

[0173] (d) sum vector u j Divide by norm |u j |, further multiplied by the activation value a j .

[0174] It should be noted that the activation value a j It is through the norm |u of all nodes in the upper layer L+1 j The normalization coefficients are obtained by normalization. Therefore, the activation value a can be normalized. j It is considered as an index representing the relative output intensity of each node among all nodes in the upper layer L+1. In typical examples, the norm used in equations (E3), (E3a), (E3b), and (4) is the L2 norm representing the vector length. At this time, the activation value a j Equivalent to output vector M L+1 j The length of the vector. Activation value a j It is only used in equations (E3) and (E4) above, so there is no need for output from the node. However, it is also possible to configure the upper layer L+1 to output the activation value a to the outside. j .

[0175] The structure of a vector neural network is largely the same as that of a capsule network; the vector neurons in a vector neural network are equivalent to the capsules in a capsule network. However, the operations used in vector neural networks based on equations (E1) to (E4) above differ from those used in capsule networks. The biggest difference lies in the fact that in capsule networks, the prediction vector v on the right-hand side of equation (E2) above... ij Each weight is multiplied by a weight, and the weight is searched through repeated dynamic routing. On the other hand, in the vector neural network of this embodiment, the output vector M is obtained by sequentially calculating equations (E1) to (E4) above. L+1 j Therefore, it has the advantages of not requiring repeated dynamic routing and faster computation. In addition, compared with capsule networks, the vector neural network of this embodiment requires less memory for computation. According to the inventors' experiments, it also has the advantage of requiring approximately 1 / 2 to 1 / 3 of the memory.

[0176] In using nodes that take vectors as input and output, vector neural networks are similar to capsule networks. Therefore, the advantages of using vector neurons are also the same as in capsule networks. Furthermore, the multiple layers 210-250 are similar to those in typical convolutional neural networks in that they exhibit features over larger regions as the layers increase, and features over smaller regions as the layers decrease. Here, "feature" refers to the characteristic portion contained in the input data of the neural network. In vector neural networks or capsule networks, the output vector of a node contains spatial information representing the spatial information of the feature represented by that node, which is superior to typical convolutional neural networks. That is, the vector length of a node's output vector represents the probability of the feature represented by that node, and the vector direction represents spatial information such as the direction or scale of that feature. Therefore, the vector directions of the output vectors of two nodes belonging to the same layer represent the positional relationship of their respective features. Alternatively, it can be said that the vector directions of the output vectors of the two nodes represent the changes in the feature. For example, if a node corresponds to the feature of "object," the direction of the output vector can represent changes in the object's fineness, enhancement method, etc. In typical convolutional neural networks, it can be said that the spatial information of features is lost due to pooling processing. The result is that vector neural networks or capsule networks have the advantage of superior performance in recognizing input data compared to conventional convolutional neural networks.

[0177] Vector neural networks (NNs) can also be considered to have the following advantages. Specifically, in NNs, the output vectors of nodes have the advantage of representing the features of the input data as coordinates in a continuous space. Therefore, output vectors can be evaluated as follows: if the vector directions are similar, the features are similar. Additionally, they have the following advantages: even if the teaching data cannot cover the features contained in the input data, the feature can still be determined through interpolation. On the other hand, conventional convolutional neural networks, due to the unordered compression caused by pooling, have the disadvantage of not being able to represent the features of the input data as coordinates in a continuous space.

[0178] The outputs of each node in ConvVN2 layer 240 and ClassVN layer 250 are also determined using the same equations (E1) to (E4) described above, so detailed explanations are omitted. The resolution of ClassVN layer 250, which is the top-level layer, is 1×1, and the number of channels is n1.

[0179] The output of ClassVN layer 250 is transformed into multiple decision values ​​for known categories, from category 0 to category 2. These decision values ​​are typically normalized using the Softmax function. Specifically, for example, based on the output vector of each node in ClassVN layer 250, the vector length of the output vector is calculated, and further, by performing an operation that normalizes the vector length of each node using the Softmax function, the decision values ​​for each category can be obtained. As described above, the activation value a obtained through equation (E3) above... j It is equivalent to the output vector M L+1 j The value of the vector length is calculated and normalized. Therefore, the activation value 'a' of each node in ClassVN layer 250 can be output. j And directly use it as the judgment value for each category.

[0180] In the above embodiments, the machine learning model 200 uses a vector neural network to obtain the output vector through the operations of equations (E1) to (E4) above, but it can also be replaced by the capsule network disclosed in U.S. Patent No. 5,210,798 and International Publication No. 2009 / 083553.

[0181] Other implementation methods:

[0182] This disclosure is not limited to the embodiments described above, and can be implemented in various ways without departing from its spirit. For example, this disclosure can also be implemented in the following aspects. The technical features in the above embodiments corresponding to the technical features in the various aspects described below can be appropriately replaced or combined to solve part or all of the problems of this disclosure, or to achieve part or all of the effects of this disclosure. Furthermore, if a technical feature is not described as an essential technical feature in this specification, it can be appropriately deleted.

[0183] (1) According to a first aspect of this disclosure, a method is provided for determining the category of discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The method includes: (a) preparing a known set of feature spectra obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input to the machine learning model for each of a plurality of categories; and (b) performing category discrimination processing on the discrimination data using the machine learning model and the known set of feature spectra. Step (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating a category-wise similarity between the feature spectrum and the known set of feature spectra for each of the plurality of categories; and (b3) determining the category of the discrimination data based on the category-wise similarity.

[0184] According to this method, the similarity of characteristic spectra is used to perform category discrimination of the discrimination data, thereby improving the accuracy of category discrimination.

[0185] (2) Alternatively, in the above method, the specific layer has a structure in which vector neurons arranged on a plane defined by the first axis and the second axis are configured as multiple channels along a third axis that is different from the two axes. In the specific layer, the region containing the multiple channels along the third axis, as specified by the plane position defined by the position of the first axis and the position of the second axis, is called a partial region. In this case, the feature spectrum is any one of the following feature spectra obtained for each of the multiple partial regions contained in the specific layer: (i) a first feature spectrum in which multiple element values ​​of the output vectors of each vector neuron contained in the partial region are arranged along the multiple channels along the third axis; (ii) a second feature spectrum obtained by multiplying the element values ​​of the first feature spectrum by a normalization coefficient equivalent to the vector length of the output vector; and (iii) a third feature spectrum in which the normalization coefficient is arranged along the multiple channels along the third axis.

[0186] According to this method, similarity can be calculated using any one of the three feature spectra obtained from the output vector of a specific layer.

[0187] (3) Alternatively, in the above method, step (b2) includes: calculating a plurality of local similarities with respect to each category for the plurality of partial regions of the specific layer by calculating the local similarity between the feature spectrum obtained from the output of a specific partial region of the specific layer and all known feature spectrum groups associated with the specific layer and each category based on the discrimination data; and calculating the category-based similarity by taking the maximum, average or minimum value of the plurality of local similarities with respect to the plurality of partial regions for each category.

[0188] According to this method, similarity by category can be calculated through relatively simple operations.

[0189] (4) Alternatively, in the above method, step (b2) includes: calculating a plurality of local similarities with respect to each category for the plurality of partial regions of the specific layer by calculating the local similarity between the feature spectrum obtained from the output of a specific partial region of the specific layer and all known feature spectrum groups associated with the specific partial region of the specific layer and each category based on the discrimination data; and calculating the category-based similarity by taking the maximum, average or minimum value of the plurality of local similarities with respect to the plurality of partial regions for each category.

[0190] According to this method, similarity by category can be calculated through relatively simple operations.

[0191] (5) Alternatively, in the above method, step (b2) includes: calculating the similarity by category by determining the similarity between all the feature spectra obtained from the output of the specific layer and all the known feature spectrum groups associated with the specific layer and each category, based on the discrimination data.

[0192] According to this method, similarity by category can be calculated through simpler operations.

[0193] (6) Alternatively, in the above method, when there are multiple specific layers, step (b3) includes: obtaining a category discrimination result for each of the multiple specific layers by calculating the category discrimination result by the category similarity for each specific layer; and determining the category with the most frequent category represented by the multiple category discrimination results as the discrimination category of the discrimination data.

[0194] According to this method, class discrimination can be performed with higher precision using multiple specific layers.

[0195] (7) Alternatively, in the above method, when there are multiple specific layers, step (b3) includes: for each partial region of each specific layer, determining the category based on the local similarity with respect to the partial region, and assigning a category parameter value representing the order of the category among the multiple categories to the partial region; for each of the multiple specific layers, calculating the statistical variance of the distribution of the category parameter values ​​in the multiple partial regions; and using the category-based similarity obtained for the specific layer with the smallest variance among the multiple specific layers to determine the category of the determined data.

[0196] According to this method, class discrimination can be performed with higher precision using multiple specific layers.

[0197] (8) According to a second aspect of this disclosure, an information processing apparatus is provided that uses a vector neural network-type machine learning model having multiple vector neuron layers to perform category discrimination processing for classifying discrimination data. The information processing apparatus includes: a memory storing the machine learning model; and a processor performing operations using the machine learning model. The processor is configured to perform the following processes: (a) when multiple teaching data are input to the machine learning model, for each of the multiple categories, reading from the memory a known set of feature spectra obtained from the output of a specific layer of the multiple vector neuron layers; and (b) using the machine learning model and the known set of feature spectra to perform category discrimination processing for the discrimination data. Process (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known set of feature spectra for each of the multiple categories; and (b3) classifying the discrimination data based on the similarity.

[0198] According to this information processing device, since the category discrimination of the discrimination data is performed by utilizing the similarity of the characteristic spectra, the accuracy of the category discrimination can be improved.

[0199] (9) According to a third aspect of this disclosure, a computer program is provided that causes a processor to perform category discrimination processing for classifying discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The computer program causes the processor to perform the following processes: (a) for each of the plurality of categories, reading from memory a known set of feature spectra obtained from the output of a specific layer of the plurality of vector neuron layers when multiple teaching data are input to the machine learning model; and (b) using the machine learning model and the known set of feature spectra, performing the category discrimination processing for the discrimination data. Process (b) includes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known set of feature spectra for each of the plurality of categories; and (b3) classifying the discrimination data based on the similarity.

[0200] According to the computer program, the accuracy of category discrimination can be improved by using the similarity of characteristic spectra to perform category discrimination of the discrimination data.

[0201] This disclosure can also be implemented in various ways other than those described above. For example, it can be implemented as a computer program for implementing the functions of the category discrimination device, a non-transitory storage medium on which the computer program is recorded, etc.

Claims

1. An information processing method characterized by, being a method of discriminating a class of discrimination data using a computer storing a machine learning model of a vector neural network type having a plurality of layers of vector neurons, the information processing method comprising: (a) a process of preparing a known feature spectrum group for each of a plurality of classes; and (b) a process of performing a class discrimination process of the discrimination data using the machine learning model and the known feature spectrum group to discriminate an object or to discriminate a kind of a printed medium or to determine whether an industrial product is a good product, the known feature spectrum group being a feature spectrum group obtained from outputs of a plurality of nodes of a certain layer among the plurality of layers of vector neurons when a plurality of spectroscopic data corresponding to a predetermined class is input to the machine learning model, the process (b) including: (b0) a process of performing spectroscopic measurement on an object or a printed medium or an industrial product to be discriminated by a spectroscopic measuring device to acquire spectroscopic data as the discrimination data; (b1) a process of calculating a feature spectrum from outputs of the plurality of nodes of the certain layer according to input of the discrimination data to the machine learning model; (b2) a process of calculating a class-wise similarity of the feature spectrum to the known feature spectrum group for each of the plurality of classes; and (b3) a process of discriminating a class of the discrimination data according to the class-wise similarity.

2. The information processing method according to claim 1, characterized in that, the certain layer has a structure in which a vector neuron arranged on a plane defined by two axes of a first axis and a second axis is arranged along a third axis different from directions of the two axes as a plurality of channels, in the certain layer, when a region designated by a position of the plane defined by a position of the first axis and a position of the second axis and including the plurality of channels along the third axis is referred to as a partial region, the feature spectrum is any one of the following feature spectra calculated for each of the plurality of partial regions included in the certain layer: (i) a first feature spectrum in which a plurality of element values of an output vector of each vector neuron included in the partial region are arranged across the plurality of channels along the third axis; (ii) a second feature spectrum obtained by multiplying each element value of the first feature spectrum by a normalization coefficient corresponding to a vector length of the output vector; and (iii) a third feature spectrum in which the normalization coefficient is arranged across the plurality of channels along the third axis.

3. The information processing method according to claim 2, characterized in that, the process (b2) includes: a process of calculating a plurality of local similarities representing a similarity with respect to each class for the plurality of partial regions of the certain layer by calculating a similarity of the feature spectrum obtained from an output of a certain partial region of the certain layer to all of the known feature spectrum groups associated with the certain layer and each class according to the discrimination data; and ​ The procedure of calculating the similarity by category is performed by taking the maximum value, the average value, or the minimum value of the plurality of local similarities with respect to the plurality of partial regions for each of the categories.

4. The information processing method according to claim 2, wherein The procedure (b2) includes: a procedure of calculating, with respect to the plurality of partial regions of the specific layer, a plurality of local similarities of similarity with respect to each category by calculating, from the discriminated data, local similarities of similarity of the feature spectrum resulting from the output of the specific partial region of the specific layer with respect to all of the known feature spectrum groups associated with the specific partial region of the specific layer and each category; and the procedure of calculating the similarity by category is performed by taking the maximum value, the average value, or the minimum value of the plurality of local similarities with respect to the plurality of partial regions for each of the categories.

5. The information processing method according to claim 2, wherein The procedure (b2) includes: a procedure of calculating the similarity by category by separately calculating, for each category, similarity of all of the feature spectra resulting from the output of the specific layer with respect to all of the known feature spectrum groups associated with the specific layer and each category from the discriminated data.

6. The information processing method according to any one of claims 1 to 5, wherein in a case where a plurality of the specific layers exist, The procedure (b3) includes: a procedure of obtaining each category discrimination result with respect to a plurality of the specific layers by calculating a category discrimination result from the similarity by category with respect to each specific layer; and a procedure of deciding, as a discrimination category of the discriminated data, a category that is most frequently represented by a plurality of the category discrimination results.

7. The information processing method according to claim 3 or 4, wherein in a case where a plurality of the specific layers exist, The procedure (b3) includes: a procedure of deciding, for each partial region of each specific layer, a category based on the local similarity with respect to the partial region, and assigning, to the partial region, a category parameter value representing an order of the category in the plurality of categories; a procedure of calculating, with respect to each of a plurality of the specific layers, a statistical variance of a distribution of the category parameter values with respect to the plurality of partial regions; and a procedure of discriminating a category of the discriminated data using the similarity by category obtained for the specific layer having the smallest variance among a plurality of the specific layers.

8. An information processing apparatus, wherein is an information processing apparatus that executes category discrimination processing of discriminating a category of discriminated data using a computer that stores a machine learning model of a vector neural network type having a plurality of layers of vector neurons, the information processing apparatus comprising: a memory that stores the machine learning model; and a processor that executes an operation using the machine learning model, the processor is configured to execute the following processing: (a) a process of reading out, from the memory, a known feature spectrum group with respect to each of a plurality of categories; and (b) a process of calculating, from the known feature spectrum group, a feature spectrum of the discriminated data. (b) performing a category discrimination process of the discriminated data using the machine learning model and the known feature spectrum group, thereby discriminating an object or discriminating a kind of a printed medium or determining whether an industrial product is a good product, the known feature spectrum group is a feature spectrum group obtained from outputs of a plurality of nodes of a certain layer among the plurality of vector neuron layers when a plurality of spectroscopic data corresponding to a predetermined category is input to the machine learning model, the process (b) includes: (b0) a process of performing spectroscopic measurement on an object or a printed medium or an industrial product to be discriminated by a spectroscopic measuring device, thereby acquiring spectroscopic data as the discriminated data; (b1) a process of calculating a feature spectrum from outputs of the plurality of nodes of the certain layer according to input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity of the feature spectrum to the known feature spectrum group with respect to each of a plurality of the categories; and (b3) a process of discriminating a category of the discriminated data according to the similarity.

9. A computer program product characterized by, including a computer program which, when executed by a processor, implements a category discrimination process of discriminating a category of discriminated data with a computer using a machine learning model in which a vector neural network type having a plurality of vector neuron layers is stored, and the computer program implements the following processes: (a) a process of reading out a known feature spectrum group from a storage with respect to each of a plurality of the categories; and (b) a process of performing a category discrimination process of the discriminated data using the machine learning model and the known feature spectrum group, thereby discriminating an object or discriminating a kind of a printed medium or determining whether an industrial product is a good product, the known feature spectrum group is a feature spectrum group obtained from outputs of a plurality of nodes of a certain layer among the plurality of vector neuron layers when a plurality of spectroscopic data corresponding to a predetermined category is input to the machine learning model, the process (b) includes: (b0) a process of performing spectroscopic measurement on an object or a printed medium or an industrial product to be discriminated by a spectroscopic measuring device, thereby acquiring spectroscopic data as the discriminated data; (b1) a process of calculating a feature spectrum from outputs of the plurality of nodes of the certain layer according to input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity of the feature spectrum to the known feature spectrum group with respect to each of a plurality of the categories; and (b3) a process of discriminating a category of the discriminated data according to the similarity. ​

Images