Method for classifying data using machine learning model, information processing device, and computer program

Abstract

To provide a technique that allows a user to learn the basis of discrimination of class discrimination using a machine learning model.SOLUTION: A class discrimination method includes: (a) a step of, when a plurality of pieces of training data is input to a machine learning model, preparing a known characteristic spectrum group obtained from output from a specific layer of a plurality of vector neuron layers for each class of one or more classes; and (b) a step of executing class discrimination processing on data to be discriminated by using the machine learning model and the known characteristic spectrum group. The step (b) includes: (b1) a step of calculating a characteristic spectrum from the output from the specific layer according to input of the data to be discriminated to the machine learning model; (b2) a step of calculating similarity between the characteristic spectrum and the known characteristic spectrum group related to each of the plurality of classes; (b3) a step of creating an explanatory text of a class discrimination result for the data to be discriminated according to the similarity; and (b4) and a step of outputting the explanatory text.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a method for determining a class of data to be classified using a machine learning model, an information processing device, and a computer program. [Background technology]

[0002] Patent Documents 1 and 2 disclose a vector neural network type machine learning model that uses vector neurons, which is called a capsule network. A vector neuron means a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons called capsules as network nodes. A vector neural network type machine learning model such as a capsule network can be used to classify input data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 5,210,798 [Patent Document 2] International Publication No. 2019 / 083553 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, when class discrimination is performed using a machine learning model, the result of the class discrimination is output, but the basis for discriminating the output class is unclear, making it difficult to know the basis for the discrimination. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a method for discriminating a class of data to be discriminated using a vector neural network type machine learning model having a plurality of vector neuron layers. This method includes the steps of (a) preparing, for each of one or more classes, a group of known feature spectra obtained from an output of a specific layer of the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, and (b) executing a class discrimination process of the data to be discriminated using the machine learning model and the group of known feature spectra. , a method for a computer to execute The step (b) includes: (b1) if the machine Ns can be set to 1 or more, it is preferable to set the machine Ns to 2 or more. The step (b) includes a step of calculating a feature spectrum from the output of the specific layer in response to input of the discriminated data to a learning model, (b2) a step of calculating a similarity between the feature spectrum and the group of known feature spectra for each class, (b3) a step of creating an explanation of the class discrimination result for the discriminated data in response to the similarity, and (b4) a step of outputting the explanation.

[0006] According to a second aspect of the present disclosure, there is provided an information processing device that executes a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers. The information processing device includes a memory that stores the machine learning model, and a processor that executes a calculation using the machine learning model. The processor is configured to execute: (a) a process of reading, for each of one or more classes, a known feature spectrum group obtained from an output of a specific layer of the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, from the memory; and (b) a process of executing a class discrimination process of the discriminated data using the machine learning model and the known feature spectrum group. The process (b) includes: (b1) a process of calculating a feature spectrum from an output of the specific layer in response to an input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity between the feature spectrum and the known feature spectrum group for each class; (b3) a process of creating an explanation of a class discrimination result for the discriminated data in response to the similarity; and (b4) a process of outputting the explanation.

[0007] According to a third aspect of the present disclosure, there is provided a computer program for causing a processor to execute a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers. This computer program causes the processor to execute the following processes: (a) a process of reading, for each of one or more classes, a known feature spectrum group obtained from an output of a specific layer of the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model; and (b) a process of executing a class discrimination process of the discriminated data using the machine learning model and the known feature spectrum group. The process (b) includes: (b1) a process of calculating a feature spectrum from an output of the specific layer in response to the input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity between the feature spectrum and the known feature spectrum group for each class; (b3) a process of creating an explanation of the class discrimination result for the discriminated data in response to the similarity; and (b4) a process of outputting the explanation. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of a class discrimination system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram of an information processing device. [Diagram 3] FIG. 1 is an explanatory diagram showing the configuration of a machine learning model. [Figure 4] FIG. 13 is an explanatory diagram showing another configuration of a machine learning model. [Diagram 5] A flowchart showing the steps for preparing a machine learning model. [Figure 6] FIG. [Figure 7] FIG. 4 is an explanatory diagram showing how a group of known characteristic spectra is created using training data. [Figure 8] FIG. 4 is an explanatory diagram showing the configuration of a group of known characteristic spectra. [Figure 9] 11 is a flowchart showing a processing procedure of a medium discrimination / printing process. [Figure 10] FIG. 4 is an explanatory diagram showing how similarity is calculated for discriminated data. [Figure 11] FIG. 4 is an explanatory diagram showing the process of creating an explanatory text in the first embodiment. [Figure 12] FIG. 4 is an explanatory diagram showing an example of an explanatory text displayed on a display unit. [Figure 13] FIG. 11 is an explanatory diagram showing a process of creating an explanatory text in the second embodiment. [Figure 14] 13 is a graph showing an infrared absorption spectrum of an object to be identified in the third embodiment. [Figure 15] FIG. 13 is an explanatory diagram showing the process of creating an explanation in the third embodiment. [Figure 16] FIG. 4 is an explanatory diagram showing a first calculation method of the similarity. [Figure 17] FIG. 11 is an explanatory diagram showing a second similarity calculation method. [Figure 18] FIG. 13 is an explanatory diagram showing a method for determining a discrimination class using multiple specific layers. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A. First embodiment: FIG. 1 is a block diagram showing a class discrimination system according to a first embodiment. The class discrimination system is a printing system including a printer 10, an information processing device 20, and a spectrometer 30. The spectrometer 30 can perform spectroscopic measurement on the printing medium PM used in the printer 10 in an unprinted state to obtain the spectral reflectance. In the present disclosure, the spectral reflectance is also referred to as "spectroscopic data." The spectrometer 30 includes, for example, a variable wavelength interference spectrometer filter and a monochrome image sensor. The spectroscopic data obtained by the spectrometer 30 is used as discriminated data to be input to a machine learning model described later. The information processing device 20 executes a class discrimination process of the spectral data using the machine learning model to discriminate which of a plurality of classes the printing medium PM belongs to. The "class of the printing medium PM" refers to the type of the printing medium PM. The information processing device 20 controls the printer 10 to execute printing under appropriate printing conditions according to the type of the printing medium PM. The class discrimination system according to the present disclosure can also be realized as a system other than a printing system, and may be realized, for example, as a system that performs class discrimination using a discriminated image, one-dimensional data other than spectroscopic data, a spectroscopic image, time series data, etc. as discriminated data.

[0010] 2 is a block diagram showing the functions of the information processing device 20. The information processing device 20 has a processor 110, a memory 120, an interface circuit 130, and an input device 140 and a display unit 150 connected to the interface circuit 130. The spectrometer 30 and the printer 10 are also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has a function of executing the processes described in detail below, but also has a function of displaying on the display unit 150 data obtained by the processes and data generated in the course of the processes.

[0011] The processor 110 functions as a print processing unit 112 that executes printing processing using the printer 10, and also functions as a class discrimination processing unit 114 that executes class discrimination processing of the spectral data of the print medium PM. The class discrimination processing unit 114 includes a similarity calculation unit 310 and an explanatory text creation unit 320. The print processing unit 112 and the class discrimination processing unit 114 are realized by the processor 110 executing a computer program stored in the memory 120. However, these units 112 and 114 may be realized by hardware circuits. The term processor in this specification includes such hardware circuits. In addition, the processor that executes the class discrimination processing may be a processor included in a remote computer connected to the information processing device 20 via a network.

[0012] The memory 120 stores the machine learning model 200, training data TD, a known feature spectrum group KSp, a print setting table PST, an explanatory text template DT, and a character string lookup table CT. The memory 120 may further store a character string generation database and a corresponding dictionary used to generate character strings. The machine learning model 200 is used for processing by the class discrimination processing unit 114. An example of the configuration and operation of the machine learning model 200 will be described later. The training data TD is a set of labeled data used for learning the machine learning model 200. In this embodiment, the training data TD is a set of spectral data. The known feature spectrum group KSp is a set of feature spectra obtained when the training data TD is input to the learned machine learning model 200. The feature spectra will be described later. The print setting table PST is a table in which print settings suitable for each printing medium are registered. The explanatory text template DT and the character string lookup table CT are used in the explanatory text creation process by the explanatory text creation unit 320. The character string lookup table CT can also be called a "character string selection unit CT."

[0013] 3 is an explanatory diagram showing the configuration of the machine learning model 200. The machine learning model 200 includes, in order from the input data IM side, a convolution layer 210, a primary vector neuron layer 220, a first convolution vector neuron layer 230, a second convolution vector neuron layer 240, and a classification vector neuron layer 250. Of these five layers 210 to 250, the convolution layer 210 is the lowest layer, and the classification vector neuron layer 250 is the highest layer. In the following description, the layers 210 to 250 are also referred to as the "Conv layer 210", the "PrimeVN layer 220", the "ConvVN1 layer 230", the "ConvVN2 layer 240", and the "ClassVN layer 250", respectively.

[0014] In this embodiment, the input data IM is spectral data, and is therefore one-dimensional array data. For example, the input data IM is data in which 36 representative values ​​are extracted at intervals of 10 nm from spectral data in the range of 380 nm to 730 nm.

[0015] 3, two convolution vector neuron layers 230 and 240 are used, but the number of convolution vector neuron layers is arbitrary, and the convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0016] The configuration of each of the layers 210 to 250 in FIG. 3 can be described as follows. <Description of the configuration of machine learning model 200> ·Conv layer 210: Conv[32,6,2] ·PrimeVN layer 220: PrimeVN[26,1,1] ·ConvVN1 layer 230:ConvVN1[20,5,2] ·ConvVN2 layer 240:ConvVN2[16,4,1] ·ClassVN layer 250:ClassVN[n1,3,1] Vector dimension VD: VD=16 In the description of each of these layers 210 to 250, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the surface size of the kernel, and the stride. For example, the layer name of the Conv layer 210 is "Conv", the number of channels is 32, the surface size of the kernel is 1 x 6, and the stride is 2. In FIG. 3, these descriptions are shown under each layer. The hatched rectangles drawn in each layer represent the surface size of the kernel used when calculating the output vector of the adjacent upper layer. In this embodiment, since the input data IM is data of a one-dimensional array, the surface size of the kernel is also one-dimensional. Note that the parameter values ​​used in the description of each of the layers 210 to 250 are merely examples and can be changed arbitrarily.

[0017] The Conv layer 210 is a layer composed of scalar neurons. The other four layers 220 to 250 are layers composed of vector neurons. A vector neuron is a neuron that takes a vector as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0018] In FIG. 3, for the Conv layer 210, a first axis x and a second axis y that define the planar coordinates of the node array, and a third axis z that represents the depth are shown. Also, it is shown that the sizes of the Conv layer 210 in the x, y, and z directions are 1, 16, and 32. The sizes in the x and y directions are called "resolutions." In this embodiment, the resolution in the x direction is always 1. The size in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes indicating the position of each node in other layers. However, in FIG. 3, the illustration of these axes x, y, and z is omitted in layers other than the Conv layer 210.

[0019] As is well known, the resolution W1 in the y direction after convolution is given by the following equation: W1 = Ceil{(W0 - Wk + 1) / S} (1) Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function that performs an operation to round up X. Note that a function that performs an operation to round down X may be used as Ceil{X}. The resolution of each layer shown in FIG. 3 is an example in which the resolution of the input data IM in the y direction is 36, and the actual resolution of each layer is changed appropriately depending on the size of the input data IM.

[0020] The ClassVN layer 250 has n1 channels. In the example of FIG. 3, n1=3. In general, n1 is an integer equal to or greater than 1, and is the number of known classes that can be discriminated using the machine learning model 200. Usually, n1 is often set to 2 or more. From the three channels of the ClassVN layer 250, judgment values ​​Class1 to Class3 for the three known classes are output. Usually, the class having the largest value among these judgment values ​​Class1 to Class3 is used as the class discrimination result of the input data IM. Also, if the largest value among the judgment values ​​Class1 to Class3 is less than a predetermined threshold, the class of the input data IM may be determined to be unknown.

[0021] In the present disclosure, as described later, instead of using the judgment values ​​Class1 to Class3 of the ClassVN layer 250, which is the output layer, it is also possible to determine the discrimination class using class-specific similarities calculated from the output of a specific vector neuron layer.

[0022] FIG. 3 further illustrates a partial region Rn in each layer 210, 220, 230, 240, 250. The subscript "n" of the partial region Rn is the code of each layer. For example, the partial region R210 indicates a partial region in the Conv layer 210. The "partial region Rn" is a region in each layer that is identified by a planar position (x, y) defined by the position of the first axis x and the position of the second axis y, and includes multiple channels along the third axis z. The partial region Rn has dimensions of "Width" x "Height" x "Depth" corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one "partial region Rn" is "1 x 1 x number of depths", that is, "1 x 1 x number of channels".

[0023] As shown in FIG. 3, a feature spectrum Sp_ConvVN1, which will be described later, is calculated from the output of the ConvVN1 layer 230 and input to the similarity calculation unit 310. Similarly, a feature spectrum Sp_ConvVN2 is calculated from the output of the ConvVN2 layer 240 and input to the similarity calculation unit 310. In another embodiment, a feature spectrum is also calculated from the output of the ClassVN layer 250 and input to the similarity calculation unit 310. The similarity calculation unit 310 calculates a similarity Sm, which will be described later, using these feature spectra Sp_ConvVN1 and Sp_ConvVN and a known feature spectrum group KSp that has been created in advance. The explanation creation unit 320 creates an explanation DS regarding the class discrimination result using this similarity Sm. As the similarity used to create the explanation, various similarities such as local similarity and class-specific similarity, which will be described later, can be used. In the embodiment described below, a method of creating an explanation using mainly local similarity will be described.

[0024] In this disclosure, the vector neuron layer used to calculate the similarity is also called the "specific layer". Any number of vector neuron layers, one or more, can be used as the specific layer. The configuration of the feature spectrum, the method of calculating the similarity using the feature spectrum, the method of creating an explanatory text using the similarity, and the method of determining the discrimination class will be described later.

[0025] Fig. 4 is an explanatory diagram showing another configuration of the machine learning model 200. This machine learning model 200 differs from the machine learning model 200 in Fig. 3, which uses input data in a one-dimensional array, in that the input data IM is data in a two-dimensional array. The configuration of each layer 210 to 250 in Fig. 4 can be described as follows. <Description of each layer configuration> ·Conv layer 210: Conv[32,5,2] ·PrimeVN layer 220: PrimeVN[16,1,1] ·ConvVN1 layer 230:ConvVN1[12,3,2] ·ConvVN2 layer 240:ConvVN2[6,3,1] ·ClassVN layer 250:ClassVN[n1,4,1] Vector dimension VD: VD=16

[0026] The machine learning model 200 shown in Fig. 4 can be used, for example, in a class discrimination system that discriminates the class of a discriminated image. However, in the following description, the machine learning model 200 shown in Fig. 3 is used.

[0027] 5 is a flowchart showing the procedure of a preparation step for a machine learning model. This preparation step is a step that is executed by, for example, the manufacturer of the printer 10.

[0028] In step S110 of FIG. 5, the class discrimination processing unit 114 executes learning of the machine learning model 200 using a plurality of training data TD. A label is assigned to each training data TD in advance. In this embodiment, it is assumed that one of labels 1 to 3 is assigned to each training data TD. These labels correspond to the three classes Class1 to Class3 of the machine learning model 200. In this disclosure, "label" and "class" mean the same thing.

[0029] When learning using the multiple pieces of training data TD is completed, the trained machine learning model 200 is stored in the memory 120. In step S120 of Fig. 5, the multiple pieces of training data TD are input again to the trained machine learning model 200 to generate a known feature spectrum group KSp. The known feature spectrum group KSp is a collection of feature spectra, which will be described below.

[0030] FIG. 6 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 200. Here, the feature spectrum Sp obtained from the output of the ConvVN1 layer 230 will be described. The horizontal axis of FIG. 6 is the position of a vector element with respect to the output vector of a plurality of nodes included in one partial region R230 of the ConvVN1 layer 230. The position of this vector element is expressed by a combination of the element number ND of the output vector in each node and the channel number NC. In this embodiment, since the vector dimension is 16, the element number ND of the output vector is 16 from 0 to 15. Also, since the number of channels in the ConvVN1 layer 230 is 20, the channel number NC is 20 from 0 to 19. In other words, this feature spectrum Sp is obtained by arranging a plurality of element values ​​of the output vector of each vector neuron included in one partial region R230 across a plurality of channels along the third axis z.

[0031] The vertical axis of Fig. 6 is the feature value C at each spectral position. V In this example, the feature value C V is the value of each element of the output vector V ND Note that the feature value C V The value of each element of the output vector V ND Alternatively, the normalization coefficient may be used as it is. In the latter case, the feature value C included in the feature spectrum Sp may be used as it is. V The number of is equal to the number of channels, which is 20. The normalization coefficient is a value corresponding to the vector length of the output vector of the node.

[0032] The number of feature spectra Sp obtained from the output of the ConvVN1 layer 230 for one piece of input data is equal to the number of planar positions (x, y) of the ConvVN1 layer 230, i.e., the number of subregions R230, and is therefore six. Similarly, three feature spectra Sp are obtained from the output of the ConvVN2 layer 240 for one piece of input data.

[0033] When the training data TD is input again to the trained machine learning model 200, the similarity calculation unit 310 calculates the feature spectrum Sp shown in FIG. 6 and registers it in the memory 120 as a known feature spectrum group KSp.

[0034] 7 is an explanatory diagram showing how a known feature spectrum group KSp is created using training data TD. In this example, training data TD with labels 1 to 3 is input to a trained machine learning model 200, and feature spectra KSp_ConvVN1 and KSp_ConvVN2 associated with the respective labels or classes are obtained from the outputs of two vector neuron layers, i.e., the ConvVN1 layer 230 and the ConvVN2 layer 240. These feature spectra KSp_ConvVN1 and KSp_ConvVN2 are stored in the memory 120 as a known feature spectrum group KSp.

[0035] Fig. 8 is an explanatory diagram showing the configuration of the known feature spectra group KSp. In this example, the known feature spectra group KSp_ConvVN2 obtained from the output of the ConvVN2 layer 240 is shown. The known feature spectra group KSp_ConvVN1 obtained from the output of the ConvVN1 layer 230 has a similar configuration, but is not shown in Fig. 8. Note that it is sufficient that the known feature spectra group KSp obtained from the output of at least one vector neuron layer is registered.

[0036] Each record of the known feature spectrum group KSp_ConvVN2 includes a parameter i indicating the order of the labels or classes, a parameter j indicating the order of the specific layers, a parameter k indicating the order of the subregions Rn, a parameter q indicating the data number, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in FIG. 6.

[0037] The parameter i of the class takes the same value as the label, 1 to 3. The parameter j of the specific layer takes the value of 1 to 2 indicating which of the two specific layers 230, 240 it is. The parameter k of the subregion Rn takes a value indicating which of the multiple subregions Rn included in each specific layer it is, i.e., which planar position (x, y) it is. For the ConvVN2 layer 240, there are three subregions R240, so k = 1 to 3. The parameter q of the data number indicates the number of the training data with the same label, and takes the value of 1 to max1 for class 1, 1 to max2 for class 2, and 1 to max3 for class 3.

[0038] The multiple teacher data TD used in step S120 do not need to be the same as the multiple teacher data TD used in step S110. However, there is an advantage in that it is not necessary to prepare new teacher data if some or all of the multiple teacher data TD used in step S110 is used in step S120 as well.

[0039] 9 is a flowchart showing the processing procedure of a medium discrimination / printing process using a trained machine learning model. This medium discrimination / printing process is executed by, for example, a user of the printer 10.

[0040] In step S210, the user instructs the class discrimination processing unit 114 as to whether or not class discrimination processing is required for the target print medium, which is the print medium to be processed. Even if the user knows the type of the target print medium, the user may instruct that class discrimination processing is required for confirmation. If class discrimination processing is not required, the process proceeds to step S270, where the user selects print settings suitable for the target print medium, and in step S280, the print processing unit 112 causes the printer 10 to print using the target print medium. On the other hand, if the type of the target print medium is unknown and class discrimination processing is required, the process proceeds to step S220.

[0041] In step S220, the class discrimination processing unit 114 acquires spectroscopic data by causing the spectrometer 30 to perform spectroscopic measurement of the target print medium. This spectroscopic data is used as discrimination target data to be input to the machine learning model 200.

[0042] In step S230, the class discrimination processing unit 114 inputs the discriminated data to the trained machine learning model 200 and calculates the feature spectrum Sp. In step S240, the similarity calculation unit 310 calculates the similarity between the feature spectrum Sp obtained in response to the input of the discriminated data and the registered known feature spectrum group KSp.

[0043] 10 is an explanatory diagram showing how similarity is calculated for discriminated data. When discriminated data is input to the machine learning model 200, the class discrimination processing unit 114 calculates feature spectra Sp_ConvVN1 and Sp_ConvVN2 from the outputs of the ConvVN1 layer 230 and the ConvVN2 layer 240. The similarity calculation unit 310 calculates similarity Sm_ConvVN1 using the feature spectrum Sp_ConvVN1 obtained from the output of the ConvVN1 layer 230 and the known feature spectrum group KSp_ConvVN1. A specific method of calculating similarity Sm will be described later. For the ConvVN2 layer 240, similarity Sm_ConvVN2 is calculated in a similar manner.

[0044] The similarity Sm can be calculated, for example, according to the following formula. Sm(i,j,k)=max[G{Sp(j,k), KSp(i,j,k=all,q=all)}] (2) Where: i is a parameter indicating the class, j is a parameter indicating a specific layer, k is a parameter indicating the subregion Rn, q is a parameter indicating the data number, G{a,b} is a function that calculates the similarity between a and b. Sp(j,k) is the feature spectrum obtained from the output of a specific subregion k of a specific layer j according to the discriminated data. KSp(i,j,k=all,q=all) is the known feature spectrum of all data numbers q in all sub-regions k of specific layer j associated with class i among the known feature spectrum group KSp shown in FIG. 8; max[X] is a logical operation that takes the maximum value of X. In the function G{a,b} for calculating similarity, a is one or a set, b is a set, and the return value is multiple. For example, a formula for calculating cosine similarity or a formula for calculating similarity according to distance can be used as the function G{a,b}.

[0045] Since this similarity Sm is obtained for each partial region, it is also called "local similarity Sm" below. Although the local similarity Sm(i,j,k) depends on the class i, the specific layer j, and the partial region k, in the following explanation, the parameter i indicating the class and the parameter j indicating the specific layer may be omitted and it may be written as "local similarity Sm(k)".

[0046] It is not necessary to generate all the similarities Sm_ConvVN1 and Sm_ConvVN2 using the two vector neuron layers 230 and 240, respectively, but it is preferable to calculate the similarity Sm using one or more of these vector neuron layers. As described above, in this disclosure, the vector neuron layer used to calculate the similarity is called a "specific layer."

[0047] In step S250, the explanation creation unit 320 creates an explanation according to the similarity obtained in step S240.

[0048] 11 is an explanatory diagram showing the process of creating an explanatory text in the first embodiment. The explanatory text creation unit 320a in the first embodiment has a gradation reduction unit 322 and a character string lookup table 324. The symbol "a" at the end of the explanatory text creation unit 320a indicates that it is for the first embodiment. The character string lookup table 324 is the same as the character string lookup table CT shown in FIG. 2.

[0049] FIG. 11 shows how a description is created using the local similarities Sm obtained from three partial regions of the ConvVN2 layer 240. As shown in the upper left of FIG. 11, the three partial regions of the ConvVN2 layer 240 correspond to three wavelength bands with central wavelengths of 300 nm, 500 nm, and 700 nm, respectively. The gradation reduction unit 322 creates three table input data D1 to D3 by reducing the number of gradations (the number of discrete values ​​that can be taken within the range between the maximum value and the minimum value) of the three local similarities Sm(k). Specifically, the gradation reduction unit 322 binarizes (quantizes) the local similarities Sm(k) using a threshold value created in advance. In this example, the local similarities Sm(k) are 16-bit data with the number of gradations being 2. 16 The table input data D1 to D3 are 1-bit data and the number of gradations is 2. However, the number of gradations of the table input data D1 to D3 may be 3 or more, and can be set to any number equal to or greater than 2.

[0050] The character string lookup table 324 outputs character strings CS1 to CS3 in response to the input of table input data D1 to D3. Examples of the character strings CS1 to CS3 corresponding to combinations of the table input data D1 to D3 are shown at the lower left of FIG. 11. The description creation unit 320 creates a description DSa by applying these three character strings CS1 to CS3 to a description template DTa having three character string frames corresponding thereto. In the example of FIG. 11, the local similarity Sm in two wavelength bands of 500 nm and 700 nm is equal to or greater than the threshold value, but the local similarity Sm in the wavelength band of 300 nm is less than the threshold value, so an appropriate description DSa corresponding thereto is created. Thus, in the present embodiment, the description DSa corresponding to the similarity Sm can be created using the character string lookup table 322 and the description template DTa. The description DSa created in this way is displayed on the display unit 150.

[0051] The various numbers used in FIG. 11 are as follows. (1) The number Nk of partial regions included in the specific layer In the example of FIG. 11, Nk = 3. (2) The number Ns of local similarities Sm used for creating the description In the example of FIG. 11, Ns = Nk = 3, but Ns can be set to any number less than or equal to Nk. Ns can be set to 1 or more, but it is preferably 2 or more. An example of Ns < Nk will be described in the third embodiment. (3) The number Nd of table input data Dk In the example of FIG. 11, Nd = Ns = 3, but Nd can be set to any number from 1 to Ns. Nd can be set to 1 or more, but it is preferably 2 or more. An example of Nd < Ns will be described in the second embodiment. (4) The number Nc of character strings output from the character string lookup table 324 In the example of FIG. 11, Nc = 3, but Nc can be set to any number of 1 or more.

[0052] FIG. 12 shows an example of an explanatory text displayed on the display unit 150. In this example, a discrimination result list DL in which class discrimination results and explanatory texts are arranged for a plurality of classes is displayed on the display unit 150. As the class discrimination result, a parameter i indicating the class, a class name, and a class probability which is the probability that the discriminated data corresponds to the class are displayed. For example, a judgment value output from the output layer of the machine learning model 200 can be used as the class probability. In addition, when class discrimination is performed using a class-specific similarity to be described later, the class-specific similarity may be used as the class probability. Note that a part of the class discrimination result may be omitted. In addition, an activation value in the ClassVN layer 250 of the machine learning model 200 may be used as the class probability, and an explanatory text may be created from the activation value. Specifically, for example, it is possible to create an explanatory text such as "Activation=0.99, so the reliability of the discrimination result is high." Furthermore, a similarity calculated from the output of the ClassVN layer 250 may be used as the class probability, and an explanatory text may be created from the similarity. Specifically, for example, it is possible to create an explanation such as "ClassVN similarity = 0.99, so it is known data."

[0053] 12, if a discrimination result list DL in which class discrimination results and explanatory text are arranged for a plurality of classes is displayed on the display unit 150, the user can know the basis of the class discrimination result for each class. Note that the discrimination result list DL does not need to include information on all of the plurality of classes that can be discriminated by the machine learning model 200, and it is preferable that the discrimination result list DL includes information on at least two or more classes. Moreover, instead of displaying the discrimination result list DL, an explanatory text for one class discriminated by the machine learning model 200 may be displayed.

[0054] In step S260, the user refers to the description created in step S250 and selects the class of the target print medium, i.e., the type of the target print medium, and indicates the selected type to the print processing unit 112. In step S270, the print processing unit 112 refers to the print setting table PST according to the type of the target print medium and selects print settings. In step S280, the print processing unit 112 executes printing according to the print settings. According to the procedure in FIG. 9, even if the type of the target print medium is unknown, the type of the target print medium can be determined using the machine learning model 200, so that printing can be executed using print settings suitable for that type.

[0055] As described above, in the first embodiment, the similarity regarding the feature vectors is used to create and output an explanation of the class discrimination result, so that the user can know the basis of the class discrimination result.

[0056] B. Second embodiment: Fig. 13 is an explanatory diagram showing the process of creating an explanatory text in the second embodiment. An explanatory text creating unit 320b in the second embodiment has an average value calculating unit 321 in addition to a gradation reducing unit 322 and a character string lookup table 324. The configuration of the device shown in Figs. 1 and 2 and the process steps shown in Figs. 5 and 9 are almost the same as those in the first embodiment.

[0057] FIG. 13 shows how an explanatory text is created using the local similarities Sm obtained from six partial regions of the ConvVN1 layer 230. The average calculation unit 321 groups the six local similarities Sm into three groups, and calculates three average similarities Sma1 to Sma3 by averaging the local similarities Sm belonging to each group. The number of average similarities is the same as the number Nd of table input data Dk. These three average similarities Sma1 to Sma3 are similarities for three wavelength bands with central wavelengths of 300 nm, 500 nm, and 700 nm, respectively, like the three local similarities Sm shown in FIG. 11 in the first embodiment. The gradation reduction unit 322 creates table input data Dk by reducing the number of gradations of these average similarities Sma1 to Sma3. The subsequent processing is the same as that of the first embodiment shown in FIG. 11, so detailed description will be omitted. In the example of FIG. 13, an explanatory text DSb is created that reads, "Since all components match, it is determined to be known."

[0058] In the example of Fig. 13 described above, the average similarity Sma is created by averaging the local similarities Sm for two adjacent partial regions. However, in general, the average similarity Sma can be calculated by grouping the Ns local similarities Sm used to create the description into Nd groups and averaging the local similarities Sm for each group. In this case, Nd is a smaller value than Ns.

[0059] As described above, in the second embodiment, Nd average similarities are calculated from Ns local similarities, and Nd pieces of table input data are created by reducing the number of gradations of the Nd average similarities, so that a description can be created based on the average similarity obtained by averaging the local similarities.

[0060] In the above description, the average similarity Sma is calculated for each group of local similarities, but a representative value other than the average similarity may be calculated. As the representative value, a maximum value or a minimum value may be used in addition to the average value. In other words, in the second embodiment, Nd representative similarities may be calculated from Ns local similarities, and Nd table input data may be created by reducing the number of gradations of the Nd representative similarities. In this way, a description can be created according to the representative similarity.

[0061] C. Third embodiment: FIG. 14 is an explanatory diagram showing infrared absorption spectra of 2-hexanone and acrylonitrile as objects to be discriminated in the class discrimination process in the third embodiment. The horizontal axis of each graph is wave number, and the vertical axis is absorbance. In the third embodiment, it is assumed that the object to be discriminated is either 2-hexanone or acrylonitrile, and it is determined whether the object to be discriminated is 2-hexanone or not. In this case, the number of classes in the output layer of the machine learning model 200 shown in FIG. 3 is 1.

[0062] The infrared absorption spectrum of 2-hexanone has a peak at 2900 cm -1 The wavenumber band includes an absorption peak due to C-H bonds at 1700 cm -1 The infrared absorption spectrum of acrylonitrile contains an absorption peak due to the C=O bond at 2900 cm -1 The wavenumber band includes an absorption peak due to C-H bonds at 2300 cm -1 The wavenumber band includes an absorption peak due to C-N bonds. Each graph shows the relationship between the parameter k that distinguishes the six partial regions of the ConvVN1 layer 230 and the wavenumber band. As described below, in the third embodiment, a description is created using a part of the local similarity Sm of the six partial regions.

[0063] FIG. 15 is an explanatory diagram showing the process of creating an explanatory text in the third embodiment. The explanatory text creating unit 320c in the third embodiment is the same as that in the first embodiment shown in FIG. 11 in that it has a gradation reducing unit 322 and a character string lookup table 324. However, the character strings stored in the character string lookup table 324 are different from those in the first embodiment. The information processing device 20 shown in FIG. 2 can be used in the third embodiment, except that the printer 10 and the print processing unit 112 are not used. In addition, the procedure of the process shown in FIG. 5 can be almost the same as that in the first embodiment. Furthermore, in the process shown in FIG. 9, the "target print medium" is replaced with the "object to be identified" and the process of steps S250 to S270 is executed, thereby making it possible to perform class discrimination in the third embodiment.

[0064] In the example of Fig. 15, an explanatory text is created using three local similarities Sm(k=1), Sm(k=3), and Sm(k=5) that are pre-selected from the local similarities Sm obtained from the six partial regions of the ConvVN1 layer 230. These three local similarities Sm(k=1), Sm(k=3), and Sm(k=5) are the 1700 cm -1 Wavenumber range and 2300cm -1 Wavenumber range and 2900cm -1 The number of selected local similarities Sm is the same as the number Nd of table input data Dk.

[0065] As in the first embodiment, the gradation reduction unit 322 creates table input data Dk by reducing the number of gradations of the local similarity Sm. The character string lookup table 324 outputs three character strings CS1 to CS3 in response to the input of these table input data D1 to D3. The explanation creation unit 320 creates an explanation DSc by applying these three character strings CS1 to CS3 to an explanation template DTc having three character string frames corresponding to these. As can be seen by comparing FIG. 11 with FIG. 15, the registered contents of the character string lookup table 324 and the contents of the explanation template DTc are different from those in the first embodiment. In the example of FIG. 15, all of the three local similarities Sm used to create the explanation are equal to or greater than the threshold value, so an appropriate explanation DSc is created to indicate that the discrimination target corresponds to 2-hexanone.

[0066] Note that an explanation may be created using a part of the total of 12 individual similarities, 6 individual similarities calculated for hexanone and 6 individual similarities calculated for acrylonitrile. For example, an explanation such as "Since there is C=O and no CN, it was determined to be hexanone" or "Since there is no C=O and no CN, it was determined to be acrylonitrile" may be created.

[0067] As described above, in the third embodiment, table input data is created from local similarities for some of the partial regions among the multiple partial regions included in a specific layer, so that an explanation can be created using local similarities suitable for explaining the discrimination result.

[0068] In the various embodiments described above, a character string lookup table is used to create character strings. However, instead of a character string lookup table, a decision tree may be used to create character strings and explanatory text.

[0069] D. Similarity calculation method: As a method for calculating the above-mentioned local similarity Sm, for example, one of the following two methods can be adopted. (1) A first calculation method M1 for calculating the local similarity Sm without considering the correspondence between the feature spectrum Sp and the partial region Rn in the known feature spectrum group KSp (2) A second calculation method M2 for calculating the local similarity Sm between the corresponding partial regions Rn of the feature spectrum Sp and the known feature spectrum group KSp Below, methods for calculating the similarity from the output of the ConvVN1 layer 230 according to these two calculation methods M1 and M2 will be described in order.

[0070] 16 is an explanatory diagram showing a first similarity calculation method M1. Here, a case is shown in which a local similarity Sm(i,j,k) indicating the similarity to each class i for each partial region k is calculated from the output of the ConvVN1 layer 230, which is a specific layer.

[0071] In the first calculation method M1, the local similarity Sm(i,j,k) is calculated using the formula (2) shown below. Sm(i,j,k)=max[G{Sp(j,k), KSp(i,j,k=all,q=all)}] (2) Where: i is a parameter indicating the class, j is a parameter indicating a specific layer, k is a parameter indicating the subregion Rn, q is a parameter indicating the data number, G{a,b} is a function that calculates the similarity between a and b. Sp(j,k) is the feature spectrum obtained from the output of a specific subregion k of a specific layer j according to the discriminated data. KSp(i,j,k=all,q=all) is the known feature spectrum of all data numbers q in all sub-regions k of specific layer j associated with class i among the known feature spectrum group KSp shown in FIG. 8; max[X] is a logical operation that takes the maximum value of X. In the function G{a,b} for calculating similarity, a is one or a set, b is a set, and the return value is multiple. For example, a formula for calculating cosine similarity or a formula for calculating similarity according to distance can be used as the function G{a,b}.

[0072] The right side of FIG. 16 shows how class-specific similarities Sclass(i,j) are calculated from these local similarities Sm(i,j,k). The three types of class-specific similarities Sclass(i,j) are obtained by taking the maximum value, average value, or minimum value of the local similarities Sm(i,j,k) for multiple partial regions k for each class i. Which of the maximum value, average value, or minimum value is used depends on the purpose of the class discrimination process. For example, when the purpose is to discriminate objects using natural images, it is preferable to obtain class-specific similarities Sclass(i,j) by taking the maximum value of the local similarities Sm(i,j,k) for each class i. Also, when the purpose is to discriminate the type of print medium or to judge the quality of industrial products using images, it is preferable to obtain class-specific similarities Sclass(i,j) by taking the minimum value of the local similarities Sm(i,j,k) for each class i. In addition, it may be preferable to calculate the class-specific similarity Sclass(i,j) by averaging the local similarities Sm(i,j,k) for each class i. Which of these three types of calculations to use is preset by the user experimentally or empirically.

[0073] In the example of Fig. 16, furthermore, a final discrimination result RD_ConvVN1 is determined from the class-specific similarity Sclass(i,j). The discrimination result RD_ConvVN1 can be expressed in a format including a discrimination class D_class and a similarity value S_value corresponding to the discrimination class D_class. The similarity value S_value is obtained by taking the maximum value of the similarity values ​​for the three classes 1 to 3 in the class-specific similarity Sclass(i,j). The discrimination class D_class is the class with the maximum similarity value in the class-specific similarity Sclass(i,j).

[0074] As described above, in the first similarity calculation method M1, (1) Calculating a local similarity Sm(i,j,k) between a feature spectrum Sp obtained from an output of a specific subregion k of a specific layer j and all known feature spectra Ksp associated with the specific layer j and each class i according to the data to be discriminated; (2) For each class i, a class-specific similarity Sclass(i,j) is calculated by taking the maximum, average, or minimum of the local similarities Sm(i,j,k) for multiple subregions k; (3) The maximum value of the class-specific similarity Sclass(i,j) for multiple classes i is calculated as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp. (4) The class associated with the maximum similarity value S_value across multiple classes is determined as the discrimination class D_class. According to the first calculation method M1, the similarity Sm(i,j,k), Sclass(i,j) and the discrimination result can be obtained by relatively simple calculations and procedures.

[0075] As the discrimination result using the machine learning model 200, a discrimination class D_class determined according to the class-specific similarity Sclass(i,j) may be used, or a discrimination class determined from a judgment value obtained from the output layer of the machine learning model 200 may be used. In the latter case, the process after the calculation of the class-specific similarity Sclass(i,j) may be omitted. The same applies to the second calculation method M2 described below.

[0076] The explanatory text creating unit 320 may create explanatory text on the discrimination result according to the class similarity Sclass(i,j). For example, the explanatory text according to the class similarity Sclass(i,j) may be "Since the similarity with class 1 is 98%, it is determined to be known."

[0077] 17 is an explanatory diagram showing a second similarity calculation method M2. In the second similarity calculation method M2, the following equation is used instead of the above-mentioned equation (2) to calculate the local similarity Sm(i,j,k). Sm(i,j,k)=max[G{Sp(j,k), KSp(i,j,k,q=all)}] (3) Where: KSp(i,j,k,q=all) is the known feature spectrum of all data numbers q in a specific partial region k of a specific layer j associated with class i, among the known feature spectrum group KSp shown in FIG.

[0078] While the above-mentioned first calculation method M1 uses known feature spectra KSp(i,j,k=all,q=all) in all partial regions k of a specific layer j, the second calculation method M2 uses only known feature spectra KSp(i,j,k,q=all) for the same partial region k as the feature spectrum Sp(j,k). The other methods in the second calculation method M2 are the same as those in the first calculation method M1.

[0079] In the second calculation method M2 of the class similarity, (1) Calculating a local similarity Sm(i,j,k) between a feature spectrum Sp obtained from an output of a specific sub-region k of a specific layer j and all known feature spectra Ksp associated with the specific sub-region k of the specific layer j and each class i according to the data to be discriminated; (2) For each class i, a class-specific similarity Sclass(i,j) is calculated by taking the maximum, average, or minimum of the local similarities Sm(i,j,k) for multiple subregions k; (3) The maximum value of the class-specific similarity Sclass(i,j) for multiple classes i is calculated as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp. (4) The class associated with the maximum similarity value S_value across multiple classes is determined as the discrimination class D_class. This second calculation method M2 also makes it possible to obtain the similarity Sm(i,j,k), Sclass(i,j) and the discrimination result through relatively simple calculations and procedures.

[0080] The above-mentioned two calculation methods M1 to M2 are both methods for calculating local similarity and class-specific similarity for each specific layer i to determine a discrimination class. As described above, in the present disclosure, the local similarity and class-specific similarity are calculated using one or more of the multiple vector neuron layers 230 and 240 shown in FIG. 3 as specific layers, and the class of the discriminated data can be determined from the class-specific similarity. When multiple specific layers are used, for example, the following discrimination class determination method can be adopted.

[0081] FIG. 18 is an explanatory diagram showing a method of determining a discrimination class using a plurality of specific layers. In this determination method, a discrimination class is determined using a specific layer that shows the most statistically significant judgment result among a plurality of specific layers. In the example of FIG. 18, the ConvVN1 layer 230 and the ConvVN2 layer 240 are used as specific layers. First, for each partial region k of the ConvVN1 layer 230, a class in which the local similarity Sm(i,j,k) is the maximum value is determined, and a process is executed to assign the class parameter value i of the class to each partial region k. Note that the class parameter value i is a value indicating the order among the plurality of classes. The class parameter value i is, for example, a consecutive integer. In this embodiment, the class parameter value i and the class i are the same. Similarly, for the ConvVN2 layer 240, a class in which the local similarity Sm(i,j,k) is the maximum value is determined, and a process is executed to assign the class parameter value i of the class to each partial region k.

[0082] Furthermore, in the second determination method MM2, if there is no difference in the local similarity S between classes for each partial region k, that is, if the error or variance across multiple classes for the local similarity S of a certain partial region k is within a threshold, a class parameter value may not be assigned to that partial region k. When calculating the variance of the class parameter values, the variance is calculated excluding partial regions k to which no class parameter value is assigned. This allows the variance to be calculated only for characteristic parts, making it possible to perform class discrimination with greater accuracy.

[0083] In this determination method, furthermore, in each specific layer, variance is calculated with respect to the distribution of the class parameter value i in a plurality of partial regions k. This variance is a statistical variance value for the class parameter value i. In the example of FIG. 18, the variance of the ConvVN1 layer 230 is 0.14, and the variance of the ConvVN2 layer 240 is 0.22. Since it is expected that the judgment result is clearer as the distribution of the class parameter value i becomes more biased among these specific layers 230, 240, the class discrimination result of the specific layer with the lower variance is adopted. In other words, the class of the discriminated data is discriminated using the class-based similarity obtained for the specific layer with the smallest variance among the multiple specific layers. According to this determination method, the class discrimination can be performed more accurately using the multiple specific layers. In this way, when the class discrimination result is determined using the class-based similarity obtained from the multiple specific layers, it is preferable to use the local similarity of the specific layer in which the class discrimination result was obtained as the local similarity used for creating the explanatory text in the above-mentioned first to third embodiments.

[0084] E. How the output vectors of each layer of the machine learning model are calculated: The method of calculating the output of each layer in the machine learning model 200 shown in Fig. 3 is as follows. The machine learning model 200 shown in Fig. 4 is the same except for the values ​​of the individual parameters.

[0085] Each node of the PrimeVN layer 220 regards the scalar output of the 1×1×32 nodes of the Conv layer 210 as a 32-dimensional vector, and obtains the vector output of the node by multiplying this vector by a transformation matrix. This transformation matrix is ​​an element of a kernel with a surface size of 1×1, and is updated by learning of the machine learning model 200. It is also possible to integrate the processing of the Conv layer 210 and the PrimeVN layer 220 and configure them as one primary vector neuron layer.

[0086] When the PrimeVN layer 220 is referred to as the "lower layer L" and the ConvVN1 layer 230 adjacent to it on the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following formula.

number

[0087] As the normalization function F(X), for example, the following formula (E3a) or (E3b) can be used.

number

[0088] In the above formula (E3a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function j On the other hand, in equation (E3b), the sum vector u j Norm of |u j | is the norm |u j Activation value a by dividing by the sum of | j It should be noted that the normalization function F(X) may be a function other than equation (E3a) or (E3b).

[0089] The ordinal number i in the above equation (E2) is the output vector M L+1 j The integer n is a number that is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine n = Nk × Nc (E5) Here, Nk is the surface size of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 220. In the example of FIG.

[0090] A kernel used to calculate the output vector of the ConvVN1 layer 230 has 1×5×26=130 elements, with the kernel size 1×5 as the surface size and the number of channels in the lower layer being 26 as the depth. Each of these elements is a prediction matrix W L ij In addition, 20 sets of this kernel are required to generate output vectors for the 20 channels of the ConvVN1 layer 230. Therefore, the prediction matrix W L ij The number of prediction matrices W is 130 × 20 = 2600. L ij is updated by learning of the machine learning model 200.

[0091] As can be seen from the above equations (E1) to (E4), the output vector M L+1 j is calculated by the following calculation: (a) The output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is the sum, i.e., linear combination, of j Seeking (c) Sum vector u j Norm of |u j The activation value a, which is a normalization factor by normalizing | j Seeking (d) Sum vector u j The norm |u j Divide by | and then the activation value a j Multiply by.

[0092] In addition, the activation value a j is the norm |u jis the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in (E3), (E3a), (E3b), and (4) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of j is only used in the above equations (E3) and (E4), and does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 to output to the outside.

[0093] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above formulas (E1) to (E4) used in a vector neural network are different from those used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v ij Each of the vector neural networks in this embodiment is multiplied by a weight, and the weight is searched for by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M L+1 j Since the above is obtained, there is no need to repeat dynamic routing, and the calculation speed is faster. In addition, the vector neural network of this embodiment has the advantage that the amount of memory required for calculation is smaller than that of a capsule network, and according to an experiment by the inventor of the present disclosure, the amount of memory required is about 1 / 2 to 1 / 3.

[0094] A vector neural network is the same as a capsule network in that it uses nodes that use vectors as input and output. Therefore, the advantage of using vector neurons is also common to capsule networks. In addition, the multiple layers 210-250 are the same as normal convolutional neural networks in that the higher the layer, the larger the area of ​​the feature represented, and the lower the layer, the smaller the area of ​​the feature represented. Here, "feature" means a characteristic part included in the input data to the neural network. In vector neural networks and capsule networks, the output vector of a certain node contains spatial information that represents the spatial information of the feature represented by the node, making them superior to normal convolutional neural networks. That is, the vector length of the output vector of a certain node represents the probability of the existence of the feature represented by the node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the positional relationship of each feature. Alternatively, it can be said that the vector direction of the output vectors of the two nodes represents the variation of the feature. For example, if a node corresponds to the feature of "eyes", the direction of the output vector can represent variations such as the thinness of the eyes and the way they are lifted. In a conventional convolutional neural network, it is said that spatial information of features is lost due to pooling processing. As a result, vector neural networks and capsule networks have the advantage of being superior in the performance of identifying input data compared to conventional convolutional neural networks.

[0095] The advantages of vector neural networks can also be considered as follows. That is, the advantage of vector neural networks is that the output vectors of nodes express the features of the input data as coordinates in a continuous space. Therefore, output vectors can be evaluated such that if the vector directions are close, the features are similar. Another advantage is that even if the features contained in the input data are not covered by the training data, the features can be determined by interpolation. On the other hand, normal convolutional neural networks have the disadvantage that the features of the input data cannot be expressed as coordinates in a continuous space because of the chaotic compression caused by the pooling process.

[0096] The output of each node in the ConvVN2 layer 240 and the ClassVN layer 250 is determined similarly using the above-mentioned formulas (E1) to (E4), and therefore a detailed description thereof is omitted. The resolution of the top layer, the ClassVN layer 250, is 1×1, and the number of channels is n1.

[0097] The output of the ClassVN layer 250 is converted into a plurality of decision values ​​Class0 to Class2 for known classes. These decision values ​​are usually normalized by a softmax function. Specifically, for example, a decision value for each class can be obtained by performing an operation in which the vector length of the output vector of each node of the ClassVN layer 250 is calculated from the output vector of that output vector, and the vector length of each node is further normalized by a softmax function. As described above, the activation value a obtained by the above formula (E3) is j is the output vector M L+1 j The activation value a at each node in the ClassVN layer 250 is a value corresponding to the vector length of j may be output and used as the judgment value for each class.

[0098] In the above-described embodiment, a vector neural network that determines an output vector by calculating the above equations (E1) to (E4) was used as the machine learning model 200. However, instead of this, a capsule network disclosed in U.S. Pat. No. 5,210,798 or WO 2009 / 083553 may be used.

[0099] Other embodiments: The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various forms without departing from the spirit of the present disclosure. For example, the present disclosure can be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

[0100] (1) According to a first embodiment of the present disclosure, there is provided a method for discriminating a class of data to be discriminated using a vector neural network type machine learning model having a plurality of vector neuron layers. The method includes: (a) preparing, for each of one or more classes, a known feature spectrum group obtained from an output of a specific layer among the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model; and (b) executing a class discrimination process of the data to be discriminated using the machine learning model and the known feature spectrum group. The step (b) includes: (b1) calculating a feature spectrum from an output of the specific layer in response to input of the data to be discriminated to the machine learning model; (b2) calculating a similarity between the feature spectrum and the known feature spectrum group for each class; (b3) creating an explanation of the class discrimination result for the data to be discriminated in response to the similarity; and (b4) outputting the explanation. According to this method, the similarity regarding the feature vectors is used to create and output an explanation of the class discrimination result, so that the user can know the basis of the class discrimination result.

[0101] (2) In the above method, the specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, and a region in the specific layer, which is specified at a planar position defined by the position of the first axis and the position of the second axis, and includes the plurality of channels along the third axis, is called a partial region. In this case, the feature spectrum may be obtained for each partial region among the plurality of partial regions included in the specific layer as any one of (i) a first type feature spectrum in which a plurality of element values ​​of an output vector of each vector neuron included in the partial region are arranged across the plurality of channels along the third axis, (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by a normalization coefficient corresponding to the vector length of the output vector, and (iii) a third type feature spectrum in which the normalization coefficient is arranged across the plurality of channels along the third axis. According to this method, the similarity can be calculated using any one of three types of feature spectra obtained from the output vector of a specific layer.

[0102] (3) In the above method, the similarity obtained in the step (b2) may be a local similarity obtained for each of the partial regions. According to this method, a description can be created according to the local similarity obtained for each partial region of a specific layer.

[0103] (4) In the above method, when Ns and Nd are each an integer of 2 or greater, Nd≦Ns, and Nc is an integer of 1 or greater, step (b3) may include a first step of creating Nd pieces of table input data having a smaller number of gradations than the number of local similarities from Ns local similarities for at least a portion of Ns partial regions among the plurality of partial regions included in the specific layer, a second step of inputting the Nd pieces of table input data into a string lookup table prepared in advance to obtain Nc strings to be output from the string lookup table, and a third step of creating the explanatory text by applying the Nc strings to an explanatory text template having Nc string frames. According to this method, the explanation can be created using a string lookup table and an explanation template.

[0104] (5) In the above method, the integer Nd may be smaller than the integer Ns, and the first step may include a step of finding Nd representative similarities by grouping the Ns local similarities into Nd groups and finding a representative value of the local similarities for each group, and a step of creating the Nd table input data by reducing the number of gradations of the Nd representative similarities. According to this method, a description can be created according to a representative similarity obtained by calculating a representative value of the local similarities of partial regions.

[0105] (6) In the above method, the local similarity for each partial region may be calculated as either a first type of local similarity which is a similarity between the feature spectrum obtained from an output of the partial region of the specific layer and all of the known feature spectra associated with the specific layer and each class, depending on the discriminated data, or a second type of local similarity which is a similarity between the feature spectrum obtained from an output of the partial region of the specific layer and all of the known feature spectra associated with the partial region of the specific layer and each class, depending on the discriminated data. According to this method, the local similarity can be found by a relatively simple calculation.

[0106] (7) In the above method, the step (b4) may include a step of displaying a discrimination result list in which the class discrimination results and the explanatory text are arranged for two or more classes among a plurality of classes that can be discriminated by the machine learning model. This method makes it possible to know the basis for classification results for two or more classes.

[0107] (8) According to a second embodiment of the present disclosure, there is provided an information processing device that executes a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers. The information processing device includes a memory that stores the machine learning model, and a processor that executes a calculation using the machine learning model. The processor is configured to execute: (a) a process of reading, for each of one or more classes, a known feature spectrum group obtained from an output of a specific layer of the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, from the memory; and (b) a process of executing a class discrimination process of the discriminated data using the machine learning model and the known feature spectrum group. The process (b) includes: (b1) a process of calculating a feature spectrum from an output of the specific layer in response to an input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity between the feature spectrum and the known feature spectrum group for each class; (b3) a process of creating an explanation of the class discrimination result for the discriminated data in response to the similarity; and (b4) a process of outputting the explanation. According to this information processing device, a description of the class discrimination result is created and output by utilizing the similarity regarding the feature vector, so that the user can know the basis of the class discrimination result.

[0108] (9) According to a third aspect of the present disclosure, there is provided a computer program for causing a processor to execute a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers. This computer program causes the processor to execute the following processes: (a) a process of reading, for each of one or more classes, a known feature spectrum group obtained from an output of a specific layer of the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model; and (b) a process of executing a class discrimination process of the discriminated data using the machine learning model and the known feature spectrum group. The process (b) includes: (b1) a process of calculating a feature spectrum from an output of the specific layer in response to the input of the discriminated data to the machine learning model; (b2) a process of calculating a similarity between the feature spectrum and the known feature spectrum group for each class; (b3) a process of creating an explanation of the class discrimination result for the discriminated data in response to the similarity; and (b4) a process of outputting the explanation. According to this computer program, a description of the class discrimination result is created and output using the similarity regarding the feature vectors, so that the user can know the basis of the class discrimination result.

[0109] The present disclosure may be realized in various forms other than those described above, for example, in the form of a computer program for implementing the functions of a classifier, a non-transitory storage medium on which the computer program is recorded, etc. [Explanation of symbols]

[0110] 10... printer, 20... information processing device, 30... spectrometer, 110... processor, 112... print processing unit, 114... class discrimination processing unit, 120... memory, 130... interface circuit, 150... display unit, 200... machine learning model, 210... convolution layer, 220... primary vector neuron layer, 230... first convolution vector neuron layer, 240... second convolution vector neuron layer, 250... classification vector neuron layer, 310... similarity calculation unit, 320... explanatory text creation unit, 321... average value calculation unit, 322... gradation reduction unit, 324... character string lookup table

Claims

1. A method for classifying a class of classified data using a vector neural network type machine learning model having a plurality of vector neuron layers, comprising: (a) preparing a group of known feature spectra obtained from an output of a specific layer among the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, for each of one or more classes; (b) performing a class discrimination process on the data to be discriminated using the machine learning model and the group of known characteristic spectra; Including, The step (b) (b1) calculating a feature spectrum from an output of the specific layer in response to input of the discriminated data to the machine learning model; (b2) calculating a similarity between the feature spectrum and the group of known feature spectra for each class; (b3) creating an explanation of the class discrimination result for the discrimination target data according to the similarity; (b4) outputting the explanatory text; A computer-implemented method comprising:

2. 2. The method of claim 1 , The specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, When a region in the specific layer is identified at a planar position defined by the position of the first axis and the position of the second axis and includes the plurality of channels along the third axis, the region is called a partial region. The characteristic spectrum is, for each of the plurality of partial regions included in the specific layer, (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of each vector neuron included in the partial region are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by a normalization coefficient corresponding to the vector length of the output vector; and (iii) a third type of feature spectrum in which the normalization coefficients are arranged across the plurality of channels along the third axis; and The method is as follows:

3. 3. The method of claim 2, The method, wherein the similarity obtained in the step (b2) is a local similarity obtained for each of the partial regions.

4. 4. The method of claim 3, When Ns and Nd are integers of 2 or more, Nd≦Ns, and Nc is an integer of 1 or more, The step (b3) a first step of generating Nd pieces of table input data having a smaller number of gradations than the Ns local similarities for at least a portion of the Ns partial regions among the plurality of partial regions included in the specific layer; a second step of inputting the Nd table input data into a character string lookup table prepared in advance, thereby obtaining Nc character strings output from the character string lookup table; a third step of creating the explanatory text by applying the Nc character strings to an explanatory text template having Nc character string frames; A method comprising:

5. 5. The method of claim 4, The integer Nd is smaller than the integer Ns, The first step comprises: A step of finding Nd representative similarities by grouping the Ns local similarities into Nd groups and finding a representative value of the local similarities for each group; creating the Nd pieces of table input data by reducing the number of gradations of the Nd pieces of representative similarities; The method includes:

6. The method according to any one of claims 3 to 5, The local similarity for each subregion is a first local similarity between the feature spectrum obtained from the output of the partial region of the specific layer and all of the known feature spectra associated with the specific layer and each class according to the discriminated data; a second type of local similarity, which is a similarity between the feature spectrum obtained from the output of the partial region of the specific layer and all of the known feature spectra associated with the partial region of the specific layer and each class according to the discriminated data; A method of calculating the above.

7. The method according to any one of claims 1 to 6, The step (b4) A method comprising a step of displaying a discrimination result list in which the class discrimination results and the explanatory text are arranged for two or more classes among a plurality of classes that can be discriminated by the machine learning model.

8. An information processing device that executes a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers, A memory that stores the machine learning model; A processor that executes a calculation using the machine learning model; Equipped with The processor, (a) reading from the memory a group of known feature spectra obtained from an output of a specific layer among the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, for each of one or more classes; (b) performing a class discrimination process on the data to be discriminated using the machine learning model and the group of known characteristic spectra; is configured to run The process (b) (b1) calculating a feature spectrum from an output of the specific layer in response to input of the discriminated data to the machine learning model; (b2) calculating a similarity between the feature spectrum and the group of known feature spectra for each class; (b3) a process of creating an explanation of the class discrimination result for the discriminated data according to the similarity; (b4) outputting the explanatory text; and An information processing device comprising:

9. A computer program that causes a processor to execute a class discrimination process for discriminating a class of discriminated data using a vector neural network type machine learning model having a plurality of vector neuron layers, (a) reading from a memory a group of known feature spectra obtained from an output of a specific layer among the plurality of vector neuron layers when a plurality of teacher data are input to the machine learning model, for each of one or more classes; (b) performing a class discrimination process on the data to be discriminated using the machine learning model and the group of known characteristic spectra; A computer program for causing the processor to execute the The process (b) (b1) calculating a feature spectrum from an output of the specific layer in response to input of the discriminated data to the machine learning model; (b2) calculating a similarity between the feature spectrum and the group of known feature spectra for each class; (b3) a process of creating an explanation of the class discrimination result for the discriminated data according to the similarity; (b4) outputting the explanatory text; and A computer program comprising:

Images