Information processing method, information processing apparatus, and computer program

Abstract

The present disclosure relates to an information processing method, an information processing apparatus, and a computer program. Provided is a technology capable of knowing a basis for discrimination of category discrimination using a machine learning model. A category discrimination method includes the following processes: (a) preparing, for each of one or more categories, a known feature spectrum group obtained from an output of a specific layer among a plurality of vector neuron layers when a plurality of teaching data is input to a machine learning model; and (b) performing category discrimination processing of discrimination data using the machine learning model and the known feature spectrum group. Process (b) includes the following processes: (b1) calculating a feature spectrum from the output of the specific layer in accordance with input of the discrimination data to the machine learning model; (b2) calculating a similarity between the feature spectrum and the known feature spectrum group respectively associated with the plurality of categories; (b3) creating an explanatory sentence for a category discrimination result of the discrimination data in accordance with the similarity; and (b4) outputting the explanatory sentence.

Description

Technical Field

[0001] This disclosure relates to methods, information processing apparatus, and computer programs for using machine learning models to determine the category of classified data. Background Technology

[0002] Patent documents 1 and 2 disclose models called capsule networks as vector neural network-type machine learning models using vector neurons. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons called capsules as nodes in a network. Capsule networks and other vector neural network-type machine learning models can be used for class discrimination of input data.

[0003] Patent Document 1: U.S. Patent No. 5,210,798

[0004] Patent Document 2: International Publication No. 2019 / 083553

[0005] However, in existing technologies, when class discrimination is performed using machine learning models, although the classification results are output, the basis for the classification is unknown, making it difficult to know the basis for the classification. Summary of the Invention

[0006] According to a first aspect of this disclosure, a method is provided for determining the category of discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The method includes the following steps: (a) preparing a set of known feature spectra for each of more than one category, obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) performing category discrimination processing on the discrimination data using the machine learning model and the known feature spectra. Step (b) includes the following steps: (b1) the machine Ns is preferably set to 1 or more, and preferably 2 or more. Based on the input of the discrimination data to the learning model, a feature spectrum is calculated from the output of the specific layer; (b2) the similarity between the feature spectrum and the known feature spectra associated with each category is calculated; (b3) based on the similarity, a descriptive statement is created for the category discrimination result of the discrimination data; and (b4) the descriptive statement is output.

[0007] According to a second aspect of this disclosure, an information processing apparatus is provided that performs category discrimination processing on discrimination data using a machine learning model of a vector neural network having multiple vector neuron layers. The information processing apparatus includes: a memory storing the machine learning model; and a processor performing operations using the machine learning model. The processor is configured to perform the following processes: (a) for each of more than one category, reading from the memory a set of known feature spectra obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) performing category discrimination processing on the discrimination data using the machine learning model and the known feature spectra. Process (b) includes the following processes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known feature spectra associated with each category; (b3) creating a descriptive statement for the category discrimination result of the discrimination data based on the similarity; and (b4) outputting the descriptive statement.

[0008] According to a third aspect of this disclosure, a computer program is configured to execute a class discrimination process using a vector neural network-type machine learning model with multiple vector neuron layers to classify the category of the discrimination data. This computer program causes the processor to perform the following processes: (a) for each of more than one category, reading from memory a set of known feature spectra obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) using the machine learning model and the known feature spectra to perform the class discrimination process on the discrimination data. Process (b) includes the following processes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known feature spectra associated with each category; (b3) creating a descriptive statement for the class discrimination result of the discrimination data based on the similarity; and (b4) outputting the descriptive statement. Attached Figure Description

[0009] Figure 1 This is a block diagram of the category discrimination system of the first embodiment.

[0010] Figure 2 This is a block diagram of an information processing device.

[0011] Figure 3 This is an explanatory diagram showing the structure of a machine learning model.

[0012] Figure 4This is an illustrative diagram representing other structures of a machine learning model.

[0013] Figure 5 This is a flowchart illustrating the preparation process for a machine learning model.

[0014] Figure 6 This is an explanatory diagram showing the characteristic spectrum.

[0015] Figure 7 This is an explanatory diagram illustrating the situation of creating a known characteristic spectral group using teaching data.

[0016] Figure 8 This is an explanatory diagram showing the structure of a known characteristic spectral group.

[0017] Figure 9 This is a flowchart illustrating the processing sequence of media identification / printing steps.

[0018] Figure 10 This is an explanatory diagram illustrating the process of solving for similarity related to the data to be judged.

[0019] Figure 11 This is an explanatory diagram illustrating the creation process of the explanatory statements in the first embodiment.

[0020] Figure 12 This is an explanatory diagram showing an example of a descriptive statement displayed on the display section.

[0021] Figure 13 This is an explanatory diagram illustrating the creation process of the explanatory statements in the second embodiment.

[0022] Figure 14 This is a coordinate graph showing the infrared absorption spectrum of the object to be identified according to the third embodiment.

[0023] Figure 15 This is an explanatory diagram illustrating the creation process of the explanatory statements in the third embodiment.

[0024] Figure 16 This is an illustration of the first method for calculating similarity.

[0025] Figure 17 This is an illustration of the second similarity calculation method.

[0026] Figure 18 This is an illustration of a method for determining categories using multiple specific layers.

[0027] Explanation of reference numerals in the attached figures

[0028] 10…Printer; 20…Information processing device; 30…Spectrometer; 110…Processor; 112…Printing processing unit; 114…Classification processing unit; 120…Memory; 130…Interface circuit; 150…Display unit; 200…Machine learning model; 210…Convolutional layer; 220…Main vector neuron layer; 230…First convolutional vector neuron layer; 240…Second convolutional vector neuron layer; 250…Classification vector neuron layer; 310…Similarity calculation unit; 320…Description statement creation unit; 321…Average calculation unit; 322…Grayscale reduction unit; 324…String lookup table. Detailed Implementation

[0029] A. First implementation method:

[0030] Figure 1 This is a block diagram illustrating the category discrimination system of the first embodiment. The category discrimination system is a printing system including a printer 10, an information processing device 20, and a spectrometer 30. The spectrometer 30 can perform spectroscopic measurements on the printing medium PM used in the printer 10 in a non-printed state to obtain the spectroscopic reflectance. In this disclosure, the spectroscopic reflectance is also referred to as "spectral data." The spectrometer 30 includes, for example, a wavelength-variable interference spectroscopic filter and a monochrome image sensor. The spectroscopic data obtained by the spectrometer 30 is used as discrimination data input to the machine learning model described later. The information processing device 20 uses the machine learning model to perform category discrimination processing on the spectroscopic data, determining which of several categories the printing medium PM corresponds to. "Category of printing medium PM" refers to the type of printing medium PM. The information processing device 20 controls the printer 10 to perform printing under appropriate printing conditions corresponding to the type of printing medium PM. Furthermore, the classification system disclosed herein can also be implemented as a system other than a printing system. For example, it can also be implemented as a system that performs classification using classification images, one-dimensional data other than spectroscopic data, spectroscopic images, time series data, etc., as classification data.

[0031] Figure 2 This is a block diagram illustrating the functions of the information processing device 20. The information processing device 20 includes a processor 110, a memory 120, an interface circuit 130, an input device 140 connected to the interface circuit 130, and a display unit 150. The interface circuit 130 is also connected to a spectrometer 30 and a printer 10. While not limited, for example, the processor 110 not only has the function of performing the processes detailed below, but also has the function of displaying the data obtained through the processing and the data generated during the processing on the display unit 150.

[0032] The processor 110 functions as a printing processing unit 112 that performs printing processing using the printer 10, and also functions as a category discrimination processing unit 114 that performs category discrimination processing of the spectroscopic data of the printing medium PM. The category discrimination processing unit 114 includes a similarity calculation unit 310 and a description statement creation unit 320. The printing processing unit 112 and the category discrimination processing unit 114 are implemented by the processor 110 executing a computer program stored in the memory 120. Alternatively, the aforementioned units 112 and 114 can also be implemented using hardware circuitry. In this specification, "processor" refers to a device that also includes such hardware circuitry. Furthermore, the processor performing the category discrimination processing can also be a processor included in a remote computer connected to the information processing device 20 via a network.

[0033] The memory 120 stores the machine learning model 200, training data TD, known feature spectrum set KSp, printing setting table PST, description statement template DT, and string lookup table CT. The memory 120 may also store a string generation database and a corresponding dictionary used for generating strings. The machine learning model 200 is used for processing by the category discrimination processing unit 114. The structure and operation of the machine learning model 200 will be described later. The training data TD is a collection of labeled data used for learning the machine learning model 200. In this embodiment, the training data TD is a collection of spectroscopic data. The known feature spectrum set KSp is a collection of feature spectra obtained when the training data TD is input into the machine learning model 200 after learning. Feature spectra will be described later. The printing setting table PST is a table that registers printing settings suitable for various printing media. The description statement template DT and the string lookup table CT are used for the creation processing of description statements created by the description statement creation unit 320. The string lookup table CT can also be called the "string selection unit CT".

[0034] Figure 3 This is an explanatory diagram illustrating the structure of machine learning model 200. From the input data IM side, machine learning model 200 sequentially includes a convolutional layer 210, a principal vector neuron layer 220, a first convolutional vector neuron layer 230, a second convolutional vector neuron layer 240, and a classification vector neuron layer 250. Of these five layers 210-250, convolutional layer 210 is the lowest layer, and classification vector neuron layer 250 is the highest layer. In the following description, layers 210-250 will also be referred to as "Conv layer 210," "PrimeVN layer 220," "ConvVN1 layer 230," "ConvVN2 layer 240," and "ClassVN layer 250."

[0035] In this embodiment, the input data IM is spectroscopic data, and it is arranged in one dimension. For example, the input data IM is spectroscopic data in the range of 380nm to 730nm, from which 36 representative values ​​are extracted every 10nm.

[0036] exist Figure 3 In the example, two convolutional vector neuron layers, 230 and 240, are used, but the number of convolutional vector neuron layers is arbitrary, and they can also be omitted. Preferably, more than one convolutional vector neuron layer is used.

[0037] Figure 3 The structure of each layer from 210 to 250 can be described as follows.

[0038] <Description of the structure of Machine Learning Model 200>

[0039] • Conv layer 210: Conv[32,6,2]

[0040] • PrimeVN layer 220: PrimeVN[26,1,1]

[0041] • ConvVN1 layer 230: ConvVN1[20,5,2]

[0042] • ConvVN2 layer 240: ConvVN2[16,4,1]

[0043] • ClassVN layer 250: ClassVN[n1,3,1]

[0044] • Vector dimension VD: VD = 16

[0045] In the descriptions of layers 210 to 250 above, the string before the parentheses is the layer name, and the numbers inside the parentheses are, in order, the number of channels, the surface size of the kernel, and the stride. For example, the layer name of Conv layer 210 is "Conv", the number of channels is 32, the surface size of the kernel is 1×6, and the stride is 2. Figure 3 The descriptions for each layer are shown below. The rectangles marked with cross-sections depicted in each layer represent the surface dimensions of the kernel used when calculating the output vectors of adjacent upper layers. In this embodiment, the input data IM is arranged in a one-dimensional array, and the surface dimensions of the kernel are also one-dimensional. Furthermore, the values ​​of the parameters used in the descriptions of layers 210–250 are illustrative and can be arbitrarily changed.

[0046] Conv layer 210 is a layer composed of scalar neurons. The other four layers 220-250 are layers composed of vector neurons. A vector neuron is a neuron that takes a vector as its input and outputs a vector. In the above description, the output vector of each vector neuron has a dimension of 16 and is constant. Hereafter, the term "node" will be used as a higher-level concept for scalar neurons and vector neurons.

[0047] Figure 3 In the diagram, for Conv layer 210, the first axis x, the second axis y, and the third axis z representing the depth of the planar coordinates defining the node arrangement are shown. Furthermore, the dimensions of Conv layer 210 in the x, y, and z directions are shown to be 1, 16, and 32, respectively. The dimensions in the x and y directions are referred to as "resolution." In this embodiment, the resolution in the x direction is always 1. The dimension in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes representing the position of each node in other layers. Figure 3 In the diagrams, except for Conv layer 210, the x, y, and z axes are omitted.

[0048] As is well known, the resolution W1 in the y-direction after convolution is given by the following formula.

[0049] W1=Ceil{(W0-Wk+1) / S} (1)

[0050] Here, W0 is the resolution before convolution, Wk is the kernel surface size, S is the stride, and Ceil{X} is the function that performs the carry operation on X. Alternatively, Ceil{X} can also be a function that performs integer operations on X.

[0051] Figure 3 The resolutions shown are examples of the input data IM with a resolution of 36 in the y-direction. The actual resolutions of each layer are adjusted according to the size of the input data IM.

[0052] ClassVN layer 250 has n1 channels. Figure 3 In the example, n1 = 3. Typically, n1 is an integer greater than or equal to 1, representing the number of known classes that can be identified using the machine learning model 200. It's common to set n1 to 2 or higher. The three channels of the ClassVN layer 250 output decision values ​​Class1 to Class3 relative to the three known classes. Typically, the class with the highest value among these decision values ​​Class1 to Class3 is used as the class classification result for the input data IM. Alternatively, if the highest value among the decision values ​​Class1 to Class3 is less than a predetermined threshold, the class of the input data IM is determined to be unknown.

[0053] In this disclosure, as described later, it is also possible to replace the use of the decision values ​​Class1 to Class3 of the output layer, i.e., the ClassVN layer 250, and use the class similarity calculated based on the output of a specific vector neuron layer to determine the class.

[0054] Figure 3 The diagram also depicts partial regions Rn of layers 210, 220, 230, 240, and 250. The suffix "n" in partial region Rn is a reference numeral for each layer. For example, partial region R210 represents a partial region of Conv layer 210. A "partial region Rn" is a region defined in each layer within a planar position (x, y) by the position of the first axis x and the position of the second axis y, and includes multiple channels along the third axis z. Partial region Rn has dimensions of "width" × "height" × "depth" corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in a "partial region Rn" is "1 × 1 × depth number", i.e., "1 × 1 × channel number".

[0055] like Figure 3 As shown, the feature spectrum Sp_ConvVN1 (described later) is calculated based on the output of ConvVN1 layer 230 and input to the similarity calculation unit 310. Similarly, the feature spectrum Sp_ConvVN2 is calculated based on the output of ConvVN2 layer 240 and input to the similarity calculation unit 310. In other embodiments, the feature spectrum is also calculated based on the output of ClassVN layer 250 and input to the similarity calculation unit 310. The similarity calculation unit 310 uses the aforementioned feature spectrum Sp_ConvVN1, Sp_ConvVN, and a pre-created known feature spectrum group KSp to calculate the similarity Sm (described later). The description statement creation unit 320 uses the similarity Sm to create a description statement DS related to the category discrimination result. As the similarity used for creating the description statement, various similarities such as local similarity and category similarity (described later) can be used. In the embodiments described below, the method of creating a description statement using local similarity will be mainly described.

[0056] In this disclosure, the vector neuron layer used for similarity calculation is also referred to as a "specific layer". As a specific layer, any number of vector neuron layers, more than one, can be used. Furthermore, the structure of the feature spectrum, the method for calculating similarity using the feature spectrum, the method for creating the descriptive statements using similarity, and the method for determining the classification will be described later.

[0057] Figure 4 This is an illustrative diagram representing other structures of the machine learning model 200. This machine learning model 200 differs from models using one-dimensionally arranged input data in that the input data IM is arranged in two dimensions. Figure 3The machine learning models are 200 different. Figure 4 The structure of each layer from 210 to 250 can be described as follows.

[0058] <Description of the structure of each layer>

[0059] • Conv layer 210: Conv[32,5,2]

[0060] • PrimeVN layer 220: PrimeVN[16,1,1]

[0061] • ConvVN1 layer 230: ConvVN1[12,3,2]

[0062] • ConvVN2 layer 240: ConvVN2[6,3,1]

[0063] • ClassVN layer 250: ClassVN[n1,4,1]

[0064] • Vector dimension VD: VD = 16

[0065] Figure 4 The machine learning model 200 shown is, for example, a class discrimination system capable of classifying images. In the following description, the term "class discrimination" is used... Figure 3 The machine learning model 200 is shown.

[0066] Figure 5 This is a flowchart illustrating the processing sequence of the preparation steps for a machine learning model. This preparation step, for example, is performed by the manufacturer of printer 10.

[0067] exist Figure 5 In step S110, the category discrimination processing unit 114 uses multiple teaching data TDs to perform learning of the machine learning model 200. Each teaching data TD is pre-labeled. In this embodiment, it is assumed that any one of labels 1 to 3 is assigned to each teaching data TD. These labels correspond to the three categories Class1 to Class3 of the machine learning model 200. In this disclosure, "label" and "category" are the same.

[0068] If learning using multiple teaching data TDs is completed, the learned machine learning model 200 is stored in memory 120. Figure 5 In step S120, multiple teaching data TDs are input again into the learned machine learning model 200 to generate a known feature spectrum set KSp. The known feature spectrum set KSp is the set of feature spectra described below.

[0069] Figure 6This is an explanatory diagram showing the feature spectrum Sp obtained by inputting arbitrary input data into the learned machine learning model 200. Here, the feature spectrum Sp obtained from the output of ConvVN1 layer 230 is explained. Figure 6 The horizontal axis represents the position of vector elements associated with the output vectors of multiple nodes included in a partial region R230 of the ConvVN1 layer 230. The position of these vector elements is expressed by a combination of the element number ND and channel number NC of the output vector of each node. In this embodiment, the vector dimension is 16, therefore, the element numbers ND of the output vectors are 16, from 0 to 15. Additionally, the ConvVN1 layer 230 has 20 channels, therefore, the channel numbers NC are 20, from 0 to 19. In other words, the feature spectrum Sp arranges the multiple element values ​​of the output vectors of each vector neuron included in a partial region R230 across multiple channels along the third axis z.

[0070] Figure 6 The vertical axis represents the characteristic value C at each spectral position. V In this example, the eigenvalue C V The values ​​V of each element in the output vector ND Furthermore, as an eigenvalue C V Alternatively, the values ​​V of each element in the output vector can be used. ND The value multiplied by the normalization coefficient described later, or the normalization coefficient can be used as is. In the latter case, the characteristic value C included in the characteristic spectrum Sp V The number of channels is equal to the number of nodes, which is 20. Furthermore, the normalization coefficient is a value equivalent to the vector length of the output vector of that node.

[0071] The number of feature spectra Sp obtained from the output of ConvVN1 layer 230 relative to one input data is equal to the number of planar positions (x, y) of ConvVN1 layer 230, i.e., the number of partial regions R230, which is 6. Similarly, three feature spectra Sp are obtained from the output of ConvVN2 layer 240 relative to one input data.

[0072] When the similarity calculation unit 310 inputs the teaching data TD into the machine learning model 200 after it has finished learning, it calculates... Figure 6 The characteristic spectrum Sp shown is registered in memory 120 as a known characteristic spectrum group KSp.

[0073] Figure 7This is an explanatory diagram illustrating the creation of a known feature spectrum group KSp using teaching data TD. In this example, by inputting the teaching data TD labeled 1 to 3 into the learned machine learning model 200, the feature spectra KSp_ConvVN1 and KSp_ConvVN2 corresponding to each label or category are obtained from the outputs of the two vector neuron layers, namely ConvVN1 layer 230 and ConvVN2 layer 240. The aforementioned feature spectra KSp_ConvVN1 and KSp_ConvVN2 are stored in memory 120 as the known feature spectrum group KSp.

[0074] Figure 8 This is an explanatory diagram representing the structure of a known characteristic spectral group KSp. In this example, the known characteristic spectral group KSp_ConvVN2 obtained from the output of ConvVN2 layer 240 is shown. The known characteristic spectral group KSp_ConvVN1 obtained from the output of ConvVN1 layer 230 also has the same structure, but... Figure 8 Illustrations omitted. Furthermore, as a known feature spectral set KSp, it is sufficient to register at least a known feature spectral set obtained from the output of a vector neuron layer.

[0075] The records in the known characteristic spectral group KSp_ConvVN2 include: parameter i representing the order of labels or categories, parameter j representing the order of specific layers, parameter k representing the order of partial regions Rn, parameter q representing the data number, and the known characteristic spectrum KSp. The known characteristic spectrum KSp and... Figure 6 The characteristic spectra Sp are the same.

[0076] The parameter i for the category is the same as the label, taking values ​​from 1 to 3. The parameter j for the specific layer takes a value from 1 to 2, indicating which of the two specific layers 230 and 240 it is. The parameter k for the partial region Rn takes a value indicating which of the multiple partial regions Rn included in each specific layer, i.e., which one of the planar positions (x, y). For ConvVN2 layer 240, there are three partial regions R240, therefore, k = 1 to 3. The parameter q for the data number indicates the number of the teaching data assigned the same label; for category 1, it takes a value from 1 to max1; for category 2, it takes a value from 1 to max2; and for category 3, it takes a value from 1 to max3.

[0077] Furthermore, the multiple teaching data TDs used in step S120 do not need to be the same as the multiple teaching data TDs used in step S110. Step S120 also has the following advantage: if some or all of the multiple teaching data TDs used in step S110 are used, no new teaching data needs to be prepared.

[0078] Figure 9This is a flowchart illustrating the processing sequence of the media identification / printing process using a learned machine learning model. This media identification / printing process is performed, for example, by a user using printer 10.

[0079] In step S210, the user instructs the category determination processing unit 114 whether category determination processing is required for the printing medium being processed, i.e., the target printing medium. Even if the user knows the type of the target printing medium, they can still instruct the user to perform category determination processing for confirmation. If category determination processing is not required, the process proceeds to step S270, where the user selects printing settings suitable for the target printing medium. In step S280, the printer 10 causes the printing processing unit 112 to perform printing using the target printing medium. On the other hand, if the type of the target printing medium is unknown and category determination processing is required, the process proceeds to step S220.

[0080] In step S220, the category discrimination processing unit 114 acquires spectrophotometric data by causing the spectrophotometer 30 to perform a spectrophotometric measurement of the target printing medium. This spectrophotometric data is then used as discrimination data input to the machine learning model 200.

[0081] In step S230, the category discrimination processing unit 114 inputs discrimination data into the learned machine learning model 200 and calculates the feature spectrum Sp. In step S240, the similarity calculation unit 310 calculates the similarity from the feature spectrum Sp obtained based on the input discrimination data and the registered known feature spectrum group KSp.

[0082] Figure 10 This is an explanatory diagram illustrating the process of calculating similarity related to the discriminant data. If the discriminant data is input into the machine learning model 200, the category discrimination processing unit 114 calculates the feature spectra Sp_ConvVN1 and Sp_ConvVN2 based on the outputs of the ConvVN1 layer 230 and the ConvVN2 layer 240, respectively. The similarity calculation unit 310 calculates the similarity Sm_ConvVN1 using the feature spectrum Sp_ConvVN1 obtained from the output of the ConvVN1 layer 230 and the known feature spectrum group KSp_ConvVN1. The specific calculation method for the similarity Sm will be described later. The similarity Sm_ConvVN2 is calculated similarly for the ConvVN2 layer 240.

[0083] The similarity Sm can be calculated, for example, according to the following formula.

[0084] Sm(i,j,k)=max[G{Sp(j,k),KSp(i,j,k=all,q=all)}] (2)

[0085] Here,

[0086] i is a parameter representing the category.

[0087] j is a parameter representing a specific layer.

[0088] k is a parameter representing a partial region Rn.

[0089] q is a parameter representing the data number.

[0090] G{a,b} is a function that calculates the similarity between a and b.

[0091] Sp(j,k) is the characteristic spectrum obtained from the output of a specific region k in a specific layer j, based on the discrimination data.

[0092] KSp(i,j,k=all,q=all) is Figure 8 The known characteristic spectra of all data numbers q of all partial regions k of a specific layer j associated with category i in the known characteristic spectrum group KSp are shown. max[X] is a logical operation that takes the maximum value of X.

[0093] Furthermore, the function G{a,b} for calculating similarity takes a single or set as 'a' and a set as 'b', and returns multiple values. As a function G{a,b}, for example, it can use formulas for calculating cosine similarity and formulas for calculating similarity corresponding to distance.

[0094] The similarity Sm is obtained for each partial region, and therefore, it is also referred to as "local similarity Sm" below. The local similarity Sm(i,j,k) depends on the category i, the specific layer j, and the partial region k. In the following description, the parameter i representing the category and the parameter j representing the specific layer are sometimes omitted and referred to as "local similarity Sm(k)".

[0095] It is not necessary to use two separate vector neuron layers 230 and 240 to generate all similarities Sm_ConvVN1 and Sm_ConvVN2, but it is preferable to use more than one of these vector neuron layers to calculate the similarity Sm. As mentioned above, in this disclosure, the vector neuron layer used for similarity calculation is referred to as a "specific layer".

[0096] In step S250, the description statement creation unit 320 creates a description statement based on the similarity obtained in step S240.

[0097] Figure 11 This is an explanatory diagram illustrating the creation process of the explanatory statements according to the first embodiment. The explanatory statement creation unit 320a of the first embodiment includes a grayscale reduction unit 322 and a string lookup table (string LUT) 324. The reference numeral "a" at the end of the explanatory statement creation unit 320a indicates that it is the first embodiment. The string lookup table 324 and... Figure 2 The string lookup table CT shown is the same.

[0098] Figure 11 The image shows the creation of descriptive statements using the local similarity Sm obtained from three partial regions of ConvVN2 layer 240. For example... Figure 11 As shown in the upper left, the three regions of the ConvVN2 layer 240 correspond to three wavelength bands with center wavelengths of 300nm, 500nm, and 700nm, respectively. The grayscale reduction unit 322 creates three table input data D1 to D3 by reducing the grayscale value (the number of discrete values ​​obtained within the range of maximum and minimum values) of the three local similarities Sm(k). Specifically, the grayscale reduction unit 322 binarizes (quantizes) the local similarities Sm(k) using a pre-created threshold. In this example, the local similarity Sm(k) is 16 bits of data with a grayscale value of 2. 16 The input data D1 to D3 are 1-bit data with a grayscale value of 2. The grayscale value of the input data D1 to D3 can also be 3 or higher, and can be any number greater than 2.

[0099] String lookup table 324 outputs strings CS1 to CS3 based on the input data D1 to D3. Figure 11 The lower left shows examples of strings CS1 to CS3 corresponding to combinations of table input data D1 to D3. The statement creation unit 320 creates the statement DSa by using the three strings CS1 to CS3 as a statement template DTa with three corresponding string boxes. Figure 11 In the example, the local similarity Sm of the two wavelength bands at 500nm and 700nm is above the threshold, but the local similarity Sm of the 300nm wavelength band is below the threshold. Therefore, an appropriate explanatory statement DSa is created to correspond to it. Thus, in this embodiment, an explanatory statement DSa corresponding to the similarity Sm can be created using the string lookup table 322 and the explanatory statement template DTa. The explanatory statement DSa created in this way is displayed on the display unit 150.

[0100] Figure 11 The various numbers used are as follows.

[0101] (1) The number of partial regions included in a specific layer, Nk

[0102] Figure 11 In the example, Nk = 3.

[0103] (2) The number of local similarities Sm used to describe the creation of the statement Ns

[0104] Figure 11In the example, Ns = Nk = 3, but Ns can be set to any number less than Nk. Ns can be set to 1 or more, but is preferably 2 or more. Examples of Ns < Nk will be described in the third embodiment.

[0105] (3) The number of input data Dk in the table, Nd

[0106] Figure 11 In the example, Nd = Ns = 3, but Nd can be set to any number greater than 1 and less than Ns. Nd can be set to 1 or more, but is preferably 2 or more. Examples of Nd < Ns will be described in the second embodiment.

[0107] (4) The number of strings Nc output from string lookup table 324

[0108] Figure 11 In the example, Nc = 3, but Nc can be set to any number greater than 1.

[0109] Figure 12 An example of explanatory statements displayed on display unit 150 is shown. In this example, a list DL of classification results, arranged with classification results and explanatory statements, is displayed on display unit 150 for multiple categories. As a classification result, the parameter i representing the category, the category name, and the probability that the classified data corresponds to that category, i.e., the category probability, are displayed. For example, the decision value output from the output layer of machine learning model 200 can be used as the category probability. Alternatively, when using category similarity for classification (described later), this category similarity can also be used as the category probability. Furthermore, a portion of the classification result can be omitted. Alternatively, the activation value of the ClassVN layer 250 of machine learning model 200 can be used as the category probability, and an explanatory statement can be created based on the activation value. Specifically, for example, an explanatory statement such as "Activation = 0.99, therefore the reliability of the classification result is high" can be created. Moreover, the similarity calculated from the output of ClassVN layer 250 can also be used as the category probability, and an explanatory statement can be created based on this similarity. Specifically, for example, it is possible to create descriptive statements such as "ClassVN similarity = 0.99, therefore known data".

[0110] like Figure 12As in the example, regarding multiple categories, if a list of classification results (DL) containing category classification results and explanatory statements is displayed on the display unit 150, the user can understand the basis for the category classification results related to each category. Furthermore, the list of classification results (DL) does not need to include information related to all categories that the machine learning model 200 can classify; preferably, it includes information related to at least two categories. Alternatively, instead of displaying the list of classification results (DL), an explanatory statement related to one category classified by the machine learning model 200 may be displayed.

[0111] In step S260, the user, referring to the description statement created in step S250, selects the category (type) of the target printing medium and instructs the printing processing unit 112 on the selected type. In step S270, the printing processing unit 112 selects the printing settings according to the type of the target printing medium and referring to the printing setting table PST. In step S280, the printing processing unit 112 performs printing according to the printing settings. Figure 9 Even when the type of the printing medium is unknown, the machine learning model 200 can determine the type of the printing medium and thus perform printing using printing settings appropriate for that type.

[0112] As described above, in the first embodiment, explanatory statements of the category discrimination results are created and output using similarity related to the feature vectors, so that the user can know the basis for the category discrimination results.

[0113] B. Second implementation method:

[0114] Figure 13 This is an explanatory diagram illustrating the creation process of the explanatory statements in the second embodiment. In addition to the grayscale reduction unit 322 and the string lookup table 324, the explanatory statement creation unit 320b in the second embodiment also includes an average value calculation unit 321. Furthermore, Figure 1 as well as Figure 2 The structure of the device shown Figure 5 as well as Figure 9 The processing order shown is almost the same as that in the first embodiment.

[0115] Figure 13 The diagram illustrates the creation of explanatory statements using local similarities Sm obtained from six partial regions of ConvVN1 layer 230. The averaging unit 321 calculates three average similarities Sma1 to Sma3 by grouping the six local similarities Sm into three groups and averaging the local similarities Sm belonging to each group. The number of average similarities is the same as the number of input data Nd in the table Dk. These three average similarities Sma1 to Sma3 are similar to those in the first embodiment. Figure 11The three local similarities Sm shown are the same, representing the similarity for three wavelength bands centered at 300nm, 500nm, and 700nm respectively. The grayscale reduction unit 322 creates table input data Dk by reducing the grayscale values ​​of the aforementioned average similarities Sma1 to Sma3. Subsequent processing is similar to... Figure 11 The first embodiment shown is the same, therefore, detailed description is omitted. Figure 13 In the example, create a declarative statement DSb such as "All ingredients are consistent, therefore it is considered known".

[0116] In the above Figure 13 In the example, the average similarity Sma is created by averaging the local similarities Sm for two adjacent partial regions. Typically, the average similarity Sma can be calculated by grouping the Ns local similarities Sm used to create the descriptive statement into Nd groups and averaging the local similarities Sm of each group. In this case, Nd becomes a value smaller than Ns.

[0117] As described above, in the second embodiment, Nd average similarities are calculated from Ns local similarities, and Nd table input data are created by reducing the gray level of the Nd average similarities. Therefore, explanatory statements can be created based on the average similarity that averages the local similarities.

[0118] Furthermore, in the above description, the average similarity Sma is calculated for each group of local similarities, but representative values ​​other than the average similarity can also be calculated. Besides the average, the maximum and minimum values ​​can also be used as representative values. In other words, in the second embodiment, Nd representative similarities can be calculated from Ns local similarities, and Nd table input data can be created by reducing the grayscale value of the Nd representative similarities. In this way, explanatory statements can be created based on the representative similarities.

[0119] C. Third implementation method:

[0120] Figure 14 This is an explanatory graph showing the infrared absorption spectra of 2-hexanone and acrylonitrile, the objects of the classification process in the third embodiment. The horizontal axis of each graph represents wavenumber, and the vertical axis represents absorbance. In the third embodiment, it is assumed that the determination of whether the object of classification is 2-hexanone is known, given that the object of classification is either 2-hexanone or acrylonitrile. In this case, Figure 3 The output layer of the machine learning model 200 shown has 1 class.

[0121] The infrared absorption spectrum of 2-hexanone is at 2900 cm⁻¹ -1 The wavenumber band includes the absorption peak based on CH coupling, at 1700 cm⁻¹.-1 The wavenumber band includes absorption peaks based on C=O coupling. On the other hand, the infrared absorption spectrum of acrylonitrile is at 2900 cm⁻¹. -1 The wavenumber band includes the absorption peak based on CH coupling, at 2300 cm⁻¹. -1 The wavenumber bands include absorption peaks based on CN coupling. The coordinate plots illustrate the relationship between the parameter k, which distinguishes six partial regions of ConvVN1 layer 230, and the wavenumber bands. As explained below, in the third embodiment, explanatory statements are created using a portion of the local similarity Sm of the six partial regions.

[0122] Figure 15 This is an explanatory diagram illustrating the creation process of the explanatory statements in the third embodiment. The explanatory statement creation unit 320c of the third embodiment, in that it includes a grayscale reduction unit 322 and a string lookup table 324, is similar to... Figure 11 The first embodiment shown is the same. However, the string stored in string lookup table 324 is different from that in the first embodiment. Furthermore, Figure 2 The information processing apparatus 20 shown can also be used in the third embodiment, except that it does not use the printer 10 and the printing processing unit 112. Furthermore, Figure 5 The processing order shown can use almost the same content as the first embodiment. Furthermore, for Figure 9 The process shown replaces "object printing medium" with "object to be identified". By executing steps S250 to S270, the category identification of the third embodiment can be performed.

[0123] Figure 15 In the example, the descriptive statement is created using three pre-selected local similarities Sm(k=1), Sm(k=3), and Sm(k=5) from the local similarities Sm obtained from the six partial regions of ConvVN1 layer 230. These three local similarities Sm(k=1), Sm(k=3), and Sm(k=5) are respectively compared with... Figure 14 The 1700cm shown -1 Wavenumber band, 2300cm -1 Wavenumber band, 2900cm -1 Wavenumber band correspondence. The number of selected local similarities Sm is the same as the number of input data Nd in the table Dk.

[0124] Similar to the first embodiment, the grayscale reduction unit 322 creates table input data Dk by reducing the grayscale value of the local similarity Sm. The string lookup table 324 outputs three strings CS1 to CS3 based on the input of the table input data D1 to D3. The description statement creation unit 320 creates a description statement DSc by applying the three strings CS1 to CS3 to a description statement template DTc with three corresponding string boxes. (Comparison) Figure 11 and Figure 15 As can be seen, the contents of the string lookup table 324 and the contents of the description statement template DTc are different from those in the first embodiment. Figure 15 In the example, all three local similarities Sm used to create the descriptive statement are above the threshold. Therefore, an appropriate descriptive statement DSc is created to indicate that the object belongs to 2-hexanone.

[0125] Alternatively, a portion of the total 12 individual similarities—6 calculated for hexanone and 6 calculated for acrylonitrile—can be used to create descriptive statements. For example, statements such as "C=O exists, CN does not exist, therefore, it is identified as hexanone" or "C=O does not exist, CN exists, therefore, it is identified as acrylonitrile" can be created.

[0126] As described above, in the third embodiment, table input data is created based on the local similarity of a portion of a subset of a plurality of subsets included in a particular layer. Therefore, explanatory statements can be created using the local similarity of the description suitable for the discrimination result.

[0127] Furthermore, while string lookup tables are used to create strings in the various implementations described above, decision trees can also be used instead of string lookup tables to create strings and describe statements.

[0128] D. Methods for calculating similarity:

[0129] As a method for calculating the aforementioned local similarity Sm, either of the following two methods can be used.

[0130] (1) The first calculation method M1 for solving the local similarity Sm without considering the correspondence between the characteristic spectrum Sp and a portion of the known characteristic spectrum group KSp.

[0131] (2) The second calculation method M2 is to solve the local similarity Sm based on the corresponding partial regions Rn of the characteristic spectrum Sp and the known characteristic spectrum group KSp.

[0132] The following sections will describe the methods for calculating similarity from the output of ConvVN1 layer 230 based on the two calculation methods M1 and M2 mentioned above.

[0133] Figure 16 This is an explanatory diagram of the first similarity calculation method M1. Here, it shows the case where the local similarity Sm(i,j,k) representing the similarity for each category i is calculated for each partial region k based on the output of a specific layer, namely ConvVN1 layer 230.

[0134] In the first operation method M1, the local similarity Sm(i,j,k) is calculated using the following reprinted equation (2).

[0135] Sm(i,j,k)=max[G{Sp(j,k),KSp(i,j,k=all,q=all)}] (2)

[0136] Here,

[0137] i is a parameter representing the category.

[0138] j is a parameter representing a specific layer.

[0139] k is a parameter representing a partial region Rn.

[0140] q is a parameter representing the data number.

[0141] G{a,b} is a function for calculating the similarity between a and b.

[0142] Sp(j,k) is the characteristic spectrum obtained from the output of a specific region k in a specific layer j, based on the discrimination data.

[0143] KSp(i,j,k=all,q=all) is Figure 8 The known characteristic spectra of all data numbers q in all partial regions k of a specific layer j associated with category i in the known characteristic spectral group KSp are shown.

[0144] max[X] is a logical operation that retrieves the maximum value among the values ​​of X.

[0145] Furthermore, the function G{a,b} for calculating similarity takes a single or set as 'a' and a set as 'b', and returns multiple values. As a function G{a,b}, it can, for example, use formulas for calculating cosine similarity or formulas for calculating similarity corresponding to distance.

[0146] Figure 16The right side shows the calculation of the category similarity Sclass(i,j) based on the aforementioned local similarity Sm(i,j,k). The three categories of category similarity Sclass(i,j) are obtained by taking the maximum, average, or minimum value of the local similarity Sm(i,j,k) for each category i across multiple partial regions k. Which operation—maximum, average, or minimum—is used depends on the purpose of the category discrimination process. For example, when the purpose is to identify objects using natural images, it is preferable to solve for the category similarity Sclass(i,j) by taking the maximum value of the local similarity Sm(i,j,k) for each category i. Furthermore, when the purpose is to identify the type of printed media or to determine the quality of images using industrial products, it is preferable to solve for the category similarity Sclass(i,j) by taking the minimum value of the local similarity Sm(i,j,k) for each category i. Alternatively, we also consider the case where the category similarity Sclass(i,j) is calculated by averaging the local similarities Sm(i,j,k) for each category i. Which of these three operations is used is preset by the user based on experimentation or experience.

[0147] Figure 16 In the example, the final discrimination result RD_ConvVN1 is also determined based on the class similarity Sclass(i,j). The discrimination result RD_ConvVN1 is expressed in the form of including the discrimination class D_class and the similarity value S_value corresponding to the discrimination class D_class. The similarity value S_value is obtained by taking the maximum of the similarity values ​​among the three classes 1 to 3 of the class similarity Sclass(i,j). The discrimination class D_class is the class with the highest similarity value in the class similarity Sclass(i,j).

[0148] As described above, in the first similarity calculation method M1,

[0149] (1) Based on the discrimination data, calculate the similarity, i.e., the local similarity Sm(i,j,k), between the characteristic spectrum Sp obtained from the output of a specific region k of a specific layer j and all known characteristic spectra KSp associated with that specific layer j and each category i.

[0150] (2) Solve for the class-specific similarity Sclass(i,j) by obtaining the maximum, average, or minimum value of the local similarity Sm(i,j,k) for multiple partial regions k for each class i.

[0151] (3) The maximum value of the category similarity Sclass(i,j) for multiple categories i is obtained as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp.

[0152] (4) The category corresponding to the largest similarity value S_value across multiple categories is determined as the discriminant category D_class.

[0153] According to the first operation method M1, the similarity Sm(i,j,k), Sclass(i,j) and the discrimination result can be solved through relatively simple operations and order.

[0154] As the discrimination result using machine learning model 200, the discrimination class D_class determined based on the class similarity Sclass(i,j) can be used, or the discrimination class determined based on the judgment value obtained from the output layer of machine learning model 200 can be used. In the latter case, the calculation of class similarity Sclass(i,j) and subsequent processing can be omitted. These aspects are also the same in the second operation method M2 described below.

[0155] The description statement creation unit 320 can also create description statements related to the judgment result based on the category similarity Sclass(i,j). For example, the description statement corresponding to the category similarity Sclass(i,j) could be "The similarity with category 1 is 98%, therefore, it is judged as known".

[0156] Figure 17 This is an explanatory diagram of the second similarity calculation method M2. In the second calculation method M2, the following formula is used instead of the above formula (2) to calculate the local similarity Sm(i,j,k).

[0157] Sm(i,j,k)=max[G{Sp(j,k),KSp(i,j,k,q=all)}] (3)

[0158] Here,

[0159] KSp(i,j,k,q=all) is Figure 8 The known characteristic spectra of all data numbered q in a specific partial region k of a specific layer j associated with category i in the known characteristic spectral group KSp shown.

[0160] In the first operation method M1 described above, the known characteristic spectra KSp(i,j,k=all,q=all) of all partial regions k of a specific layer j are used. In contrast, in the second operation method M2, only the known characteristic spectra KSp(i,j,k,q=all) of partial regions k that are identical to the characteristic spectra Sp(j,k) are used. The other methods of the second operation method M2 are the same as those of the first operation method M1.

[0161] In the second calculation method M2 based on category similarity

[0162] (1) Based on the discrimination data, calculate the similarity, i.e., the local similarity Sm(i,j,k), between the feature spectrum Sp obtained from the output of a specific region k of a specific layer j and all known feature spectra KSp associated with the specific region k of the specific layer j and each category i.

[0163] (2) Solve for the category similarity Sclass(i,j) by obtaining the maximum, average, or minimum value of the local similarity Sm(i,j,k) for multiple partial regions k for each category i.

[0164] (3) The maximum value of the category similarity Sclass(i,j) for multiple categories i is obtained as the similarity value S_value between the feature spectrum Sp and the known feature spectrum group KSp.

[0165] (4) The category corresponding to the largest similarity value S_value across multiple categories is determined as the discriminant category D_class.

[0166] The second operation method M2 can also be used to solve the similarity Sm(i,j,k), Sclass(i,j) and the discrimination result based on relatively simple operations and order.

[0167] The two computational methods M1 to M2 described above are both methods for calculating local similarity and determining the classification based on category similarity for each specific layer i. As mentioned above, in this disclosure, it is possible to... Figure 3 One or more of the multiple vector neuron layers 230 and 240 shown are used as specific layers to calculate local similarity and class-specific similarity, and the class-specific similarity is used to determine the category of the data. When using multiple specific layers, for example, the following method for determining the category can be used.

[0168] Figure 18 This is an illustration of a method for determining a discrimination category using multiple specific layers. In this method, the discrimination category is determined using the specific layer that represents the statistically most desirable decision result among multiple specific layers. Figure 18In the example, ConvVN1 layer 230 and ConvVN2 layer 240 are used as specific layers. First, the following process is performed: For each partial region k of ConvVN1 layer 230, the category whose local similarity Sm(i,j,k) is the maximum value is determined, and the category parameter value i of that category is assigned to each partial region k. Furthermore, the category parameter value i is a value representing the order among multiple categories. The category parameter value i is, for example, a consecutive integer. In this embodiment, the category parameter value i is the same as the category i. The same process is performed for ConvVN2 layer 240: The category whose local similarity Sm(i,j,k) is the maximum value is determined, and the category parameter value i of that category is assigned to each partial region k.

[0169] Alternatively, in the second determination method MM2, for each region k, if there is no difference in the local similarity S between categories (i.e., if the error and variance of multiple categories related to the local similarity S of a certain region k are within a threshold), no category parameter value is assigned to that region k. When solving for the variance of the category parameter values, the region k that has not been assigned a category parameter value is removed to calculate the variance. Therefore, the variance of only the feature portion can be calculated, thus enabling higher accuracy in category discrimination.

[0170] In this determination method, the variance is calculated based on the distribution of the category parameter values ​​i for multiple partial regions k in each specific layer. This variance is the value of the statistical variance for the category parameter values ​​i. Figure 18 In the example, the variance of ConvVN1 layer 230 is 0.14, and the variance of ConvVN2 layer 240 is 0.22. It is estimated that in the aforementioned specific layers 230 and 240, the greater the deviation in the distribution of the category parameter value i, the clearer the determination result. Therefore, the category discrimination result for the specific layer with low variance can be adopted. In other words, the category similarity obtained relative to the specific layer with the smallest variance among multiple specific layers is used to determine the category of the data to be discriminated. According to this determination method, category discrimination can be performed with higher accuracy using multiple specific layers. Thus, when determining the category discrimination result using the category similarity obtained from multiple specific layers, in the first to third embodiments described above, as the local similarity used for creating explanatory statements, it is preferable to use the local similarity of the specific layer from which the category discrimination result is obtained.

[0171] E. Methods for calculating the output vectors of each layer in a machine learning model:

[0172] Figure 3 The computation method for the output of each layer of the machine learning model 200 shown is as follows. Figure 4 The machine learning model 200 shown is the same except for the values ​​of each parameter.

[0173] Each node in PrimeVN layer 220 obtains its vector output by treating the scalar outputs of the 1×1×32 nodes in Conv layer 210 as 32-dimensional vectors and multiplying these vectors by a transformation matrix. This transformation matrix consists of elements of a kernel with a surface size of 1×1 and is updated through learning by machine learning model 200. Furthermore, the processing of Conv layer 210 and PrimeVN layer 220 can be unified and constructed as a single main vector neuron layer.

[0174] When the PrimeVN layer 220 is referred to as the "lower layer L" and the ConvVN1 layer 230 adjacent to it is referred to as the "upper layer L+1", the output of each node of the upper layer L+1 is determined using the following formula.

[0175] Mathematical Formula 1

[0176]

[0177] u j =∑ i v ij (E2)

[0178] a j =F(||u j ||) (E3)

[0179]

[0180] Here,

[0181] M L i It is the output vector of the i-th node in the lower layer L.

[0182] M L+1 j It is the output vector of the j-th node in the upper layer L+1.

[0183] v ij It is the output vector M L+1 j The predicted vector,

[0184] W L ij It is used to determine the output vector M of the lower layer L. L i To calculate the prediction vector v ij The prediction matrix,

[0185] u j It is the prediction vector v ij The sum is linear coupling, which is also the sum vector.

[0186] aj By making the sum vector u j norm |u j The normalization coefficients obtained through normalization are also known as activation values.

[0187] F(X) is the normalization function that normalizes X.

[0188] As a normalization function F(X), it is possible to use, for example, the following equation (E3a) or (E3b).

[0189] Mathematical formula 2

[0190]

[0191]

[0192] Here,

[0193] k is the ordinal number relative to all nodes in the upper layer L+1.

[0194] β is any positive coefficient, i.e., adjustment parameter, for example, β = 1.

[0195] In equation (E3a) above, the sum vector u is obtained by using the Softmax function with respect to all nodes of the upper layer L+1. j norm |u j |Normalization yields the activation value a j On the other hand, in equation (E3b), by summing the vector u... j norm |u j |Divided by the norm associated with all nodes in the upper layer L+1|u j The sum of these values ​​yields the activation value a. j Furthermore, other functions besides equations (E3a) and (E3b) can also be used as the normalization function F(X).

[0196] The ordinal number i in equation (E2) above is assigned for convenience to determine the output vector M of the j-th node in the upper layer L+1. L+1 j The ordinal numbers of the nodes in the lower-level layer L are taken as values ​​from 1 to n. Additionally, the integer n is used to determine the output vector M of the j-th node in the upper-level layer L+1. L+1 j The number of nodes in the lower-level layer L used. Therefore, the integer n is given by the following formula.

[0197] n = Nk × Nc(E5)

[0198] Here, Nk is the surface size of the kernel, and Nc is the number of channels in the PrimeVN layer 220, which is the lower layer. Figure 3In the example, Nk = 5, Nc = 26, therefore, n = 130.

[0199] The kernel used to solve for the output vector of ConvVN1 layer 230 has 130 elements with a surface size of 1×5 and a depth of 26 channels in the lower layer. These elements are the prediction matrix W. L ij Furthermore, to generate the output vectors of the 20 channels of ConvVN1 layer 230, this kernel requires 20 sets. Therefore, the prediction matrix W of the kernel used to solve for the output vectors of ConvVN1 layer 230... L ij The number is 130 × 20 = 2600. The above prediction matrix W L ij Updated through learning from machine learning model 200.

[0200] As can be seen from equations (E1) to (E4) above, the output vector M of each node in the upper layer L+1 L+1 j The solution is obtained through the following calculations.

[0201] (a) Output vector M of each node in the lower layer L L i Multiply by the prediction matrix W L ij And solving for the prediction vector v ij ,

[0202] (b) Solve for the prediction vector v obtained from each node of the lower layer L. ij The sum is linear coupling, that is, the sum vector u. j ,

[0203] (c) By making the sum vector u j norm |u j |Normalization is used to solve for the normalization coefficient, which is also the activation value a. j ,

[0204] (d) sum vector u j Divide by norm |u j | and multiplied by the activation value a j .

[0205] In addition, activation value a j The norm |u is obtained by considering all nodes in the upper layer L+1. j The normalization coefficients are obtained through normalization. Therefore, the activation value a can be considered normalized. jIt is an index representing the relative output intensity of each node among all nodes in the upper layer L+1. The norm used in equations (E3), (E3a), (E3b), and (4) is the L2 norm representing the vector length in a typical example. At this time, the activation value a j Equivalent to output vector M L+1 j The length of the vector. Activation value a j It is only used in equations (E3) and (E4) above, therefore, no output from the node is required. It is also possible to use the activation value a. j The way of outputting to the outside constitutes the upper layer L+1.

[0206] The structure of a vector neural network is almost identical to that of a capsule network; the vector neurons in a vector neural network are equivalent to the capsules in a capsule network. However, the operations used in equations (E1) to (E4) in a vector neural network differ from those used in a capsule network. The biggest difference lies in the following: In a capsule network, the prediction vector v on the right-hand side of equation (E2) is... ij Each factor is multiplied by a weighted sum, which is searched through repeated dynamic path selection. On the other hand, in the vector neural network of this embodiment, the output vector M is obtained by sequentially calculating equations (E1) to (E4) once. L+1 j Therefore, it eliminates the need for repeated dynamic path selection, resulting in faster computation. Furthermore, compared to capsule networks, the vector neural network of this embodiment requires less memory for computation; according to experiments conducted by the inventors of this disclosure, it requires only about 1 / 2 to 1 / 3 of the memory.

[0207] In using nodes with vectors as input and output, vector neural networks (NNs) are similar to capsule networks. Therefore, the advantages of using vector neurons are also shared with capsule networks. Furthermore, in multiple layers (210-250), the characteristics of regions with larger features further up and smaller features further down are the same as in conventional convolutional neural networks (CNNs). Here, "feature" refers to the feature portion included in the input data of the neural network. NNs and capsule networks are superior to conventional CNNs in that the output vector of a node includes spatial information representing the spatial information of the feature represented by that node. That is, the length of the output vector of a node represents the probability of the feature represented by that node, and the vector direction represents the spatial information such as the direction and scale of that feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the positional relationship of each feature. Alternatively, the vector direction of the output vectors of these two nodes can also be said to represent a change in the feature. For example, if it is a node corresponding to a feature of "object," the direction of the output vector can represent a change in the object's fineness, enhancement method, etc. In conventional CNNs, it is believed that pooling processing causes the spatial information of features to disappear. As a result, vector neural networks and capsule networks have the advantage of superior performance in recognizing input data compared to conventional convolutional neural networks.

[0208] The advantages of vector neural networks can also be considered as follows: The output vectors of nodes represent the features of the input data as coordinates in a continuous space. Therefore, the output vectors can be evaluated in a way that features are similar if their directions are similar. Furthermore, they have the advantage of being able to interpolate and determine features even if the features included in the input data are not covered by the teaching data. On the other hand, conventional convolutional neural networks suffer from disordered compression due to pooling, and therefore have the disadvantage of not being able to represent the features of the input data as coordinates in a continuous space.

[0209] The outputs of each node in ConvVN2 layer 240 and ClassVN layer 250 are also determined using the same equations (E1) to (E4) described above; therefore, detailed explanations are omitted. The resolution of the top-level ClassVN layer 250 is 1×1, and the number of channels is n1.

[0210] The output of ClassVN layer 250 is converted into multiple decision values ​​Class0 to Class2 relative to known classes. These decision values ​​are typically normalized using the Softmax function. Specifically, for example, based on the output vector of each node in ClassVN layer 250, the vector length of the output vector is calculated, and an operation is performed to normalize the vector length of each node using the Softmax function, thereby obtaining the decision value relative to each class. As described above, the activation value a obtained in equation (E3) above... j It is equivalent to the output vector M L+1 j The value of the vector length is normalized. Therefore, it can also be the output of the activation value 'a' of each node in ClassVN layer 250. j And keep it as is and use it as the judgment value relative to each category.

[0211] In the above-described embodiments, the machine learning model 200 uses a vector neural network that solves for the output vector through the operations of equations (E1) to (E4) above. However, it is also possible to use the capsule network disclosed in U.S. Patent No. 5,210,798 and International Publication No. 2009 / 083553 instead.

[0212] Other implementation methods:

[0213] This disclosure is not limited to the embodiments described above and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following aspects. The technical features in the above embodiments corresponding to the technical features in the various aspects described below can be appropriately replaced or combined to solve part or all of the problems of this disclosure, or to achieve part or all of the effects of this disclosure. In addition, technical features that are not necessary to be described in this specification can be appropriately deleted.

[0214] (1) According to a first aspect of this disclosure, a method is provided for determining the category of discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The method includes the steps of: (a) preparing a set of known feature spectra for each of more than one category, obtained from the output of a specific layer of the prepared multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) performing category discrimination processing on the discrimination data using the machine learning model and the known feature spectra. Step (b) includes the steps of: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known feature spectra associated with each category; (b3) creating a descriptive statement for the category discrimination result of the discrimination data based on the similarity; and (b4) outputting the descriptive statement.

[0215] According to this method, explanatory statements for the category classification results are created and output using similarity related to feature vectors, so that users can understand the basis for the category classification results.

[0216] (2) In the above method, the specific layer has a structure in which vector neurons arranged on a plane defined by the first axis and the second axis are configured as multiple channels along a third axis in a direction different from the two axes. In the specific layer, the region defined by the plane position defined by the position of the first axis and the position of the second axis and including the multiple channels along the third axis is called a partial region. At this time, the feature spectrum may also be any one of the following feature spectra obtained for each of the multiple partial regions included in the specific layer: (i) a first feature spectrum in which multiple element values ​​of the output vectors of each vector neuron included in the partial region are arranged along the multiple channels along the third axis; (ii) a second feature spectrum obtained by multiplying the element values ​​of the first feature spectrum by a normalization coefficient equivalent to the vector length of the output vector; and (iii) a third feature spectrum in which the normalization coefficient is arranged along the multiple channels along the third axis.

[0217] According to this method, similarity can be solved using any of the three feature spectra obtained from the output vector of a specific layer.

[0218] (3) In the above method, the similarity obtained in the above process (b2) may also be the local similarity calculated for each of the above partial regions.

[0219] According to this method, explanatory statements can be created based on the local similarity calculated for a portion of each specific layer.

[0220] (4) In the above method, when Ns and Nd are integers greater than 2, Nd ≤ Ns, and Nc is an integer greater than 1, the above step (b3) includes: a first step, creating Nd table input data less than the number of gray levels of the local similarity based on Ns local similarities of at least a portion of the multiple partial regions included in the above specific layer; a second step, obtaining Nc strings output from the above string lookup table by inputting the above Nd table input data into a pre-prepared string lookup table; and a third step, creating the above description statement by using the above Nc strings in a description statement template with Nc string boxes.

[0221] According to this method, descriptive statements can be created using string lookup tables and descriptive statement templates.

[0222] (5) In the above method, the integer Nd may be less than the integer Ns, and the first step includes the following steps: solving Nd representative similarities by grouping the Ns local similarities into Nd groups and solving the representative value of the local similarity of each group; and creating the Nd table input data by reducing the gray level of the Nd representative similarities.

[0223] According to this method, descriptive statements can be created based on representative similarities obtained by solving for representative values ​​of local similarities in partial regions.

[0224] (6) In the above method, the local similarity for each part of the region may be calculated as either a first type of local similarity or a second type of local similarity. The first type of local similarity is the similarity between the feature spectrum obtained from the output of the part of the specific layer based on the discrimination data and all the known feature spectrum groups associated with the specific layer and each category. The second type of local similarity is the similarity between the feature spectrum obtained from the output of the part of the specific layer based on the discrimination data and all the known feature spectrum groups associated with the part of the specific layer and each category.

[0225] According to this method, local similarity can be solved through relatively simple calculations.

[0226] (7) In the above method, the above step (b4) may also include the following steps: for two or more categories among the multiple categories that can be distinguished by the above machine learning model, display a list of discrimination results arranged with the above category discrimination results and the above explanatory statements.

[0227] According to this method, the basis for class discrimination results related to two or more categories can be known.

[0228] (8) According to a second aspect of this disclosure, an information processing apparatus is provided that uses a vector neural network-type machine learning model having multiple vector neuron layers to perform category discrimination processing for discrimination data. The information processing apparatus includes: a memory storing the machine learning model; and a processor executing operations using the machine learning model. The processor is configured to perform the following processes: (a) for each of more than one category, reading from the memory a known feature spectrum set obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) performing the category discrimination processing for the discrimination data using the machine learning model and the known feature spectrum set. Process (b) includes the following processes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known feature spectrum set associated with each category; and (b3) creating a descriptive statement for the category discrimination result of the discrimination data based on the similarity; and (b4) outputting the descriptive statement.

[0229] According to this information processing device, explanatory statements of the category discrimination results are created and output using similarity related to feature vectors, so that users can know the basis of the category discrimination results.

[0230] (9) According to a third aspect of this disclosure, a computer program is provided that enables a processor to perform category discrimination processing of discrimination data using a vector neural network-type machine learning model having multiple vector neuron layers. The computer program enables the processor to perform the following processes: (a) for each of more than one category, reading from memory a set of known feature spectra obtained from the output of a specific layer among the multiple vector neuron layers when multiple teaching data are input into the machine learning model; and (b) using the machine learning model and the known feature spectra to perform category discrimination processing of the discrimination data. Process (b) includes the following processes: (b1) calculating a feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) calculating the similarity between the feature spectrum and the known feature spectra associated with each category; (b3) creating a descriptive statement for the category discrimination result of the discrimination data based on the similarity; and (b4) outputting the descriptive statement.

[0231] According to this computer program, explanatory statements for the category classification results are created and output using similarity related to feature vectors, so that users can know the basis for the category classification results.

[0232] This disclosure can also be implemented in various other forms besides those described above. For example, it can be implemented as a computer program for implementing the functions of the category discrimination device, a non-transitory storage medium on which the computer program is recorded, etc.

Claims

1. An information processing method, characterized in that, The information processing method uses a vector neural network-type machine learning model with multiple layers of vector neurons to determine the category of the data to be classified. The method includes the following steps: (a) For each of the more than one categories, prepare a set of known feature spectra from the output of a specific layer in the plurality of vector neuron layers when multiple teaching data are input into the machine learning model, the specific layer being a vector neuron layer used for similarity calculation; (b) Using the machine learning model and the known feature spectral set to perform category discrimination processing on the discriminant data; as well as (c) The printing process of the medium, The process (b) includes the following processes: (b1) Calculate the feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) Calculate the similarity between the characteristic spectrum and the known characteristic spectrum group associated with each category; (b3) Based on the similarity, create descriptive statements for the category determination results of the determined data; and (b4) Output the aforementioned explanatory statement. The step (b4) includes the following steps: for two or more categories that can be distinguished by the machine learning model, displaying a list of discrimination results arranged with the category discrimination results and the explanatory statements. The process (c) includes the following processes: (c1) After the discrimination result list is displayed, receive the selection of the type of printing medium; (c2) Select the printing settings according to the received selection of the type of printing medium; And (c3) perform printing according to the selected printing settings. The category determination result displays parameters representing the category, the category name, and the category probability, where the category probability is the probability that the determined data corresponds to that category. The category probability can be a determination value output from the output layer of the machine learning model. The discrimination data is the spectroscopic data of the printing medium, the teaching data is a collection of spectroscopic data of multiple printing media, and the explanatory statement is a statement based on the similarity of each wavelength band contained in the spectroscopic data.

2. The information processing method according to claim 1, characterized in that, The specific layer has a structure in which vector neurons, configured as multiple channels, are arranged in a plane defined by the first and second axes along a third axis in a direction different from the two axes. In the specific layer, when the region defined by the planar position determined by the position of the first axis and the position of the second axis, and including the plurality of channels along the third axis, is referred to as a partial region, The characteristic spectrum is any one of the following characteristic spectra obtained for each of the multiple partial regions included in the specific layer: (i) A first feature spectrum is formed by spreading multiple element values ​​of the output vectors of each vector neuron included in the region across the multiple channels along the third axis; (ii) A second characteristic spectrum obtained by multiplying the element values ​​of the first characteristic spectrum by a normalization coefficient equivalent to the vector length of the output vector; as well as (iii) The normalization coefficients are spread across the plurality of channels arranged along the third axis to form a third characteristic spectrum.

3. The information processing method according to claim 2, characterized in that, The similarity obtained in step (b2) is the local similarity calculated for each of the said partial regions.

4. The information processing method according to claim 3, characterized in that, When Ns and Nd are both integers greater than 2, Nd ≤ Ns, and Nc is an integer greater than 1, The process (b3) includes: The first step is to create Nd tables of input data with fewer grayscale values ​​than the local similarities of Ns partial regions for at least a portion of the multiple partial regions included in the specific layer. The second step involves inputting the Nd input data into a pre-prepared string lookup table to obtain Nc strings output from the string lookup table; and The third step involves creating the description statement by using the Nc strings in a description statement template with Nc string boxes.

5. The information processing method according to claim 4, characterized in that, The integer Nd is less than the integer Ns. The first process includes the following steps: The Nd representative similarities are solved by grouping the Ns local similarities into Nd groups and solving for the representative value of the local similarity of each group; and The Nd tables of input data are created by reducing the number of grayscale values ​​representing similarity among the Nd tables.

6. The information processing method according to any one of claims 3 to 5, characterized in that, The local similarity for each region is calculated as either a first type of local similarity or a second type of local similarity. The first type of local similarity is the similarity between the feature spectrum obtained from the output of that region of the specific layer based on the discrimination data and all known feature spectrum groups associated with the specific layer and each category. The second type of local similarity is the similarity between the feature spectrum obtained from the output of that portion of the specific layer based on the discrimination data and all known feature spectrum groups associated with that portion of the specific layer and each category.

7. An information processing device, characterized in that, The information processing device comprises: a vector neural network-type machine learning model with multiple layers of vector neurons to perform category discrimination processing for classifying data; and a vector neural network-type machine learning model with multiple layers of vector neurons. Memory, storing the machine learning model; and The processor executes the computations using the machine learning model. The processor is configured to perform the following processes: (a) For each of more than one category, read from the memory a known feature spectrum obtained from the output of a specific layer in the plurality of vector neuron layers when multiple teaching data are input into the machine learning model, the specific layer being a vector neuron layer used for similarity calculation; (b) Using the machine learning model and the known feature spectral set to perform category discrimination processing on the discriminant data; as well as (c) Printing treatment of the medium, The process (b) includes the following processes: (b1) Calculate the feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) Calculate the similarity between the characteristic spectrum and the known characteristic spectrum group associated with each category; (b3) Based on the similarity, create descriptive statements for the category determination results of the determined data; and (b4) Output the aforementioned explanatory statement. The process (b4) includes the following: for two or more categories that can be distinguished by the machine learning model, displaying a list of discrimination results arranged with the category discrimination results and the explanatory statements. The process (c) includes the following processes: (c1) After the discrimination result list is displayed, receive the selection of the type of printing medium; (c2) Select the printing settings according to the received selection of the type of printing medium; And (c3) perform printing according to the selected printing settings. The category determination result displays parameters representing the category, the category name, and the category probability, where the category probability is the probability that the determined data corresponds to that category. The category probability can be a determination value output from the output layer of the machine learning model. The discrimination data is the spectroscopic data of the printing medium, the teaching data is a collection of spectroscopic data of multiple printing media, and the explanatory statement is a statement based on the similarity of each wavelength band contained in the spectroscopic data.

8. A recording medium, characterized in that, The computer program contains a computer program that causes a processor to perform class discrimination processing to determine the category of the data using a vector neural network-type machine learning model with multiple layers of vector neurons. The computer program causes the processor to perform the following processing: (a) For each of more than one category, read from memory a known feature spectrum obtained from the output of a specific layer in the plurality of vector neuron layers when multiple teaching data are input into the machine learning model, the specific layer being a vector neuron layer used for similarity calculation; (b) Using the machine learning model and the known feature spectral set to perform category discrimination processing on the discriminant data; as well as (c) Printing treatment of the medium, The process (b) includes the following processes: (b1) Calculate the feature spectrum from the output of the specific layer based on the input of the discrimination data to the machine learning model; (b2) Calculate the similarity between the characteristic spectrum and the known characteristic spectrum group associated with each category; (b3) Based on the similarity, create descriptive statements for the category determination results of the determined data; and (b4) Output the aforementioned explanatory statement. The process (b4) includes the following: for two or more categories that can be distinguished by the machine learning model, displaying a list of discrimination results arranged with the category discrimination results and the explanatory statements. The process (c) includes the following processes: (c1) After the discrimination result list is displayed, receive the selection of the type of printing medium; (c2) Select the printing settings according to the received selection of the type of printing medium; And (c3) perform printing according to the selected printing settings. The category determination result displays parameters representing the category, the category name, and the category probability, where the category probability is the probability that the determined data corresponds to that category. The category probability can be a determination value output from the output layer of the machine learning model. The discrimination data is the spectroscopic data of the printing medium, the teaching data is a collection of spectroscopic data of multiple printing media, and the explanatory statement is a statement based on the similarity of each wavelength band contained in the spectroscopic data.

Images