Method for performing classification processing using machine learning model, information processing device, and computer program

Abstract

To provide a technique capable of executing class classification processing at high speed.SOLUTION: A method of performing class classification processing of data to be classified includes the steps of: (a) preparing N machine learning models; (b) preparing a known feature vector group obtained from the output of at least one specific layer among a plurality of vector neuron layers when a plurality of sets of teacher data is input to the N machine learning models; and (c) using a selected machine learning model selected from the N machine learning models to calculate, for each class, the similarity between the group of known feature vectors and the feature vector obtained from the output of the specific layer when data to be classified are input to the selected machine learning model so as to determine the class of the data to be classified by use of the similarity.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a method, an information processing device, and a computer program for performing a class classification process using a machine learning model. [Background technology]

[0002] Patent Document 1 discloses a technique for classifying input data using a VAE (Variational Autoencoder). In this technique, a VAE is constructed for each class to be distinguished, and by comparing the input data with the data output from one VAE, it is determined whether the input data corresponds to the class corresponding to that VAE. If the input data does not correspond to the class corresponding to that VAE, another VAE is used to perform the classification again. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-204321 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned technology requires building a VAE for each class, which results in a large amount of processing time and requires a lot of calculation time. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a method for performing a class classification process on data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers, the method including: (a) preparing N machine learning models, where N is an integer equal to or greater than 1, each of the N machine learning models configured to classify input data into one of multiple classes and having at least one class different from other machine learning models among the N machine learning models; (b) preparing a set of known feature spectra obtained from the output of at least one specific layer among the multiple vector neuron layers when multiple pieces of training data are input to the N machine learning models; and (c) using a selected machine learning model selected from the N machine learning models, calculating a similarity between the set of known feature spectra and a feature spectrum obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model, for each class, and determining the class of the data to be classified using the similarity. The way computers do it .

[0006] According to a second aspect of the present disclosure, there is provided an information processing device that performs a class classification process on data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. The information processing device includes a memory that stores the machine learning model and one or more processors that perform calculations using the machine learning model. The one or more processors perform the following steps: (a) a process of preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models being configured to classify input data into one of multiple classes and having at least one class that is different from other machine learning models among the N machine learning models; and (b) a process of generating known features obtained from the output of at least one specific layer among the multiple vector neuron layers when multiple pieces of training data are input to the N machine learning models. Spectrum(c) using a selected machine learning model selected from the N machine learning models, to calculate features obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model. Spectrum and the known feature Spectrum The similarity to the group is calculated for each class, and the class of the classified data is determined using the similarity.

[0007] According to a third aspect of the present disclosure, there is provided a computer program that causes one or more processors to execute a class classification process for data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. The computer program includes: (a) a process for preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models configured to classify input data into one of multiple classes and having at least one class different from other machine learning models among the N machine learning models; and (b) a process for preparing known features obtained from the output of at least one specific layer among the multiple vector neuron layers when multiple training data are input to the N machine learning models. Spectrum (c) using a selected machine learning model selected from the N machine learning models, to calculate features obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model. Spectrum and the known feature Spectrum and calculating a similarity to a group for each class, and determining a class of the classified data using the similarity. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Figure 2] FIG. [Figure 3] FIG. 1 is an explanatory diagram showing the configuration of a first machine learning model. [Figure 4]FIG. 10 is an explanatory diagram showing the configuration of a second machine learning model. [Figure 5] 10 is a flowchart showing the processing procedure of a preparation step. [Figure 6] FIG. [Figure 7] FIG. 2 is an explanatory diagram showing the configuration of a group of known characteristic spectra. [Figure 8] 10 is a flowchart showing the processing procedure of a classification step. [Figure 9] 10 is a flowchart showing the processing steps of a machine learning model update process. [Figure 10] 10 is a flowchart showing the processing procedure of a class deletion step. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. Implementation: FIG. 1 is a block diagram illustrating the functions of a printer 10 as an information processing device according to one embodiment. The printer 10 includes a control device 20, a printing mechanism 30, a print medium holder 40, and a spectrometer 50. The print medium holder 40 holds the print medium PM. The spectrometer 50 performs spectroscopic measurement of the print medium PM held in the print medium holder 40 to obtain spectral data. The spectrometer 50 includes, for example, a tunable interference spectral filter and a monochrome image sensor. The spectral data obtained by the spectrometer 50 is used as input data or classified data for a machine learning model (described below). As described below, the control device 20 uses the machine learning model to perform classification processing on the spectral data and classify the print medium PM into one of several classes. The "class of the print medium PM" refers to the type of print medium PM. The control device 20 controls the printing mechanism 30 to print under appropriate printing conditions according to the type of print medium PM. Note that the present disclosure is applicable to various information processing devices other than the printer 10.

[0010] 2 is a block diagram showing the functions of the control device 20. The control device 20 includes a processor 110, a memory 120, an interface circuit 130, and an input device 140, a display device 150, and an audio output device 160 connected to the interface circuit 130. For example, but not limited to, the processor 110 not only performs the processes described in detail below, but also displays and / or outputs audio data obtained by the processes and data generated during the processes on the display device 150 and / or audio output device 160. The spectrometer 50 and the printing mechanism 30 are also connected to the interface circuit 130.

[0011] The processor 110 may be composed of one or more processors and functions as a print control unit 112 that controls the printing mechanism 30 and as a classification processing unit 114 that performs classification processing on input data. These units 112 and 114 are implemented by the processor 110 executing computer programs stored in the memory 120. However, these units 112 and 114 may also be implemented as hardware circuits. The term "processor" used in this specification also includes such hardware circuits. The one or more processors that perform classification processing may also be processors included in one or more remote computers connected to the printer 10 via a network. The memory 120 stores multiple machine learning models 201 and 202, multiple sets of training data TD1 and TD2, multiple sets of known feature spectra KS1 and KS2, and data to be classified Di. The machine learning models 201 and 202 are used in calculations by the classification processing unit 114. Configuration examples and operations of the machine learning models 201 and 202 will be described later. The training data groups TD1 and TD2 are sets of labeled spectral data used to train the machine learning models 201 and 202. The known feature spectrum groups KS1 and KS2 are sets of feature spectra obtained when the training data groups TD1 and TD2 are input again into the trained machine learning models 201 and 202. The feature spectra will be described later. The classified data Di is the spectral data of new printing media PM that will be the target of the class classification process.

[0012] 3 is an explanatory diagram showing the configuration of a first machine learning model 201. This machine learning model 201 includes, in order from the input data IM side, a convolutional layer 211, a primary vector neuron layer 221, a first convolutional vector neuron layer 231, a second convolutional vector neuron layer 241, and a classification vector neuron layer 251. Of these five layers 211 to 251, the convolutional layer 211 is the lowest layer, and the classification vector neuron layer 251 is the highest layer. In the following description, the layers 211 to 251 are also referred to as the "Conv layer 211," the "PrimeVN layer 221," the "ConvVN1 layer 231," the "ConvVN2 layer 241," and the "ClassVN layer 251," respectively.

[0013] In this embodiment, the input data IM is spectral data, and therefore is one-dimensionally arranged data. For example, the input data IM is data obtained by extracting 36 representative values ​​at intervals of 10 nm from spectral data in the range of 380 nm to 730 nm. However, two-dimensionally arranged data such as an image may also be used as the input data IM.

[0014] 3, two convolution vector neuron layers 231 and 241 are used, but the number of convolution vector neuron layers is arbitrary, and a convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0015] 3 further includes a similarity calculation unit 261 that generates similarities. The similarity calculation unit 261 can calculate similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN, which will be described later, from the outputs of the ConvVN1 layer 231, the ConvVN2 layer 241, and the ClassVN layer 251. However, the similarity calculation unit 261 may be omitted.

[0016] The configuration of each of the layers 211 to 251 can be described as follows. <Description of the configuration of the first machine learning model 201> ·Conv layer 211: Conv[32,6,2] ·PrimeVN layer 221: PrimeVN[26,1,1] ·ConvVN1 layer 231:ConvVN1[20,5,2] ·ConvVN2 layer 241:ConvVN2[16,4,1] ·ClassVN layer 251:ClassVN[n1,3,1] Vector dimension VD: VD=16 In the description of each of these layers 211 to 251, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, kernel size, and stride. For example, the layer name of the Conv layer 211 is "Conv," the number of channels is 32, the kernel size is 1 x 6, and the stride is 2. In FIG. 3, these descriptions are shown below each layer. The hatched rectangles drawn in each layer represent the kernels used when calculating the output vectors of the adjacent higher layer. In this embodiment, since the input data IM is data of a one-dimensional array, the kernels also have one-dimensional arrays. Note that the parameter values ​​used in the description of each of the layers 211 to 251 are merely examples and can be changed as desired.

[0017] The Conv layer 211 is a layer composed of scalar neurons. The other four layers 221 to 251 are layers composed of vector neurons. A vector neuron is a neuron that uses vectors as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0018] FIG. 3 shows the first axis x and second axis y that define the planar coordinates of the node array for the Conv layer 211, and the third axis z that represents depth. It also shows that the sizes of the Conv layer 211 in the x, y, and z directions are 1, 16, and 32. The sizes in the x and y directions are called "resolution." In this embodiment, the resolution in the x direction is always 1. The size in the z direction is the number of channels. These three axes x, y, and z are also used in other layers as coordinate axes that indicate the position of each node. However, in FIG. 3, these axes x, y, and z are omitted from illustration in layers other than the Conv layer 211.

[0019] As is well known, the resolution W1 in the y direction after convolution is given by the following equation: W1=Ceil{(W0-Wk+1) / S} (1) Here, W0 is the resolution before convolution, Wk is the kernel size, S is the stride, and Ceil{X} is a function that rounds up the decimal point of X. The resolution of each layer shown in FIG. 3 is an example in which the resolution in the y direction of the input data IM is set to 36, and the actual resolution of each layer is changed appropriately depending on the size of the input data IM.

[0020] The ClassVN layer 251 has n1 channels. In the example of FIG. 3, n1=10. These channels output judgment values ​​Class1-1 to Class1-10 for multiple known classes. The class with the largest value among these judgment values ​​Class1-1 to Class1-10 corresponds to the class to which the input data IM belongs. Generally, n1 is an integer greater than or equal to 1 and represents the number of known classes that can be classified using the first machine learning model 201. In any one machine learning model, it is preferable to set an upper limit nmax to the number n1 of known classes that can be classified. In the example of FIG. 3, the number n1 of known classes is equal to 10, which is the upper limit nmax. The upper limit nmax may be a theoretical upper limit that can satisfy a predetermined classification accuracy, or it may be an operational upper limit that is smaller than the theoretical upper limit. Essentially, the operational upper limit provides higher classification accuracy than the theoretical upper limit. Furthermore, the upper limit nmax can be set independently for the multiple machine learning models 201 and 202, and can be set to different values ​​for each.

[0021] 4 is an explanatory diagram showing the configuration of the second machine learning model 202. Like the first machine learning model 201, this machine learning model 202 has a Conv layer 212, a PrimeVN layer 222, a ConvVN1 layer 232, a ConvVN2 layer 242, a ClassVN layer 252, and a similarity calculation unit 262.

[0022] The configuration of each of the layers 212 to 252 can be described as follows. <Description of the configuration of the second machine learning model 202> ·Conv layer 212: Conv[32,6,2] ·PrimeVN layer 222: PrimeVN[26,1,1] ·ConvVN1 layer 232:ConvVN1[20,5,2] ·ConvVN2 layer 242:ConvVN2[16,4,1] ·ClassVN layer 252:ClassVN[n2,3,1] Vector dimension VD: VD=16

[0023] As can be seen by comparing FIGS. 3 and 4, among layers 212 to 252 of second machine learning model 202, the four lowest layers 212 to 242 have the same configuration as layers 211 to 241 of first machine learning model 201. On the other hand, top layer 252 of second machine learning model 202 differs from top layer 251 of first machine learning model 201 only in the number of channels. In the example of FIG. 4, ClassVN layer 252 has n2 channels, where n2=2. These channels output judgment values ​​Class2-1 to Class2-2 for multiple known classes. In second machine learning model 202, it is preferable to set the number of known classes n2 to the same upper limit value nmax as in first machine learning model 201. In the example of FIG. 4, the number of known classes n2 is smaller than the upper limit value nmax. In this case, second machine learning model 202 is expressed as having a number of classes less than the upper limit value nmax. However, the upper limit nmax of the number of classes can also be set to different values ​​for the multiple machine learning models 201 and 202.

[0024] The second machine learning model 202 is configured to have at least one known class that is different from any of the known classes of the first machine learning model 201. Furthermore, since the first machine learning model 201 and the second machine learning model 202 can classify different classes, the values ​​of the kernel elements are also different from each other. In the present disclosure, when N is an integer equal to or greater than 2, any one of the N machine learning models is configured to have at least one known class that is different from any of the known classes of the other machine learning models.

[0025] FIG. 5 is a flowchart showing the processing steps for preparing a machine learning model. In step S110, a user creates a machine learning model to be used in classification processing and sets its parameters. In this embodiment, two machine learning models 201 and 202 shown in FIGS. 3 and 4 are created and their parameters are set. However, in step S110, only one machine learning model may be created, or three or more machine learning models may be created. In step S120, the classification processing unit 114 performs training on the machine learning models 201 and 202 using the training data sets TD1 and TD2. Each piece of training data is assigned a label in advance. For example, since the first machine learning model 201 has 10 known classes, the training data is assigned a label with a value between 1 and 10. When training is completed, the trained machine learning models 201 and 202 are stored in the memory 120.

[0026] In step S130, the classification processing unit 114 re-inputs the plurality of training data groups TD1 and TD2 into the trained machine learning models 201 and 202 to generate known feature spectrum groups KS1 and KS2. The known feature spectrum groups KS1 and KS2 are sets of feature spectra, which will be described below. The following mainly describes a method for generating the known feature spectrum group KS1 associated with the machine learning model 201.

[0027] FIG. 6 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 201. Here, the feature spectrum Sp obtained from the output of the ConvVN1 layer 231 will be described. The horizontal axis in FIG. 6 is the spectral position represented by a combination of the element number ND of the output vector of a node at one planar position (x, y) of the ConvVN1 layer 231 and the channel number NC. In this embodiment, since the vector dimension of the node is 16, the element number ND of the output vector is 16, ranging from 0 to 15. Furthermore, since the number of channels in the ConvVN1 layer 231 is 20, the channel number NC is 20, ranging from 0 to 19.

[0028] The vertical axis of Fig. 6 represents the feature value C at each spectral position. V In this example, the feature value C V is the value of each element of the output vector V ND Note that the feature value C V As the value of each element of the output vector V ND Alternatively, the activation value may be used as is. In the latter case, the feature value C included in the feature spectrum Sp is used as is. V The number of nodes is equal to the number of channels, which is 20. The activation value is a value corresponding to the vector length of the output vector of the node.

[0029] The number of feature spectra Sp obtained from the output of the ConvVN1 layer 231 for one piece of input data is equal to the number of planar positions (x, y) of the ConvVN1 layer 231, that is, 1 × 6 = 6. Similarly, for one piece of input data, three feature spectra Sp are obtained from the output of the ConvVN2 layer 241, and one feature spectrum Sp is obtained from the output of the ClassVN layer 251.

[0030] When the training data group TD1 is input again to the trained machine learning model 201, the similarity calculation unit 261 calculates the feature spectrum Sp shown in FIG. 6 and registers it in the known feature spectrum group KS1.

[0031] 7 is an explanatory diagram showing the configuration of the known feature spectrum set KS1. In this example, the known feature spectrum set KS1 includes a known feature spectrum set KS1_ConvVN1 obtained from the output of the ConvVN1 layer 231, a known feature spectrum set KS1_ConvVN2 obtained from the output of the ConvVN2 layer 241, and a known feature spectrum set KS1_ConvVN1 obtained from the output of the ClassVN layer 251.

[0032] Each record in the known feature spectrum group KS1_ConvVN1 includes a record number, a layer name, a label Lb, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in FIG. 6 obtained in response to the input of training data. In the example of FIG. 7, by inputting the training data group TD1 into the trained machine learning model 201, known feature spectra KSp associated with each label Lb value are generated and registered from the output of the ConvVN1 layer 231. For example, N1_1max known feature spectra KSp are registered in association with label Lb=1, N1_2max known feature spectra KSp are registered in association with label Lb=2, and N1_n1max known feature spectra KSp are registered in association with label Lb=n1. N1_1max, N1_2max, and N1_n1max are each an integer greater than or equal to 2. As described above, each label Lb corresponds to a different known class. Therefore, it can be understood that each known feature spectrum KSp in the known feature spectrum group KS1_ConvVN1 is registered in association with one of a plurality of known classes, as is the case with the other known feature spectrum groups KS1_ConvVN2 and KS1_ConvVN1.

[0033] The training data used in step S130 does not need to be the same as the training data groups TD1 and TD2 used in step S120. However, if some or all of the training data groups TD1 and TD2 used in step S120 are used in step S130, there is an advantage in that there is no need to prepare new training data.

[0034] 8 is a flowchart showing the processing steps of the classification process using a trained machine learning model. In step S210, the classification processing unit 114 determines whether the print medium holder 40 has been opened or closed, and waits until it has been opened or closed. This determination is made in response to an output signal from an open / close switch (not shown) installed on the print medium holder 40. If the print medium holder 40 has been opened or closed, the process proceeds to step S220, where the classification processing unit 114 causes the spectrometer 50 to perform spectroscopic measurement of the print medium PM, thereby acquiring spectral data. This spectral data becomes the classified data Di, which is the target of the classification process.

[0035] In step S230, the class classification processing unit 114 selects one of the existing trained machine learning models 201, 202. The machine learning model selected in step S230 is referred to as the "selected machine learning model." In the following description, it is first assumed that the first machine learning model 201 is selected as the selected machine learning model.

[0036] In step S240, the selected machine learning model 201 is used to calculate the similarity with the known feature spectrum group, and the class of the classified data Di is determined based on the similarity. Specifically, the similarity calculation unit 261 of the selected machine learning model 201 calculates the similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN with the known feature spectrum group KS1 for each class from the outputs of the ConvVN1 layer 231, the ConvVN2 layer 241, and the ClassVN layer 251. The following describes a method for calculating the class-specific similarity S1_ConvVN1 from the output of the ConvVN1 layer 231 of the selected machine learning model 201.

[0037] The similarity S1_ConvVN1 can be calculated using, for example, the following formula. S1_ConvVN1(Class)=max[G{Sp(i,j),KSp(Class,k)}] Here, "Class" is an ordinal number for multiple classes, G{a,b} is a function for calculating the similarity between a and b, Sp(i,j) is the feature spectrum at all planar positions (i,j) obtained according to the classified data Di, KSp(Class,k) is all known feature spectra associated with the ConvVN1 layer 231 and a specific "Class," k is the ordinal number of the known feature spectrum, and max[X] is a logical operation that takes the maximum value of X. That is, the similarity S1_ConvVN1 is the maximum similarity calculated between each of the feature spectra Sp(i,j) at all planar positions (i,j) of the ConvVN1 layer 231 and all of the known feature spectra KSp(k) corresponding to a specific class. Such a similarity S1_ConvVN1 is calculated for each of multiple classes corresponding to multiple labels Lb. The similarity S1_ConvVN1 represents the degree to which the classified data Di is similar to the features of each class.

[0038] Similarities S1_ConvVN2 and S1_ClassVN relating to the outputs of the ConvVN2 layer 241 and the ClassVN layer 251 are also generated in the same manner as similarity S1_ConvVN1. Note that it is not necessary to generate all three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN, but it is preferable to generate one or more of them. In this disclosure, the layer used to generate the similarity is also referred to as a "specific layer."

[0039] These similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN represent the degree to which the classified data Di resembles the characteristics of each class, so the class of the classified data Di can be determined using at least one of these similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN. For example, if all of the three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN related to a certain class are equal to or greater than a predetermined threshold, it can be determined that the classified data Di belongs to that class. On the other hand, if at least one of the three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN related to a certain class is less than the threshold, it can be determined that the classified data Di does not belong to that class. If this method results in the classified data Di not belonging to any of the classes associated with the known feature spectra obtained from the machine learning model 201, the classified data Di is determined to belong to a class unknown to the machine learning model 201. In another embodiment, if a predetermined number or more of the three similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN related to a certain class are equal to or greater than a threshold, the classified data Di may be determined to belong to that class. In general, if a predetermined number of similarities among multiple similarities generated from the outputs of multiple specific layers are equal to or greater than a predetermined threshold, the classified data Di may be determined to belong to that class.

[0040] In the class determination process described above, the class of the classified data Di is determined using only similarity. However, instead, the class of the classified data Di may be determined using similarity and the judgment values ​​Class1-1 to Class1-10 of the output layer of the selected machine learning model 201. In the latter case, if the class determined from the similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN matches the class determined from the judgment values ​​Class1-1 to Class1-10, it can be determined that the classified data Di belongs to that class. Furthermore, if the class determined from the similarities S1_ConvVN1, S1_ConvVN2, and S1_ClassVN does not match the class determined from the class judgment values ​​Class1-1 to Class1-10, it can be determined that the classified data Di belongs to an unknown class. However, from the perspective of simplifying the calculations, it is preferable to determine the class using only similarity.

[0041] In this way, in step S240, it is determined to which of the multiple classes of the selected machine learning model 201 the classified data Di corresponds. That is, in the example of FIG. 3, it is determined to which of the 10 known classes the classified data Di corresponds. At this time, if the classified data Di does not correspond to any of the 10 known classes, it is determined that the classified data Di belongs to an unknown class. Note that the processing of step S240 may be executed by the class classification processing unit 114.

[0042] If the classified data Di is determined to be of a known class in step S240 described above, the process proceeds from step S250 to step S280, where the print control unit 112 executes printing using print settings suitable for that known class, and the process in Fig. 8 ends. On the other hand, if the classified data Di is determined to be of an unknown class, the process proceeds from step S250 to step S260.

[0043] In step S260, the classification processing unit 114 determines whether any machine learning models that have not been selected are present among the multiple machine learning models 201, 202. If an unselected machine learning model is present, the process returns to step S230, where the next machine learning model is selected. On the other hand, if no unselected machine learning models are present, the process proceeds to step S270, where it is determined whether a class corresponding to the classified data Di should be added. This determination may be made by the classification processing unit 114 in response to a query from the user regarding whether a class needs to be added. If it is determined that a class corresponding to the classified data Di should be added, the process proceeds to step S300, where a machine learning model update process is performed. The details of step S300 will be described later. On the other hand, if it is determined that a class corresponding to the classified data Di does not need to be added, the classification processing of FIG. 8 ends. In this case, it may be impossible to print using print settings suitable for the classified data Di. However, printing may be performed using print settings for another print medium.

[0044] In the above-described steps S230 to S260, the class of the classified data Di is determined by sequentially selecting the multiple machine learning models 201, 202 one by one. However, instead, the class of the classified data Di can be determined by simultaneously using the multiple machine learning models 201, 202. In the latter method, the two machine learning models 201, 202 are simultaneously used to perform classification processing on the same classified data Di in parallel, and the classification processing unit 114 integrates the processing results. However, if the machine learning models are selected one by one and classification processing is performed, the class of the classified data Di may be determined more quickly.

[0045] FIG. 9 is a flowchart showing the processing steps for updating a machine learning model in step S300. In step S310, it is determined whether or not there is a machine learning model with fewer classes than the upper limit among the existing machine learning models. When N is an integer equal to or greater than 2, if there are N existing machine learning models, it is determined whether or not there is a machine learning model with fewer classes than the upper limit among them. However, the number N of existing machine learning models may be 1. In this embodiment, there are two existing machine learning models 201 and 202 as shown in FIGS. 3 and 4. The first machine learning model 201 has the same number of classes as the upper limit, and the second machine learning model 202 has the same number of classes as the upper limit. If there is no existing machine learning model with fewer classes than the upper limit, the processing proceeds to step S340, which will be described later, where a new machine learning model is added. On the other hand, if there is a machine learning model with fewer classes than the upper limit, the processing proceeds to step S320, where the machine learning model is updated.

[0046] In step S320, the classification processing unit 114 updates a machine learning model with a number of classes less than the upper limit by increasing the number of channels in its top layer by one. In this embodiment, the number of channels n2 in the top layer of the second machine learning model 202 is changed from 2 to 3. In step S330, the classification processing unit 114 performs training on the machine learning model updated in step S320. During this training, the classified data Di acquired in step S220 of FIG. 8 is used as new training data, along with the previously used training data set TD2 for the second machine learning model 202. Note that, in addition to the classified data Di acquired in step S220, it is preferable to use multiple other spectral data obtained from spectroscopic measurements of the same printing medium PM as the new training data. For this reason, it is preferable that the spectrometer 50 acquires spectral data at multiple positions on a single printing medium PM. When training is completed in this manner, the updated machine learning model 202 will have known classes corresponding to the classified data Di. Therefore, the type of this print medium PM can be recognized using the updated machine learning model 202.

[0047] In step S340, the class classification processing unit 114 adds a new machine learning model having a class corresponding to the data to be classified Di and sets its parameters. This new machine learning model preferably has the same configuration as the first machine learning model 201 shown in FIG. 3, except for the number of channels in the top layer. This new machine learning model preferably has two or more known classes, similar to the second machine learning model 202 shown in FIG. 4, for example. One of the two or more known classes is a class corresponding to the data to be classified Di. Furthermore, it is preferable that at least one of the two or more known classes is the same as at least one known class of an existing machine learning model. Equalizing one class of the new machine learning model to a known class of an existing machine learning model can be achieved by training the new machine learning model using the same training data for that known class as the training data used in training the existing machine learning model. The reason for providing two or more known classes in the new machine learning model is that if there is only one known class, the training may not be performed with sufficient accuracy.

[0048] The class of the existing machine learning model to be adopted for the new machine learning model is preferably selected from, for example, the following classes: (a) Among multiple known classes in an existing machine learning model, the class corresponding to the optical spectrum data with the highest similarity to the data to be classified Di. (b) Among multiple known classes in the existing machine learning model, the class corresponding to the optical spectrum data with the lowest similarity to the data to be classified Di. (c) A class to which the data to be classified Di is mistakenly classified as belonging in step S240 of FIG. 7, among a plurality of known classes in an existing machine learning model. Of these, adopting class (a) or (c) above can reduce misclassification in the new machine learning model, while adopting class (b) above can shorten the learning time of the new machine learning model.

[0049] In step S350, the class classification processing unit 114 performs training of the added machine learning model. In this training, the classified data Di acquired in step S220 of FIG. 8 is used as new training data. Furthermore, in addition to the classified data Di acquired in step S220, it is preferable to also use, as the new training data, multiple other optical spectrum data obtained from spectroscopic measurements of the same printing medium PM. Furthermore, if one or more classes of the new machine learning model are to be the same as known classes of an existing machine learning model, the training data used for those known classes in training the existing machine learning model is also used.

[0050] When the number of known classes in the second machine learning model 202 reaches the upper limit, a third machine learning model is added in steps S340 and S350 of FIG. 9 . The same applies to the fourth and subsequent machine learning models. In this embodiment, when N is an integer equal to or greater than 2, the (N-1) machine learning models have a number of classes equal to the upper limit, and one other machine learning model has a number of classes equal to or less than the upper limit. Furthermore, when class classification processing is performed on the data to be classified Di using the N machine learning models, if it is determined that the data to be classified Di belongs to an unknown class, one of the following processes is executed. (1) If the "one other machine learning model" has a number of classes less than the upper limit, the processing of steps S320 and S330 adds a new class for the classified data Di by training the "one other machine learning model" using training data including the classified data Di. (2) If the "one other machine learning model" has a number of classes equal to the upper limit, a new machine learning model having a class corresponding to the classified data Di is added by the processing of steps S340 and S350. According to these processes, even if the N machine learning models are unable to successfully classify the classified data Di, it is possible to classify the classified data Di into a class corresponding to the classified data Di.

[0051] The machine learning model update process shown in Figure 9 can also be performed when there is one existing machine learning model. A case where there is one existing machine learning model is, for example, when the second machine learning model 202 shown in Figure 4 is not prepared and the process of Figure 9 is performed using only the first machine learning model 201 shown in Figure 3. In this case, steps S340 and S350 in Figure 9 are processes of adding the second machine learning model 202 as a new machine learning model. In this way, the process of adding the second machine learning model 202 as a new machine learning model when it is determined that the input data belongs to an unknown class in the classification process performed using only the first machine learning model 201 can also be understood as an example of a process of preparing two machine learning models 201 and 202.

[0052] In step S360, the classification processing unit 114 generates a group of known feature spectra by inputting the training data again into the updated or added machine learning model. This process is the same as the process in step S130 in Fig. 5, and therefore a description thereof will be omitted.

[0053] As described above, in the machine learning model update process shown in FIG. 9, when the existing multiple machine learning models are unable to successfully classify the classified data Di, it is possible to classify the classified data Di into a class corresponding to the classified data Di by adding a class to the existing machine learning model or by adding a new machine learning model.

[0054] 10 is a flowchart showing the processing procedure of the class deletion process, which is initiated when the user wishes to delete one known class.

[0055] In step S410, the user selects a class to be deleted and instructs the class classification processing unit 114. In step S420, the class classification processing unit 114 receives this instruction and asks the user whether or not the data of the class to be deleted should be deleted.

[0056] If an instruction not to delete the data of the class to be deleted is received in step S420, the process proceeds to step S430, where the classification processing unit 114 changes the output name of the class to be deleted to a name indicating "deleted" or "unknown." Therefore, if a classification process is subsequently performed using the machine learning model, a result such as "Deleted medium" or "Unknown medium" may be output. On the other hand, if an instruction to delete the data of the class to be deleted is received, the process proceeds to step S440, where an update process is performed on the machine learning model that included the class to be deleted as a known class. This update process involves constructing a new machine learning model by deleting one channel from the output layer of the machine learning model, and performing new learning using training data that excludes data related to the class to be deleted.

[0057] In the class deletion process described above, when an instruction is received to designate a known class as a class to be deleted, the output name of the class to be deleted is changed to a name indicating "Deleted" or "Unknown," or a machine learning model is reconstructed by deleting one channel from the machine learning model that has the class to be deleted, and learning is performed. In this way, when a known class is no longer necessary, the known class can be deleted, thereby improving the accuracy of the class classification process by the machine learning model.

[0058] As described above, in this embodiment, when N is an integer equal to or greater than 2, class classification processing is performed using N machine learning models, so processing can be performed faster than when classification processing into multiple classes is performed using a single machine learning model. Spectrum Since the class of the data to be classified is determined using the similarity between the data, the classification process can be performed with high accuracy. Furthermore, when an existing machine learning model is unable to classify the data to be classified successfully, it is possible to classify the data to a class corresponding to the data to be classified by adding a class to the existing machine learning model or by adding a new machine learning model.

[0059] In the above embodiment, a vector neural network type machine learning model using vector neurons is used, but instead, a machine learning model using scalar neurons such as a normal convolutional neural network may be used. However, a vector neural network type machine learning model is preferable because it has higher accuracy in class classification processing than a machine learning model using scalar neurons.

[0060] B. How to calculate the output vector of each layer of the machine learning model: The method of calculating the output of each layer in the first machine learning model 201 shown in Fig. 3 is as follows: The same applies to the second machine learning model 202.

[0061] Each node in the PrimeVN layer 221 regards the scalar output of the 1x1x32 nodes in the Conv layer 211 as a 32-dimensional vector and obtains the vector output of that node by multiplying this vector by a transformation matrix. This transformation matrix is ​​an element of a 1x1 kernel and is updated by learning in the machine learning model 201. Note that the processing of the Conv layer 211 and the PrimeVN layer 221 can be integrated to form a single primary vector neuron layer.

[0062] When the PrimeVN layer 221 is referred to as the "lower layer L" and the ConvVN1 layer 231 adjacent to it on the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following equation.

number

[0063] As the normalization function F(X), for example, the following formula (4a) or (4b) can be used.

number

[0064] In the above equation (4a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function | j On the other hand, in equation (4b), the sum vector u j Norm of |u j | is the norm |u j Activation value a by dividing by the sum of | j It should be noted that a function other than equation (4a) or (4b) may be used as the normalization function F(X).

[0065] The ordinal number i in the above equation (3) is the output vector M of the jth node in the upper layer L+1. L+1 jThe integer n is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine . Thus, the integer n is given by n = Nk × Nc (6) Here, Nk is the number of elements of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 221. In the example of Fig. 3, Nk = 3 and Nc = 26, so n = 78.

[0066] One kernel used to calculate the output vector of the ConvVN1 layer 231 has 1 × 3 × 26 = 78 elements, with the kernel size 1 × 3 as the surface size and the number of channels in the lower layer being 26 as the depth. Each of these elements is a prediction matrix W L ij In addition, 20 sets of this kernel are required to generate output vectors for 20 channels of the ConvVN1 layer 231. Therefore, the prediction matrix W of the kernel used to obtain the output vector of the ConvVN1 layer 231 is L ij The number of prediction matrices W is 78 × 20 = 1560. L ij is updated by learning of the machine learning model 201.

[0067] As can be seen from the above equations (2) to (5), the output vector M of each node in the upper layer L+1 L+1 j is calculated by the following calculation: (a) Output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is a linear combination of j Seeking (c) Sum vector u j Norm of |u j The activation value a is normalized by normalizing | j Seeking (d) Sum vector u j norm |u j Divide by | and then use the activation value a j Multiply by.

[0068] In addition, the activation value a j is the norm |u j is the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in equations (4), (4a), (4b), and (5) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of j is only used in the above equations (4) and (5), and does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 so that it outputs

[0069] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above equations (2) to (5) used in a vector neural network are different from those used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v on the right side of the above equation (3) ijare multiplied by weights, and the weights are searched by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M is calculated by calculating the above-mentioned equations (2) to (5) once in order. L+1 j Therefore, there is no need to repeat dynamic routing, which has the advantage of allowing faster calculations. In addition, the vector neural network of this embodiment has the advantage that it requires less memory for calculations than a capsule network, and according to experiments by the inventors of this disclosure, it only requires about 1 / 2 to 1 / 3 of the memory required.

[0070] Vector neural networks are similar to capsule networks in that they use nodes that use vectors as input and output. Therefore, they share the advantage of using vector neurons with capsule networks. Furthermore, the multiple layers 211-251 are similar to conventional convolutional neural networks in that the higher layers represent features of larger areas and the lower layers represent features of smaller areas. Here, "feature" refers to a characteristic part contained in the input data to the neural network. Vector neural networks and capsule networks are superior to conventional convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of a node's output vector represents the probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the relative positions of the respective features. Alternatively, the vector direction of the output vectors of the two nodes can be said to represent the variation of the feature. For example, for a node corresponding to the "eye" feature, the direction of the output vector can represent variations such as the narrowness of the eyes or the way they are lifted. In conventional convolutional neural networks, it is said that spatial information of features is lost due to the pooling process. As a result, vector neural networks and capsule networks have the advantage of being superior to conventional convolutional neural networks in terms of the performance of identifying input data.

[0071] The advantages of vector neural networks can also be considered as follows. In other words, the advantage of vector neural networks is that the output vectors of nodes represent the features of input data as coordinates in continuous space. Therefore, output vectors can be evaluated such that the closer the vector directions, the more similar the features. Another advantage is that even if the features contained in the input data are not covered by the training data, they can be determined by interpolation. On the other hand, conventional convolutional neural networks have the disadvantage that the features of input data cannot be represented as coordinates in continuous space due to the chaotic compression caused by the pooling process.

[0072] The outputs of each node in the ConvVN2 layer 241 and the ClassVN layer 251 are similarly determined using the above equations (2) to (5), and therefore detailed explanations are omitted. The ClassVN layer 251, which is the top layer, has a resolution of 1×1 and a number of channels of n1.

[0073] The output of the ClassVN layer 251 is converted into a plurality of decision values ​​Class1-1 to Class1-10 for known classes. These decision values ​​are usually normalized by a softmax function. Specifically, for example, the decision value for each class can be obtained by performing the following operation: calculating the vector length of the output vector from the output vector of each node of the ClassVN layer 251, and then normalizing the vector length of each node by a softmax function. As described above, the activation value a obtained by the above formula (4) is j is the output vector M L+1 j The activation value a at each node in the ClassVN layer 251 is a value corresponding to the vector length of j may be output and used as the judgment value for each class.

[0074] In the above-described embodiment, a vector neural network that calculates an output vector by the calculations of the above equations (2) to (5) is used as the machine learning models 201 and 202. However, instead of this, a capsule network disclosed in U.S. Patent No. 5,210,798 or WO 2019 / 083553 may be used. Also, a neural network that uses only scalar neurons may be used.

[0075] Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0076] <1> According to a first aspect of the present disclosure, there is provided a method for performing a classification process on data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers. The method includes the steps of: (a) preparing N machine learning models, where N is an integer equal to or greater than 2, each of which is configured to classify input data into one of a plurality of classes and to have at least one class different from other machine learning models among the N machine learning models; and (b) preparing known features obtained from the output of at least one specific layer among the plurality of vector neuron layers when a plurality of training data are input to the N machine learning models. Spectrum (c) preparing a group of the N machine learning models, and using a selected machine learning model selected from the N machine learning models, extracting features obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model. Spectrum and the known feature Spectrumand calculating the similarity to the group for each class, and determining the class of the classified data using the similarity. According to this method, class classification processing is performed using N machine learning models, so the processing can be performed faster than when classification processing into multiple classes is performed using one machine learning model. Spectrum Since the class of the data to be classified is determined using the similarity, the classification process can be performed with high accuracy.

[0077] <2> In the above method, step (c) may include: (c1) selecting one machine learning model from the N machine learning models as the selected machine learning model; (c2) calculating the similarity using the selected machine learning model and determining the class of the classified data using the similarity; (c3) if the classified data is not determined to belong to a known class in step (c2), returning to step (c1), selecting a next machine learning model, and executing step (c2); and (c4) if results of the class classification process using all of the N machine learning models indicate that the classified data does not belong to any of the known classes, determining that the classified data belongs to an unknown class. According to this method, machine learning models are selected one by one to perform classification processing, which may enable the class of the data to be classified to be determined more quickly.

[0078] <3> In the above method, an upper limit is set for the number of classes that can be classified by any one of the N machine learning models, and (N-1) of the N machine learning models have a number of classes equal to the upper limit, and another machine learning model has a number of classes equal to or less than the upper limit. Step (c) may include, when the data to be classified is determined to belong to an unknown class when the class classification process is performed on the data to be classified using the N machine learning models, (1) if the other one of the machine learning models has a number of classes less than the upper limit, adding a new class for the data to be classified by training the other one of the machine learning models using training data that includes the data to be classified, and (2) if the other one of the machine learning models has a number of classes equal to the upper limit, adding a new machine learning model having a class corresponding to the data to be classified. According to this method, when N machine learning models are unable to classify the data to be classified successfully, it is possible to classify the data into a class corresponding to the data to be classified by adding a class to an existing machine learning model or by adding a new machine learning model.

[0079] <4> In the above method, step (2) may include a step of performing training of the new machine learning model using training data including the classified data used in step (c), and the training data may further include existing training data for training on at least one class included in the N machine learning models. According to this method, a new machine learning model is trained using not only training data for learning new classes but also existing training data for learning existing classes, making it possible to perform more accurate class classification with the new machine learning model.

[0080] <5> In the above method, the specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, and the characteristic Spectrum may be any one of (i) a first type of feature spectrum in which multiple element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the multiple channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the multiple channels along the third axis. According to this method, the features Spectrum can be easily obtained.

[0081] <6> The above method may further include the steps of receiving an instruction to designate one known class among the plurality of classes as a class to be deleted, and reconstructing the machine learning model in a machine learning model having the class to be deleted by changing the output name of the class to be deleted to a name indicating that it has been deleted or is unknown, or by deleting one channel from the output layer of the machine learning model having the class to be deleted, and performing learning of the reconstructed machine learning model. According to this method, when a known class is no longer needed, the known class can be deleted, thereby improving the accuracy of the class classification process using the machine learning model.

[0082] <7> According to a second aspect of the present disclosure, there is provided an information processing device that performs a class classification process on data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. The information processing device includes a memory that stores the machine learning model and one or more processors that perform calculations using the machine learning model. The one or more processors perform the following steps: (a) a process of preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models being configured to classify input data into one of multiple classes and having at least one class that is different from other machine learning models among the N machine learning models; and (b) a process of generating known features obtained from the output of at least one specific layer among the multiple vector neuron layers when multiple pieces of training data are input to the N machine learning models. Spectrum (c) using a selected machine learning model selected from the N machine learning models, to calculate features obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model. Spectrum and the known feature Spectrum The similarity to the group is calculated for each class, and the class of the classified data is determined using the similarity. According to this information processing device, since the classification process is performed using N machine learning models, the process can be performed faster than when a classification process is performed into multiple classes using a single machine learning model. Spectrum Since the class of the data to be classified is determined using the similarity, the classification process can be performed with high accuracy.

[0083] <8> According to a third aspect of the present disclosure, there is provided a computer program that causes one or more processors to execute a class classification process for data to be classified using a machine learning model including a vector neural network having multiple vector neuron layers. The computer program includes: (a) a process for preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models configured to classify input data into one of multiple classes and having at least one class different from other machine learning models among the N machine learning models; and (b) a process for preparing known features obtained from the output of at least one specific layer among the multiple vector neuron layers when multiple training data are input to the N machine learning models. Spectrum (c) using a selected machine learning model selected from the N machine learning models, to calculate features obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model. Spectrum and the known feature Spectrum and calculating a similarity to a group for each class, and determining a class of the classified data using the similarity. According to this computer program, class classification processing is performed using N machine learning models, so the processing can be performed faster than when classification processing into multiple classes is performed using a single machine learning model. Spectrum Since the class of the data to be classified is determined using the similarity, the classification process can be performed with high accuracy.

[0084] The present disclosure may be realized in various forms other than those described above, such as a computer program for implementing the functions of a classification device, a non-transitory storage medium on which the computer program is recorded, or the like. [Explanation of symbols]

[0085] 10...printer, 20...control device, 30...printing mechanism, 40...printing medium holder, 50...spectrometer, 110...processor, 112...printing control unit, 114...classification processing unit, 120...memory, 130...interface circuit, 140...input device, 150...display device, 160...audio output device, 201, 202...machine learning model, 211, 212...convolutional layer, 221, 222...primary vector neuron layer, 231, 232...first convolutional vector neuron layer, 241, 242...second convolutional vector neuron layer, 251, 252...classification vector neuron layer, 261, 262...similarity calculation unit

Claims

1. 1. A method for performing a classification process on data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers, comprising: (a) preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models being configured to classify input data into one of a plurality of classes and having at least one class that is different from other machine learning models among the N machine learning models; (b) preparing a group of known feature spectra obtained from the output of at least one specific layer among the plurality of vector neuron layers when a plurality of training data are input to the N machine learning models; (c) using a selected machine learning model selected from the N machine learning models, calculating the similarity between the feature spectrum obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model and the group of known feature spectra for each class, and determining the class of the data to be classified using the similarity; A computer-implemented method, including:

2. 10. The method of claim 1, The step (c) (c1) selecting one machine learning model from the N machine learning models as the selected machine learning model; (c2) calculating the similarity using the selected machine learning model and determining the class of the classified data using the similarity; (c3) if the classified data is not determined to belong to a known class in the step (c2), returning to the step (c1), selecting a next machine learning model, and executing the step (c2); (c4) determining that the classified data belongs to an unknown class when the results of the classification process by all of the N machine learning models indicate that the classified data does not belong to any known class; A method comprising:

3. 3. The method of claim 1 or 2, an upper limit is set for the number of classes that can be classified by any one of the N machine learning models; Among the N machine learning models, (N-1) machine learning models have a number of classes equal to the upper limit value, and the other machine learning model has a number of classes equal to or less than the upper limit value, The step (c) When the classification process is performed on the classified data using the N machine learning models and the classified data is determined to belong to an unknown class, (1) if the other one of the machine learning models has a number of classes less than the upper limit, adding a new class for the classified data by performing learning on the other one of the machine learning models using training data including the classified data; (2) if the other one machine learning model has a number of classes equal to the upper limit, adding a new machine learning model having a class corresponding to the classified data; A method comprising the steps of:

4. 4. The method of claim 3, The step (2) includes a step of executing learning of the new machine learning model using training data including the classified data used in the step (c), The method, wherein the training data further includes existing training data for learning about at least one class included in the N machine learning models.

5. The method according to any one of claims 1 to 4, The specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, The characteristic spectrum is (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one plane position in the specific layer are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by an activation value corresponding to the vector length of the output vector; (iii) a third type of feature spectrum in which the activation values ​​at one planar position of the specific layer are arranged across the plurality of channels along the third axis; A method, which is one of the following:

6. The method according to any one of claims 1 to 5, further comprising: receiving an instruction to designate one known class among the plurality of classes as a class to be deleted; In a machine learning model having the class to be deleted, changing the output name of the class to be deleted to a name indicating that the class has been deleted or is unknown, or a step of reconstructing the machine learning model by deleting one channel from an output layer of the machine learning model having the class to be deleted, and performing training of the reconstructed machine learning model; A method comprising:

7. An information processing device that performs a class classification process on data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers, a memory that stores the machine learning model; one or more processors that perform operations using the machine learning model; Equipped with the one or more processors: (a) a process of preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models being configured to classify input data into one of a plurality of classes and having at least one class that is different from other machine learning models among the N machine learning models; (b) preparing a group of known feature spectra obtained from the output of at least one specific layer among the plurality of vector neuron layers when a plurality of training data are input to the N machine learning models; (c) using a selected machine learning model selected from the N machine learning models, calculating the similarity between the feature spectrum obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model and the group of known feature spectra for each class, and determining the class of the data to be classified using the similarity; An information processing device that executes the above.

8. A computer program that causes one or more processors to perform a class classification process on data to be classified using a machine learning model including a vector neural network having a plurality of vector neuron layers, The computer program comprises: (a) a process of preparing N machine learning models, where N is an integer equal to or greater than 2, each of the N machine learning models being configured to classify input data into one of a plurality of classes and having at least one class that is different from other machine learning models among the N machine learning models; (b) preparing a group of known feature spectra obtained from the output of at least one specific layer among the plurality of vector neuron layers when a plurality of training data are input to the N machine learning models; (c) using a selected machine learning model selected from the N machine learning models, calculating the similarity between the feature spectrum obtained from the output of the specific layer when the data to be classified is input to the selected machine learning model and the group of known feature spectra for each class, and determining the class of the data to be classified using the similarity; a computer program for causing the one or more processors to execute the

Images