Evaluation method, evaluation device, and computer program

Abstract

To provide a technique of evaluating a learned mechanical learning model without generating difference between users.SOLUTION: The method for evaluating a learned mechanical learning model includes the steps of (a) of: inputting evaluation data into a learned mechanical learning model and generating primary explanation information used for evaluation of the mechanical learning model; (b) generating secondary explanation information showing the evaluation of the learned mechanical learning model by using the value shown by each piece of the primary explanation information; and (c) outputting the generated secondary explanation information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technique for evaluating a trained machine learning model. [Background technology]

[0002] Patent Documents 1 and 2 disclose a vector neural network type machine learning model that uses vector neurons, called a capsule network. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons called capsules as network nodes. A vector neural network type machine learning model such as a capsule network can be used to classify input data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 5,210,798 [Patent Document 2] International Publication No. 2019 / 083553 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technologies, events can occur in which a trained machine learning model does not achieve the initial goal, such as not achieving the desired discrimination accuracy. Evaluations of trained machine learning models, such as identifying the causes of such events, are prone to differences between users depending on their experience. Therefore, there has been a demand for technology that can evaluate trained machine learning models without causing differences between users. Evaluations of trained machine learning models include evaluations of the trained machine learning model itself as well as evaluations of evaluation data used to evaluate the trained machine learning model. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a method for evaluating a trained machine learning model. The machine learning model is a vector neural network model having multiple vector neuron layers, and is trained using multiple training data each having input data and a prior label associated with the input data. This evaluation method includes: (a) inputting the evaluation data into the trained machine learning model to generate primary explanation information used for evaluating the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanation information to generate secondary explanation information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanation information. The step (a) includes: (a1) inputting the evaluation data into the trained machine learning model to obtain a feature spectrum from the output of a specific layer of the trained machine learning model; (a2) calculating the feature spectrum and The method includes: (a) a step of calculating, for each of a plurality of known feature spectra included in a group of known feature spectra obtained from the output of the specific layer by inputting the plurality of training data again into the trained machine learning model, spectral similarity, which is the similarity between the group of known feature spectra and a known feature spectrum included in the group of known feature spectra obtained from the output of the specific layer; (a3) ​​a step of calculating data similarity, which is the similarity between the input data and the evaluation data; and (a4) a step of generating the primary explanation information including spectral similarity information on the spectral similarity and data similarity information on the data similarity.

[0006] According to a second aspect of the present disclosure, there is provided an evaluation device for a trained machine learning model, the evaluation device comprising: a memory for storing the machine learning model, the machine learning model being a vector neural network type model having a plurality of vector neuron layers, trained using a plurality of training data each having input data and a prior label associated with the input data; and a processor, the processor executing: (a) inputting the evaluation data into the trained machine learning model to generate primary explanation information to be used for evaluation of the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanation information to generate secondary explanation information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanation information, the process (a) including: (a1) (a2) a process of inputting the evaluation data into a trained machine learning model and determining a feature spectrum from an output of a specific layer of the trained machine learning model; (a3) ​​a process of determining data similarity, which is the similarity between the feature spectrum and a known feature spectrum included in a group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model, for each of the plurality of known feature spectra included in the group of known feature spectra; (a4) a process of generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity.

[0007] According to a third aspect of the present disclosure, there is provided a computer program for causing a computer to execute evaluation of a trained machine learning model. The machine learning model is a vector neural network model having multiple vector neuron layers, and is trained using multiple training data each having input data and a prior label associated with the input data. The computer program causes a computer to execute the following functions: (a) inputting evaluation data into the trained machine learning model to generate primary explanatory information used for evaluating the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanatory information to generate secondary explanatory information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanatory information. The function (a) includes: (a1) inputting the evaluation data into the trained machine learning model to obtain a feature spectrum from the output of a specific layer of the trained machine learning model; and (a2) generating the secondary explanatory information. The machine learning model has: (a) a function for calculating, for each of a plurality of known feature spectra included in a group of known feature spectra, spectral similarity, which is the similarity between a feature spectrum and a known feature spectrum included in a group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model; (a3) ​​a function for calculating data similarity, which is the similarity between the input data and the evaluation data; and (a4) a function for generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an evaluation system according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram showing the configuration of a machine learning model. [Figure 3] 1 is a flowchart showing the learning process of a machine learning model. [Figure 4] FIG. 10 is a diagram for explaining learning data. [Figure 5] FIG. [Figure 6] FIG. 2 is an explanatory diagram showing the configuration of a group of known characteristic spectra. [Figure 7] 10 is a flowchart of an evaluation process executed by an evaluation processing unit. [Figure 8] FIG. 1 illustrates primary explanatory information. [Figure 9] FIG. 2 illustrates primary explanatory information. [Figure 10] FIG. 10 is an explanatory diagram showing a first calculation method for the maximum spectral similarity. [Figure 11] FIG. 10 is a diagram showing a second method for calculating the maximum spectral similarity. [Figure 12] FIG. 10 is an explanatory diagram showing a third method for calculating the maximum spectral similarity. [Figure 13] FIG. 10 is a diagram showing histograms of individual local spectral similarities. [Figure 14] FIG. 10 is a diagram showing a histogram of individual spectral similarities. [Figure 15] FIG. 1 is a diagram illustrating data similarity. [Figure 16] 10 is a first flowchart showing details of step S30. [Figure 17] 10 is a second flowchart showing details of step S30. [Figure 18] 10 is a third flowchart showing details of step S30. [Figure 19] 4 is a fourth flowchart showing details of step S30. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. Implementation: A-1. Overview of the evaluation system: FIG. 1 is a block diagram showing an evaluation system 5 according to a first embodiment. The evaluation system 5 includes an evaluation device 100 and an imaging device 400 as a sensor. The imaging device 400 captures an image of a target object. The captured image obtained by the imaging device 400 is used as data for training, evaluation, and class discrimination of a machine learning model 200. The evaluation device 100 executes training processing for the machine learning model 200, evaluation processing for the trained machine learning model 200, and class discrimination processing for data using the trained machine learning model 200. Note that the evaluation device 100 is only required to be able to execute at least the evaluation processing; a device other than the evaluation device 100 may execute the learning processing and discrimination processing. The "class of data" refers to the type of data. The evaluation device 100 may output the determined data type to a display unit, which is an output unit. This allows the user to easily understand the type of data. Note that the evaluation system according to the present disclosure can also be implemented as a system other than the above. For example, the evaluation processing may be executed using data such as one-dimensional data or time-series data.

[0010] The evaluation device 100 includes a processor 110, a memory 120, an interface circuit 130, and an input device 140 and a display unit 150 connected to the interface circuit 130. The evaluation device 100 is, for example, a personal computer. An imaging device 400 is also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has the function of executing the processes described in detail below, but also has the function of displaying data obtained by the processes and data generated during the processes on the display unit 150.

[0011] The processor 110 executes various programs stored in the memory 120, thereby comprising a learning execution unit 112, a class discrimination unit 113, and an evaluation processing unit 114. The learning execution unit 112 executes a learning process for the machine learning model 200 using a training data group TDG. The class discrimination unit 113 inputs data IM to the trained machine learning model 200, thereby executing class discrimination of the input data IM. The evaluation processing unit 114 inputs evaluation data ED to the trained machine learning model 200, thereby generating primary explanation information FEI, and generates secondary explanation information SEI indicating an evaluation of the machine learning model 200 from this primary explanation information FEI. The generated secondary explanation information SEI is output to the display unit 150.

[0012] The evaluation processing unit 114 includes a similarity calculation unit 310 and an evaluation unit 330. The similarity calculation unit 310 inputs evaluation data ED to the trained machine learning model 200 and generates primary explanation information FEI including spectral similarity information IRSp and data similarity information IDa. The spectral similarity information IRSp is information indicating the degree of similarity between a known feature spectrum KSp obtained by inputting training data TD to the trained machine learning model 200 and a feature spectrum Sp obtained by inputting the evaluation data ED to the trained machine learning model 200. Furthermore, the data similarity information IDa is information indicating the data similarity Da between the training data TD that generated a specific known feature spectrum KSp identified based on the spectral similarity RSp between the known feature spectrum KSp and the feature spectrum Sp and the evaluation data ED. The primary explanation information FEI is used to evaluate the trained machine learning model 200. The evaluation unit 330 generates secondary explanation information SEI that indicates an evaluation of the trained machine learning model 200, using values ​​indicated by various pieces of information contained in the primary explanation information FEI.

[0013] In the above, at least some of the functions of the learning execution unit 112, the class discrimination unit 113, and the evaluation processing unit 114 may be realized by a hardware circuit. The term "processor" in this specification also includes such hardware circuits. Furthermore, the processor that executes the class discrimination process may be a processor included in a remote computer connected to the evaluation device 100 via a network.

[0014] The memory 120 stores a machine learning model 200, a training data group TDG, an evaluation data group EDG, and a known feature spectrum group KSpG. The machine learning model 200 is used to classify input data IM. Each of the machine learning models 200 is a vector neural network type machine learning model having multiple vector neuron layers. The machine learning model 200 is trained using multiple training data TD. An example configuration and operation of the machine learning model 200 will be described later.

[0015] The training data group TDG is a collection of training data TD, which is teacher data. In this embodiment, each training data TD in the training data group TDG has input data IM and a prior label LB associated with the input data IM. In this embodiment, the input data IM is a captured image of a target object captured by the imaging device 400. In this embodiment, the prior label LB is a label indicating the type of the target object. Note that in this embodiment, "label" and "class" have the same meaning.

[0016] The evaluation data group EDG is a collection of evaluation data ED used to evaluate the trained machine learning model 200. The evaluation data ED is at least one type of data: training data TD, verification data VD, and anomalous data AD. The training data TD is data from the training data group TDG that was used to train the machine learning model 200. The verification data VD is data that has not been used to train the machine learning model 200 and includes input data IM and a priori labels LB associated with the input data IM. In this embodiment, the verification data VD is data generated by cross-validation for the training data group TDG. In other words, a portion of the multiple training data TD prepared for training is used as the verification data VD. The anomalous data AD is input data IM that is not associated with a priori label LB. The anomalous data AD is input data IM that is expected to be classified by the machine learning model 200 into an unknown class different from the class corresponding to the priori label LB.

[0017] The known feature spectrum group KSpG is a set of feature spectra Sp obtained when the training data group TDG is input to the trained machine learning model 200. The feature spectra Sp will be described later. Note that the feature spectra Sp of the known feature spectrum group KSpG are also referred to as known feature spectra KSp.

[0018] FIG. 2 is an explanatory diagram showing the configuration of a machine learning model 200. This machine learning model 200 has an input layer 210, a hidden layer 280, and an output layer 260. The hidden layer 280 includes a convolutional layer 220, a primary vector neuron layer 230, a first convolutional vector neuron layer 240, and a second convolutional vector neuron layer 250. Of these layers, the input layer 210 is the lowest layer, and the output layer 260 is the highest layer. In the following description, the layers of the hidden layer 280 are referred to as the "Conv layer 220," the "PrimeVN layer 230," the "ConvVN1 layer 240," and the "ConvVN2 layer 250," respectively, and the output layer 260 is referred to as the "ClassVN layer 260."

[0019] 2, two convolution vector neuron layers 240 and 250 are used, but the number of convolution vector neuron layers is arbitrary, and a convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0020] A captured image of 28×28 pixels is input to the input layer 210. The configuration of each layer other than the input layer 210 can be described as follows. ·Conv layer 220: Conv[32,5,2] ·PrimeVN layer 230: PrimeVN[16,1,1] ·ConvVN1 layer 240:ConvVN1[12,3,2] ·ConvVN2 layer 250:ConvVN2[6,3,1] ·ClassVN layer 260:ClassVN[M,3,1] Vector dimension VD: VD=16 In the description of each layer, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the kernel surface size, and the stride. For example, the layer name of the Conv layer 220 is "Conv," the number of channels is 32, the kernel surface size is 5x5, and the stride is 2. In Figure 2, these descriptions are shown below each layer. The hatched rectangles drawn in each layer represent the kernel surface size used when calculating the output vector of the adjacent higher layer. In this embodiment, since the input data is image data, the kernel surface size is also two-dimensional. Note that the parameter values ​​used in the description of each layer are merely examples and can be changed as desired.

[0021] The input layer 210 and the Conv layer 220 are layers composed of scalar neurons. The other layers 230 to 260 are layers composed of vector neurons. A vector neuron is a neuron that uses vectors as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0022] FIG. 2 shows the first axis x and second axis y that define the planar coordinates of the node array for the Conv layer 220, and the third axis z that represents depth. It also shows that the sizes of the Conv layer 220 in the x, y, and z directions are 12, 12, and 32. The sizes in the x and y directions are called "resolution." The size in the z direction is the number of channels. These three axes x, y, and z are also used in other layers as coordinate axes that indicate the position of each node. However, in FIG. 2, these axes x, y, and z are omitted from the illustration for layers other than the Conv layer 220.

[0023] As is well known, the resolution W1 after convolution is given by the following equation: W1=Ceil{(W0-Wk+1) / S} (A1) Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function that rounds up the decimal point of X. The resolution of each layer shown in FIG. 2 is an example in which the resolution of the input data is 28, and the actual resolution of each layer is changed appropriately depending on the size of the input data IM.

[0024] The ClassVN layer 260 has M channels. M is the number of classes discriminated by the machine learning model 200. In this embodiment, M is 2, and two class determination values ​​Class_1 and Class_2 are output. The number of channels M of the ClassVN layer 260 can be set to any integer equal to or greater than 2. If both of the two class determination values ​​Class_1 and Class_2 are less than a threshold, an unknown class indicating that the data belongs to a different class from the training data TD used for training is output as the class discrimination result.

[0025] FIG. 2 also illustrates subregions Rn in each layer 220, 230, 240, 250, and 260. The subscript "n" in subregion Rn refers to the reference number of the layer. For example, subregion R220 indicates a subregion in the Conv layer 220. A "subregion Rn" is a region in each layer that is identified by a planar position (x, y) defined by the position of the first axis x and the position of the second axis y, and that includes multiple channels along the third axis z. The subregion Rn has dimensions of "Width" × "Height" × "Depth," corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one "subregion Rn" is "1 × 1 × number of depths," i.e., "1 × 1 × number of channels."

[0026] As shown in Fig. 2, a feature spectrum Sp, which will be described later, is calculated from the output of the ConvVN2 layer 250. The similarity calculation unit 310 uses this feature spectrum Sp to calculate spectral similarities RSp between each piece of training data TD and evaluation data ED. The spectral similarities RSp are used to generate spectral similarity information IRSp. The similarity calculation unit 310 also uses the spectral similarities RSp to generate data similarity Da, which is the basis of data similarity information IDa.

[0027] In this disclosure, the vector neuron layer used to calculate the spectral similarity RSp is also referred to as a "specific layer." As the specific layer, a vector neuron layer other than the ConvVN2 layer 250 may be used, and any number of vector neuron layers greater than or equal to one may be used. Note that the configuration of the feature spectrum Sp, the method for generating the spectral similarity information IRSp, and the method for generating the data similarity information IDa will be described later.

[0028] FIG. 3 is a flowchart showing the learning process of the machine learning model 200. FIG. 4 is a diagram for explaining training data TD. As shown in FIG. 3, in step S10, a plurality of training data TD is prepared, which includes input data IM, which is a captured image of a target object, and a priori labels LB associated with the input data IM. As shown in FIG. 4, the input data IM is acquired by capturing an image of a target object, whose class, which is classified by type, is known in advance, using the imaging device 400. Furthermore, the training data TD is generated by associating the priori labels LB corresponding to the known types with the input data IM. In this embodiment, the priori labels LB are two labels, "1" and "2." The input data IM1 to IM2 are associated with labels 1 to 2, which are the corresponding priori labels LB. The plurality of training data TD is stored in the memory 120 as a training data group TDG.

[0029] 3, in step S12, the learning execution unit 112 executes learning by inputting the learning data group TDG into the machine learning model 200. Specifically, the learning execution unit 112 executes learning of the machine learning model 200 so as to reproduce the correspondence between the input data IM and the prior label LB associated with the input data IM. When learning of the machine learning model 200 is completed, the trained machine learning model 200 is stored in the memory 120.

[0030] In step S14, the learning execution unit 112 generates a known feature spectrum group KSpG by again inputting the training data group TDG used for learning into the trained machine learning model 200. The generated known feature spectrum group KSpG is stored in the memory 120. The known feature spectrum group KSpG is a collection of known feature spectra KSp, which will be described below.

[0031] FIG. 5 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 200. As shown in FIG. 2, in this embodiment, the feature spectrum Sp is created from the output of the ConvVN2 layer 250. The horizontal axis in FIG. 5 represents the position of a vector element in the output vector of multiple nodes included in one subregion R250 of the ConvVN2 layer 250. The position of this vector element is represented by a combination of the element number ND of the output vector at each node and the channel number NC. In this embodiment, since the vector dimension is 16 (the number of elements of the output vector output by each node), the element number ND of the output vector is 16, ranging from 0 to 15. Furthermore, since the ConvVN2 layer 250 has six channels, the channel number NC is 6, ranging from 0 to 5. In other words, this feature spectrum Sp is obtained by arranging multiple element values ​​of the output vector of each vector neuron included in one subregion R250 across multiple channels along the third axis z.

[0032] The vertical axis in FIG. 5 indicates the feature value CV at each spectral position. In this example, the feature value CV is the value VND of each element of the output vector. The feature value CV may be subjected to statistical processing such as centering to an average value of 0. Note that the feature value CV may be a value obtained by multiplying the value VND of each element of the output vector by a normalization coefficient (described later), or the normalization coefficient may be used as is. In the latter case, the number of feature values ​​CV included in the feature spectrum Sp is equal to the number of channels, i.e., six. Note that the normalization coefficient is a value corresponding to the vector length of the output vector of that node.

[0033] The number of feature spectra Sp obtained from the output of the ConvVN2 layer 250 for one piece of input data IM is equal to the number of planar positions (x, y) of the ConvVN2 layer 250, that is, the number of subregions R250, and is therefore nine.

[0034] 6 is an explanatory diagram showing the configuration of the known feature spectra group KSpG. In this example, the known feature spectra group KSpG obtained from the output of the ConvVN2 layer 250 is shown. Note that it is sufficient that the known feature spectra group KSpG is obtained from the output of at least one vector neuron layer, and known feature spectra obtained from the output of the ConvVN1 layer 240 or the ClassVN layer 260 may also be registered.

[0035] Each record in the known feature spectrum group KSpG includes a parameter k indicating the order of the subregion Rn within the layer, a parameter c indicating the class, a parameter q indicating the data number, and a known feature spectrum KSp. The known feature spectrum KSp is the same as the feature spectrum Sp in Figure 5.

[0036] The parameter k of the subregion Rn takes a value indicating which of the multiple subregions Rn included in a specific layer the subregion Rn is, i.e., which planar position (x, y) it is. For the ConvVN2 layer 250, there are nine subregions R250, so k = 1 to 9. The parameter c representing the class takes a value indicating which of the M classes distinguishable by the machine learning model 200 the subregion is. In this embodiment, M = 2, so c = 1 to 2. The parameter q representing the data number indicates the sequential number of the training data belonging to each class, and takes a value from 1 to max1 for c = 1, and a value from 1 to max2 for c = 2. In this way, the feature spectrum Sp is associated with the class c and the data number q of the training data. Furthermore, the feature spectrum Sp is classified by class.

[0037] A-2. Overview of the evaluation process: FIG. 7 is a flowchart of the evaluation process executed by the evaluation processing unit 114. The evaluation process can be executed after the learning process shown in FIG. 3 is completed. In step S20, the similarity calculation unit 310 inputs the evaluation data ED into the trained machine learning model 200 to generate primary explanation information FEI. Next, in step S30, the evaluation unit 330 generates secondary explanation information SEI from the primary explanation information FEI. Next, in step S90, the evaluation unit 330 outputs the secondary explanation information SEI to the display unit 150.

[0038] Fig. 8 is a first diagram illustrating the primary explanation information FEI. Fig. 9 is a second diagram illustrating the primary explanation information FEI. Fig. 8 is a diagram illustrating primary explanation information FEI generated when the evaluation data ED is either learning data TD or verification data VD. Fig. 9 is a diagram illustrating primary explanation information FEI generated when the evaluation data ED is abnormal data AD.

[0039] 8, in the first case where the evaluation data ED is the training data TD or the verification data ED, the primary explanation information FEI includes spectral similarity information IRSp and data similarity information IDa. The spectral similarity information IRSp is information related to the spectral similarity RSp, and indicates at least one of the spectral similarity RSp and a statistical representative value PRSp of the distribution of the spectral similarity RSp.

[0040] The spectral similarities RSp include a maximum in-class spectral similarity RSp_maxA and a maximum other-class spectral similarity RSp_maxB. The maximum in-class spectral similarity RSp_maxA is the similarity indicating the maximum value among the multiple in-class spectral similarities RSp_A. The in-class spectral similarity RSp_A is the spectral similarity RSp between the feature spectrum Sp of the evaluation data ED and a known in-class feature spectrum KSp_A of the same class as the evaluation class indicated by the a priori label LB associated with the evaluation data ED, among the known feature spectra KSpG. The maximum other-class spectral similarity RSp_maxB is the similarity indicating the maximum value among the multiple other-class spectral similarities RSp_B. The other-class spectral similarity RSp_B is the spectral similarity RSp between the feature spectrum Sp of the evaluation data ED and a known other-class feature spectrum KSp_B of a class different from the evaluation class, among the known feature spectra KSpG.

[0041] The distribution representative value PRSp includes a self-class spectrum representative value PRSp_A and an other-class spectrum representative value PRSp_B. The self-class spectrum representative value PRSp_A is a representative value of the distribution of a plurality of self-class spectral similarities RSp_A. The other-class spectrum representative value PRSp_B is a representative value of the distribution of a plurality of other-class spectral similarities RSp_B. A detailed method for generating the representative values ​​PRSp will be described later.

[0042] The spectral similarity information IRSp includes own-class spectral similarity information IRSp_A related to the own-class maximum spectral similarity RSp_maxA and other-class spectral similarity information IRSp_B related to the other-class maximum spectral similarity RSp_maxB. The own-class spectral similarity information IRSp_A includes the above-mentioned own-class maximum spectral similarity RSp_maxA and the own-class spectral representative value PRSp_A. Note that in other embodiments, the own-class spectral similarity information IRSp_A may include at least one of the own-class maximum spectral similarity RSp_maxA and the own-class spectral representative value PRSp_A. The other-class spectral similarity information IRSp_B includes the above-mentioned other-class maximum spectral similarity RSp_maxB and the other-class spectral representative value PRSp_B. Note that in other embodiments, the other-class spectral similarity information IRSp_B may include at least one of the other-class maximum spectral similarity RSp_maxB and the other-class spectral representative value PRSp_B.

[0043] The data similarity information IDa is information related to the data similarity Da. In this embodiment, the data similarity information IDa is information indicating the data similarity Da. The data similarity Da includes a self-class maximum data similarity Da_A and an other-class maximum data similarity Da_B. The self-class maximum data similarity Da_A is the similarity between the input data IM associated with the self-class known feature spectrum KSp_A, from which the self-class maximum spectral similarity RSp_maxA is calculated, and the evaluation data ED. The other-class maximum data similarity Da_B is the similarity between the input data IM associated with the other-class known feature spectrum KSp_B, from which the other-class maximum spectral similarity RSp_maxB is calculated, and the evaluation data ED.

[0044] The data similarity information IDa includes own-class data similarity information IDa_A and other-class data similarity information IDa_B. The own-class data similarity information IDa_A is information relating to the above-mentioned own-class maximum data similarity Da_A, and in this embodiment, is information indicating the own-class maximum data similarity Da_A. The other-class data similarity information IDa_B is information relating to the above-mentioned other-class maximum data similarity Da_B, and in this embodiment, is information indicating the other-class maximum data similarity Da_B.

[0045] As shown in FIG. 9 , in the second case where the evaluation data ED is abnormal data AD, the abnormal data AD is not associated with a priori label LB, so it is impossible to distinguish whether the abnormal data AD belongs to the same class as the known feature spectrum KSp or a different class. Therefore, in the second case, the primary explanation information FEI includes spectral similarity information IRSp and data similarity information IDa, which do not distinguish between its own class and other classes. The spectral similarity information IRSp is information related to the spectral similarity RSp and indicates the maximum spectral similarity RSp_max. The data similarity information IDa is information related to the maximum data similarity Da_max, and in this embodiment, indicates the maximum data similarity Da_max. The maximum spectral similarity RSp_max is the maximum similarity among the spectral similarities RSp calculated for each known feature spectrum KSp in the known feature spectrum group KSpG. Note that, even in the second case, the spectral similarity information IRSp may include a representative value PRSp of the distribution. The representative value PRSp of the distribution is a statistical representative value of the distribution of the spectral similarity RSp obtained for each known feature spectrum KSp.

[0046] A-3. Spectral similarity calculation method: As a method for calculating the above-mentioned spectral similarity, for example, any of the following methods can be adopted. (1) A first calculation method M1 for calculating the spectral similarity RSp without considering the correspondence between the known feature spectrum KSp of the training data TD and the subregion Rn of the feature spectrum Sp of the evaluation data ED. (2) A second calculation method M2 for calculating the spectral similarity RSp between the corresponding partial regions Rn of the known feature spectrum KSp of the training data TD and the feature spectrum Sp of the evaluation data ED. (3) A third calculation method M3 for calculating the spectral similarity RSp without considering the subregion Rn at all Below, methods for calculating the spectral similarity RSp from the output of the ConvVN2 layer 250 according to these calculation methods M1, M2, and M3 will be described in order.

[0047] FIG. 10 is an explanatory diagram illustrating a first calculation method M1 of the maximum spectral similarity. FIG. 10 illustrates a calculation method for the maximum spectral similarity RSp_max when the evaluation data ED is either the training data TD or the verification data VD. In the first calculation method M1, first, a local spectral similarity S(i,j,k) is calculated for each subregion Rn from the output of the ConvVN2 layer 250, which is a specific layer. When the evaluation data ED is either the training data TD or the verification data VD, the local spectral similarity S(i,j,k) includes a local spectral similarity S(i,j,k) for the own class, which is the same class as the a priori label LB associated with the evaluation data ED, and a local spectral similarity S(i,j,k) for the other class, which is a class different from the a priori label LB associated with the evaluation data ED. On the other hand, when the evaluation data ED is anomalous data AD, the local spectral similarity S(i,j,k) for all classes is calculated. Then, from these local spectral similarities S(i,j,k), one of three types of maximum spectral similarities RSp_max shown on the right side of FIG. 10 is calculated.

[0048] In the first calculation method M1, the local spectral similarity S(i,j,k) is calculated using the following equation: S(i,j,k)=max[G{Sp(j,k), KSp(i,j,k=all,q=all)}] (c1) where: i is a parameter indicating whether it is the same class or another class. j is a parameter indicating a specific layer, k is a parameter indicating the subregion Rn, q is a parameter indicating the data number, G{a,b} is a function that calculates the spectral similarity between a and b. Sp(j,k) is the feature spectrum obtained from the output of a specific subregion Rn of a specific layer j according to the evaluation data ED. KSp(i, j, k=all, q=all) is the known feature spectrum of all data numbers q in all subregions Rn of a specific layer j associated with the current class i or other classes i among the known feature spectrum group KSpG shown in FIG. 6; max[X] is a logical operation that takes the maximum value of X. As the function G{a,b} for calculating the local spectral similarity, for example, an equation for calculating cosine similarity or an equation for calculating similarity according to distance can be used. Furthermore, when the evaluation data ED is abnormal data AD, the parameter i above specifies all classes, so KSp(i,j,k=all,q=all) in the above equation (c1) becomes KSp(i=all,j,k=all,q=all).

[0049] The three types of maximum spectral similarities RSp_max shown on the right side of Fig. 10 are calculated as representative similarities by statistically processing the local spectral similarities S(i, j, k) for multiple partial regions Rn. The statistical processing is performed by taking the maximum, average, or minimum value of the multiple local spectral similarities S(i, j, k). Note that, although not shown in the figure, which of the maximum, average, or minimum values ​​to use is determined in advance by the user experimentally or empirically.

[0050] As described above, in the first maximum spectral similarity calculation method M1, the maximum spectral similarities RSp_maxA, RSp_maxB, and RSp_max are calculated in the following manner. <Maximum spectral similarity of own class RSp_maxA> (1) Calculating the local spectral similarity S(i, j, k) between the feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with the same class as the prior label LB of the specific layer j and the evaluation data ED, according to the evaluation data ED; (2) The maximum spectral similarity RSp_maxA of the own class is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. <Maximum other class spectral similarity RSp_maxB> (1) Calculating local spectral similarities S(i, j, k) between a feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with classes different from the prior label LB of the specific layer j and the evaluation data ED, according to the evaluation data ED; (2) The maximum other-class spectral similarity RSp_maxB is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. <Maximum spectral similarity RSp_max> (1) Calculating the local spectral similarity S(i, j, k) between the feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with the specific layer j according to the evaluation data ED; (2) The maximum spectral similarity RSp_max is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. According to the first calculation method M1, the maximum spectral similarities RSp_max, RSp_maxA, and RSp_maxB can be found using relatively simple calculations and procedures.

[0051] Fig. 11 shows a second calculation method M2 for the maximum spectral similarity. Fig. 11 shows a calculation method for the maximum spectral similarity RSp_max when the evaluation data ED is either the training data TD or the verification data VD. The similarity calculation unit 310 calculates the local spectral similarity S(i,j,k) using the following equation instead of the above-mentioned equation (c1): S(i,j,k)=max[G{Sp(j,k), KSp(i,j,k,q=all)}] (c2) where: KSp(i,j,k,q=all) is the known feature spectrum of all data numbers q in a specific partial region Rn of a specific layer j, associated with the current class i or another class i, among the known feature spectrum group KSpG shown in Fig. 6. Furthermore, as with the first calculation method, when the evaluation data ED is abnormal data AD, the above parameter i specifies all classes, so KSp(i,j,k,q=all) in the above formula (c2) becomes KSp(i=all,j,k,q=all).

[0052] While the first calculation method M1 described above uses known feature spectra KSp(i,j,k=all,q=all) in all partial regions Rn of a specific layer j, the second calculation method M2 uses only known feature spectra KSp(i,j,k,q=all) for the partial region Rn that is the same as the partial region Rn of the feature spectrum Sp(j,k). The other aspects of the second calculation method M2 are the same as those of the first calculation method M1.

[0053] As described above, in the second maximum spectral similarity calculation method M2, the maximum spectral similarities RSp_maxA, RSp_maxB, and RSp_max are calculated in the following manner. <Maximum spectral similarity of own class RSp_maxA> (1) According to the evaluation data ED, a local spectral similarity S(i, j, k) is calculated between the feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with the same class as the specific subregion Rn of the specific layer j and the prior label LB of the evaluation data ED; (2) The maximum spectral similarity RSp_maxA of the own class is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. <Maximum other class spectral similarity RSp_maxB> (1) Calculating local spectral similarities S(i, j, k) between a feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with classes different from the specific subregion Rn of the specific layer j and the prior label LB of the evaluation data ED, according to the evaluation data ED; (2) The maximum other-class spectral similarity RSp_maxB is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. <Maximum spectral similarity RSp_max> (1) Calculating local spectral similarities S(i,j,k) between a feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with the specific layer j and the specific subregion Rn according to the evaluation data ED; (2) The maximum spectral similarity RSp_max is calculated by taking the maximum, average, or minimum value of the local spectral similarities S(i,j,k) for the plurality of partial regions Rn. According to the second calculation method M2, the maximum spectral similarities RSp_max, RSp_maxA, and RSp_maxB can be found using relatively simple calculations and procedures.

[0054] Fig. 12 is an explanatory diagram showing a third calculation method M3 of the maximum spectral similarity. Fig. 12 shows a calculation method of the maximum spectral similarity RSp_max when the evaluation data ED is either the training data TD or the verification data VD. In the third calculation method M3, the maximum spectral similarities RSp_max, RSp_maxA, and RSp_maxB are calculated from the output of the specific layer, ConvVN2 layer 250, without calculating the local spectral similarity S(i,j,k). When the maximum spectral similarities RSp_max, RSp_maxA, and RSp_maxB are used without distinction, they are referred to as the maximum spectral similarity RSp.

[0055] The maximum spectral similarity RSp(i,j) obtained by the third calculation method M3 is calculated using the following equation. RSp(i,j)=max[G{Sp(j,k=all), KSp(i,j,k=all,q=all)}] (c3) where: Sp(j, k=all) is a feature spectrum obtained from the output of all partial regions Rn of a specific layer j according to the evaluation data ED. Note that when the evaluation data ED is abnormal data AD, the above parameter i specifies all classes, so KSp(i, j, k=all, q=all) in the above formula (c3) becomes KSp(i=all, j, k=all, q=all).

[0056] As described above, in the third maximum spectral similarity calculation method M3, the maximum spectral similarities RSp_maxA, RSp_maxB, and RSp_max are calculated in the following manner. <Maximum spectral similarity of own class RSp_maxA> (1) Calculating the self-class spectrum similarity RSp_A, which is the similarity between all feature spectra Sp obtained from the output of a specific layer j according to the evaluation data and all known feature spectra KSp associated with the specific layer j and the self-class i, (2) The maximum value among the multiple self-class spectral similarities RSp_A is set as the self-class maximum spectral similarity RSp_maxA. <Maximum other class spectral similarity RSp_maxB> (1) Calculating the other-class spectrum similarity RSp_B, which is the similarity between all feature spectra Sp obtained from the output of a specific layer j according to the evaluation data and all known feature spectra KSp associated with that specific layer j and other classes i, (2) The maximum value among the multiple other-class spectral similarities RSp_B is set as the other-class maximum spectral similarity RSp_maxB. <Maximum spectral similarity RSp_max> (1) According to the evaluation data ED, a spectral similarity RSp is calculated between a feature spectrum Sp obtained from the output of a specific subregion Rn of a specific layer j and all known feature spectra KSp associated with the specific layer j; (2) The maximum value among the multiple spectral similarities RSp is set as the maximum spectral similarity RSp_max. According to the third calculation method M3, the maximum spectral similarities RSp_maxA, RSp_maxB, and RSp_max can be found using even simpler calculations and procedures.

[0057] A-4. Distribution and representative values ​​of spectral similarity: As a method for calculating the distribution and representative value of the above-mentioned spectral similarity, for example, any one of the following three methods, a first representative value calculation method MR1 to a third representative value calculation method MR3, can be adopted.

[0058] (1) First representative value calculation method MR1: For each partial region Rn, the individual local spectral similarity SI_max, which is the maximum value of the spectral similarity RSp between the known feature spectrum KSp obtained from one piece of training data TD and the feature spectrum Sp of the evaluation data ED, is calculated for each of the multiple training data TD, and a histogram is created as a distribution. A partial representative value, which is the representative value of the histogram created for each partial region Rn, is determined. The partial representative value can be the maximum value, median value, or mode of the histogram. In the first representative value calculation method MR1, the correspondence between the known feature spectrum KSp and the partial region Rn in the feature spectrum Sp of the evaluation data ED is taken into consideration.

[0059] Alternatively, the partial representative value may be determined by the following method. In this method, first, at least one unimodal distribution is obtained by fitting a histogram created for each partial region Rn with a mixed Gaussian distribution using the EM algorithm (Expectation-Maximization Algorithm). If multiple unimodal distributions are generated, one representative unimodal distribution is determined using the following selection conditions C1 and C2. Note that if one unimodal distribution is obtained, this unimodal distribution is used as the representative unimodal distribution. <Condition C1> The ratio of the area of ​​one unimodal distribution to the total area of ​​the histogram of the individual local spectral similarity SI_max is equal to or greater than an area threshold. <Condition C2> Among the unimodal distributions that satisfy condition C1, the average value of the individual local spectral similarity SI_max is the largest.

[0060] The area threshold in the above condition C1 is set to, for example, a value of about 5 to 10%. The representative value of the representative unimodal distribution becomes the partial representative value of the histogram created for each partial region Rn. The representative value of the representative unimodal distribution is, for example, the mode of the representative unimodal distribution.

[0061] Then, the respective partial representative values ​​are integrated to generate a representative value PRSp of the distribution of the spectral similarity RSp. The integration process may be, for example, a process in which the maximum value among the plurality of partial representative values ​​is set as the representative value PRSp, or a process in which the average value of the plurality of partial representative values ​​is set as the representative value PRSp.

[0062] (2) Second representative value calculation method MR2: The individual local spectral similarity SI_max, which is the maximum value of the spectral similarity RSp between a known feature spectrum KSp obtained from one piece of training data TD and a feature spectrum Sp of the evaluation data ED for each partial region Rn, is calculated for each of the multiple training data TDs, and a histogram is created as a distribution. A partial representative value, which is a representative value of the histogram created for each partial region Rn, is determined. The partial representative value can be the maximum value, median value, or mode of the histogram. The second representative value calculation method MR2 differs from the first representative value calculation method MR1 in that it compares corresponding partial regions Rn in the known feature spectrum KSp and the feature spectrum Sp of the evaluation data ED. As with the first representative value calculation method MR1, a representative value PRSp may be determined from at least one unimodal distribution obtained by fitting with a Gaussian mixture distribution using an EM algorithm (Expectation-Maximization Algorithm).

[0063] (3) Third representative value calculation method MR3: Without considering the partial region Rn at all, an individual spectral similarity Sa_max, which is the maximum value of the spectral similarity between a known feature spectrum KSp obtained from one piece of training data TD and the spectral similarity RSp between the feature spectrum Sp of the evaluation data ED, is calculated for each of the multiple training data TD, and a histogram is created as a distribution. The maximum value, median, or mode of the created histogram is used as the representative value PRSp. Alternatively, as with the first representative value calculation method MR1, the representative value PRSp may be determined from at least one unimodal distribution obtained by fitting with a mixed Gaussian distribution using an EM algorithm (Expectation-Maximization Algorithm).

[0064] Regarding the above-described first representative value calculation method MR1 to third representative value calculation method MR3, when the evaluation data ED is the training data TD or the verification data VD, the representative value PRSp_A of the own class spectral similarity RSp_A and the representative value PRSp_B of the other class spectral similarity RSp_B are calculated separately for the own class and other classes. On the other hand, when the evaluation data ED is the abnormal data AD, the representative value PRSp is calculated without distinguishing between the own class and other classes.

[0065] In the first representative value calculation method MR1, the individual local spectral similarity SI_max(i,j,k,q) is calculated using the following equation. SI_max(i,j,k,q)=max[G{Sp(j,k), KSp(i,j,k=all,q)}] (d1) Furthermore, when the evaluation data ED is abnormal data AD, the parameter i specifies all classes, so KSp(i,j,k=all,q) in the above formula (d1) becomes KSp(i=all,j,k=all,q). The individual local spectral similarity SI_max in the above formula (d1) can be calculated in the process of calculating the local spectral similarity S(i,j,k) using the first calculation method M1.

[0066] In the second representative value calculation method MR2, the individual local spectral similarity SI_max(i,j,k,q) is calculated using the following equation. SI_max(i,j,k,q)=max[G{Sp(j,k), KSp(i,j,k,q)}] (d2) Furthermore, when the evaluation data ED is abnormal data AD, the above parameter i specifies all classes, so KSp(i,j,k,q) in the above equation (d2) becomes KSp(i=all,j,k,q). The individual local spectral similarity SI_max in the above formula (d2) can be calculated in the process of calculating the local spectral similarity S(i,j,k) using the second calculation method M2.

[0067] FIG. 13 is a diagram showing a histogram of the individual local spectral similarity SI_max. The vertical axis represents the number of data, i.e., frequency, and the horizontal axis represents the individual local spectral similarity SI_max. When the evaluation data ED is the training data TD or the verification data VD, a histogram is generated for each of the own class and other classes and for each subregion Rn. FIG. 13 shows a histogram for the own class as an example. The number written after the symbol SI in FIG. 13 is the number of the subregion Rn. When the specific layer is the ConvVN2 layer 250, there are nine subregions Rn, and therefore nine histograms are generated, one for the own class and one for the other classes. Note that when the evaluation data ED is the abnormal data AD, the individual local spectral similarity SI_max(i,j,k,q) is calculated without distinguishing between the own class and other classes. Therefore, when the specific layer is the ConvVN2 layer 250, nine histograms are generated, which corresponds to the number of subregions Rn.

[0068] For example, when the evaluation data ED is training data TD or verification data VD, the similarity calculation unit 310 calculates a representative value PRSp from each partial representative value calculated from the histogram corresponding to the partial region Rn for each of the own class and other classes.

[0069] In the third representative value calculation method MR3, the individual spectral similarity Sa_max(i,j,q) is calculated using the following equation. Sa_max(i,j,q)=max[G{Sp(j,k=all), KSp(i,j,k=all,q)}] (d3) Furthermore, when the evaluation data ED is abnormal data AD, the parameter i specifies all classes, so KSp(i,j,k=all,q) in the above formula (d3) becomes KSp(i=all,j,k=all,q). The above individual spectral similarity Sa_max can be calculated in the process of calculating the spectral similarity RSp using the third calculation method M3.

[0070] 14 is a diagram showing a histogram of individual spectral similarity Sa_max. The vertical axis represents the number of data, i.e., frequency, and the horizontal axis represents individual spectral similarity Sa_max. When evaluation data ED is training data TD or verification data VD, a histogram is generated for each of the own class and other classes. On the other hand, when evaluation data ED is abnormal data AD, a single histogram is generated without distinguishing between the own class and other classes.

[0071] A-5. Data similarity calculation method: FIG. 15 is a diagram illustrating the data similarity Da. FIG. 15 illustrates an example in which the specified layer is the ConvVN2 layer 250, and the numbers in the grid in FIG. 15 indicate the numbers of the partial regions Rn. Among the numbers in parentheses in the grid of the training data TD, the first number indicates the data number and the second number indicates the partial region Rn. When the local spectral similarity S(i,j,k) is calculated using the first calculation method M1 or the second calculation method M2, the similarity calculation unit 310 identifies, for each partial region Rn, the training partial data that is the partial data of the training data TD from which the local spectral similarity S(i,j,k) was generated. The similarity calculation unit 310 then calculates (i) for each partial region Rn, the partial data similarity Da_k between the evaluation partial data that is the partial data of the input data IM of the evaluation data ED and (ii) the training partial data that is the corresponding partial data of the input data IM of the training data TD. The "k" in the partial data similarity Da_k is a parameter indicating the partial region Rn. If there are nine partial regions Rn, nine partial data similarities Da_1 to Da_9 are calculated. The partial data similarity Da_k is calculated using, for example, cosine similarity, mean square error (MSE), peak signal-to-noise ratio (PSNR), or structural similarity index measure (SSIM). The similarity calculation unit 310 generates the data similarity Da by integrating the calculated multiple partial data similarities Da_k. For example, the similarity calculation unit 310 performs the integration process by setting the maximum value of the multiple partial data similarities Da_k as the data similarity Da, or by setting the average value of the multiple partial data similarities Da_k as the data similarity Da. The similarity calculation unit 310 may generate the data similarity Da using the following method. In this method, the similarity calculation unit 310 identifies partial training data that is partial data of the training data TD from which the local spectral similarity S calculated for each partial region Rn was generated, and that indicates the maximum or minimum value of the local spectral similarity S. Next, the similarity calculation unit 310 calculates partial data similarity Da_k between the identified partial training data and partial evaluation data that is partial data of the input data IM of the evaluation data ED. The similarity calculation unit 310 sets the calculated partial data similarity Da_k as data similarity Da.

[0072] 7, the process by which the similarity calculation unit 310 generates primary explanation information FEI includes the steps of inputting the evaluation data ED to the trained machine learning model 200 and determining a feature spectrum Sp for each partial region Rn from the output of a specific layer of the trained machine learning model 200. Also, as shown in FIGS. 10 to 12, the process by which the similarity calculation unit 310 generates primary explanation information FEI includes the steps of determining, for each of the multiple known feature spectra KSp in the known feature spectrum group KSpG, the similarity between the feature spectrum Sp for each partial region Rn and a known feature spectrum KSp for each partial region Rn that the known feature spectrum group KSpG has, and determining a maximum spectral similarity RSp_max from each similarity. Also, as shown in Figure 15, the process in which the similarity calculation unit 310 generates the primary explanation information FEI includes a process in which, for each partial region Rn, data similarity Da is calculated from the partial data of the input data IM of the evaluation data ED and the partial data similarity Da_k of the partial data of the input data IM in the training data TD, which is the source of calculation of the local spectral similarity S(i,j,k).

[0073] A-6. Detailed description of the evaluation process: Fig. 16 is a first flowchart showing details of step S30 in Fig. 7. Fig. 17 is a second flowchart showing details of step S30. Fig. 18 is a third flowchart showing details of step S30. Fig. 19 is a fourth flowchart showing details of step S30.

[0074] 16, in step S31, the type of evaluation data ED is determined. For example, if the processor 110 has previously received the type of evaluation data ED before executing the evaluation step, the evaluation unit 330 executes the determination in step S31 by referencing the previously received information. Furthermore, if type data indicating whether the evaluation data ED to be input to the trained machine learning model 200 is training data TD, verification data VD, or abnormal data AD is attached to the evaluation data ED, the evaluation unit 330 executes the determination in step S31 by referencing the attached type data.

[0075] 17, the evaluation unit 330 determines whether the value SpBr indicated by the other-class spectral similarity information IRSp_B is equal to or greater than the first other-class spectral similarity threshold thSpBr1. When the training data TD is used as the evaluation data ED, the value SpBr is, for example, the other-class spectral representative value PRSp_B. As described above, in step S32, a first learning comparison result is generated, which is the result of comparing the value SpBr indicated by the other-class spectral similarity information IRSp_B with the first other-class spectral similarity threshold thSpBr1.

[0076] If the determination in step S32 is "Yes," in step S33, the evaluation unit 330 determines whether the value DaBm indicated by the other class data similarity information IDa_B is equal to or greater than the first other class data similarity threshold thDaBm1. In this embodiment, the value DaBm indicated by the other class data similarity information IDa_B is the other class maximum data similarity Da_B. As described above, in step S33, a second learning comparison result is generated, which is the comparison result between the value DaBm indicated by the other class data similarity information IDa_B and the first other class data similarity threshold thDaBm1.

[0077] If the determination in step S33 is "Yes," i.e., if the value SpBr is equal to or greater than the first other-class spectrum similarity threshold thSpBr1 and the value Dabm is equal to or greater than the first other-class data similarity threshold thDaBm1, it is assumed that the following event has occurred. The trained machine learning model 200 has determined that the evaluation input data IM, which is input data of the training data TD as the evaluation data ED, is similar in data terms to the input data IM of the training data TD, which is of a different class from the evaluation data ED and whose known feature spectrum KSp is similar to the feature spectrum Sp of the evaluation input data IM. Therefore, it may be preferable that the input data IM of the other class that is similar to the evaluation input data IM be classified as the same class. It may also be preferable that the evaluation input data IM and the input data IM of the training data TD are not similar in data terms. Therefore, in this case, in step S34, the evaluation unit 330 generates, as secondary explanation information SEI, information that includes at least one of the following: that there is inappropriate defective data among the training data TD as the multiple evaluation data TD; and that the information in the training data TD as the evaluation data ED used for evaluation is insufficient as information necessary for class discrimination. Defective data is data that has characteristics that are significantly similar to normal training data of a different class. Examples of cases in which the information in the training data TD is insufficient as information necessary for class discrimination include when the resolution of the input data IM for the training data TD is too low or when the input data IM is from a region different from the region necessary for class discrimination.

[0078] If the determination in step S33 is “No,” i.e., if the value SpBr is equal to or greater than the first other-class spectral similarity threshold thSpBr1 and the value DaBm is less than the first other-class data similarity threshold thDaBm1, it is assumed that the following event has occurred. If the other-class spectral representative value PRSp_B as the value SpBr is large, it is assumed that the trained machine learning model 200 has determined that the evaluation input data IM of the evaluation data ED is similar in characteristics to the training data TD of another class different from the class indicated by the prior label LB associated with the evaluation input data IM. On the other hand, if the other-class maximum data similarity Da_B, which is the value DaBm, is low, it means that the evaluation input data IM and the input data IM of the training data TD of the other class, which the trained machine learning model 200 has determined to be similar in characteristics, are not similar as data using an index for calculating data similarity, such as the mean square error (MSE). In this case, it is considered that the trained machine learning model 200 may not be able to correctly classify the evaluation data ED. Therefore, in step S35, the evaluation unit 330 generates, as secondary explanation information SEI, information indicating that the machine learning model 200 may lack the ability to correctly classify the evaluation data ED.

[0079] In steps S34 and S35, secondary explanation information SEI is generated using the first learning comparison result generated in step S32 and the second learning comparison result generated in step S33. Note that steps S34 and S35 may be executed regardless of the number of evaluation data ED that satisfy the conditions, or may be executed when the number of evaluation data ED that satisfy the conditions is equal to or greater than a predetermined threshold.

[0080] If the determination in step S32 is "No," then in step S36, the evaluation unit 330 determines whether the value SpAr indicated by the own-class spectral similarity information IRSp_A is less than the first own-class spectral similarity threshold thSpAr1. In this embodiment, the value SpAr is the own-class spectral representative value PRSp_A. The evaluation unit 330 executes step S36 for each of the multiple evaluation data ED to count the number NmSpAr of evaluation data ED that satisfy the condition that the value SpAr is less than the first own-class spectral similarity threshold thSpAr1. Next, in step S37, the evaluation unit 330 determines whether the number NmSpAr is equal to or greater than a predetermined first data threshold thNm1. If the determination in step S37 is "Yes," then it is assumed that the following event has occurred. In other words, if a large number of evaluation data ED satisfy the condition, it is assumed that the machine learning model 200 has determined that the characteristics of the evaluation input data IM are not similar to those of the input data IM of the training data group TDG that is in the same class as the class indicated by the prior label LB associated with the evaluation input data IM. Therefore, in this case, the characteristics of the large number of evaluation input data IM may deviate from the characteristics of the input data IM of other training data TD that belong to the same class. Therefore, in step S38, the evaluation unit 330 generates first learning evaluation information indicating that there is a large variation among the characteristics of the multiple input data IM possessed by the multiple training data TD used as the evaluation data ED.

[0081] Furthermore, if the determination in step S37 is "No," step S39 is executed. That is, if the number of evaluation data ED that satisfy the condition is small, the evaluation unit 330 generates, as secondary explanation information SEI, second learning evaluation information indicating that there is outlier data among the multiple input data IM contained in the multiple learning data TD used as the evaluation data ED. Outlier data is data whose characteristics are significantly different from those of a normal learning data set in general.

[0082] If a "No" judgment is made for each evaluation data ED in step S36, in step S40, the evaluation unit 330 generates information indicating that the evaluation of the machine learning model 200 is normal as secondary explanation information SEI.

[0083] 18, when the evaluation data ED is the verification data VD, in step S52, the evaluation unit 330 determines whether the value SpBr indicated by the other-class spectral similarity information IRSp_B is equal to or greater than the second other-class spectral similarity threshold thSpBr2. When the verification data VD is used as the evaluation data ED, the value SpBr is, for example, the other-class maximum spectral similarity RSp_maxB. As described above, in step S52, a first verification comparison result is generated, which is the result of comparing the value SpBr indicated by the other-class spectral similarity information IRSp_B with the second other-class spectral similarity threshold thSpBr2. The second other-class spectral similarity threshold thSpBr2 may be the same as or different from the first other-class spectral similarity threshold thSpBr1.

[0084] If the determination in step S52 is "Yes," in step S53, the evaluation unit 330 determines whether the value DaBm indicated by the other class data similarity information IDa_B is equal to or greater than the second other class data similarity threshold thDaBm2, as in step S33. In step S53, a second verification comparison result is generated, which is the comparison result between the value DaBm indicated by the other class data similarity information IDa_B and the other class data similarity threshold thDaBm. The second other class data similarity threshold thDaBm2 may be the same value as the first other class data similarity threshold thDaBm1, or may be a different value.

[0085] If the determination in step S53 is "Yes," that is, if the value SpBr is equal to or greater than the second other-class spectral similarity threshold thSpBr2 and the value Dabm is equal to or greater than the second other-class data similarity threshold thDaBm2, it is assumed that the same phenomenon as that occurring when the evaluation data ED is training data TD has occurred. Therefore, in step S54, the evaluation unit 330 generates, as secondary explanation information SEI, information including at least one of the fact that there is inappropriate defective data among the multiple pieces of verification data ED and the fact that the information in the verification data ED is insufficient as information necessary for class discrimination.

[0086] If the determination in step S53 is "No," that is, if the value SpBr is equal to or greater than the second other-class spectral similarity threshold thSpBr2 and the value Dabm is less than the second other-class data similarity threshold thDaBm2, it is assumed that the same phenomenon as when the evaluation data ED is the training data TD has occurred. Therefore, in step S55, the evaluation unit 330 generates, as secondary explanation information SEI, information indicating that the machine learning model 200 may lack the ability to correctly classify the verification data VD, which is the evaluation data ED.

[0087] In the above steps S54 and S55, secondary explanation information SEI is generated using the first verification and comparison result generated in step S52 and the second verification and comparison result generated in step S53. Note that steps S54 and S55 may be executed regardless of the number of evaluation data ED that satisfy the conditions, or may be executed when the number of evaluation data ED that satisfy the conditions is equal to or greater than a predetermined threshold.

[0088] If the determination in step S52 is "No," the evaluation unit 330 determines in step S56 whether the value SpAr indicated by the own-class spectral similarity information IRSp_A is less than a second own-class spectral similarity threshold thSpAr2. When the verification data VD is used as the evaluation data ED, the value SpAr is, for example, the own-class maximum spectral similarity RSp_maxA. If the determination in step S56 is "No" for all of the evaluation data ED, the evaluation unit 330 generates, as secondary explanation information SEI, information indicating that the evaluation of the machine learning model 200 is normal. The second own-class spectral similarity threshold thSpAr2 may be the same value as or different from the first own-class spectral similarity threshold thSpAr1.

[0089] If the determination in step S56 is "Yes," the evaluation unit 330 determines in step S57 whether the value DaAm indicated by the own-class data similarity information IDa_A is equal to or greater than the own-class data similarity threshold thDaAm. In this embodiment, the value DaAm indicated by the own-class data similarity information IDa_A is the own-class maximum data similarity Da_A. If the determination in step S57 is "Yes" for each evaluation data ED, that is, if the value SpAr indicated by the own-class spectral similarity information IRSp_A is less than the second own-class spectral similarity threshold thSpAr2 and the value DaAm indicated by the own-class data similarity information IDa_A is equal to or greater than the own-class data similarity threshold thDaAm, it is assumed that the following event has occurred. In other words, the machine learning model 200 determines that the features of the evaluation input data IM, which is input data for the verification data VD, are dissimilar to the features of the input data IM for the training data TD, which is the same class as the class indicated by the prior class LB of the verification data VD. On the other hand, the input data IM for the evaluation input data IM determined by the trained machine learning model 200 to have the most similar features and the evaluation input data IM are similar in data terms using an index for calculating data similarity, such as mean square error (MSE). As described above, the machine learning model 200 may determine that even if the evaluation input data IM is similar in data terms to the input data IM used for training, it belongs to a different class if the evaluation input data IM was not used for training. Therefore, in step S59, the evaluation unit 330 generates information indicating that overfitting of the machine learning model 200 will occur as secondary explanation information SEI.

[0090] If the determination in step S57 is "No," the evaluation unit 330 executes steps S56 and S57 for each of the multiple evaluation data ED, and counts the number NmCa of evaluation data ED that satisfy the conditions that the value SpAr is less than the second own-class spectral similarity threshold thSpAr2 and the value DaAm is less than the own-class data similarity threshold thDaAm. Next, in step S60, the evaluation unit 330 determines whether the number NmCa is equal to or greater than the predetermined second data threshold thNm2. If the determination in step S60 is "Yes," i.e., if the number NmCa is equal to or greater than the predetermined second data threshold thNm2, it is assumed that the following event has occurred. In other words, the machine learning model 200 has determined that the features of many of the evaluation input data IM are not similar to the features of the input data IM of the training data TD of the own class that is the same class as the class indicated by the prior class LB of the verification data VD. Furthermore, the machine learning model 200 determines that the evaluation input data IM and the input data IM determined to be most similar in terms of characteristics are not similar in terms of data similarity. When the number of verification data VD for which the above determination has been made is large, it is expected that there is a large deviation in characteristics between the evaluation input data IM of the verification data VD and the input data IM of the training data TD. Therefore, in step S62, the evaluation unit 330 generates, as secondary explanation information SEI, first verification evaluation information indicating that there is a large deviation in characteristics between each evaluation input data IM as input data contained in the multiple verification data and the input data IM contained in the training data TD.

[0091] If the answer is "No" in step S60, that is, if the number NmCa is less than the predetermined second data threshold thNm2, it is assumed that the following event has occurred: That is, it is assumed that outlier data exists among the multiple input data IM included in the multiple verification data VD. Therefore, in step S64, the evaluation unit 330 generates, as secondary explanation information SEI, second verification evaluation information indicating that outlier data exists among the multiple input data IM included in the multiple verification data VD.

[0092] When the evaluation data ED shown in Figure 18 above is verification data VD, the value SpAr indicated by the own class spectral similarity information IRSp_A and the value SpBr indicated by the other class spectral similarity information IRSp_B may be the own class spectral representative value PRSp_A and the other class spectral representative value PRSp_B.

[0093] 19, if the evaluation data ED is abnormal data AD, in step S72, the evaluation unit 330 determines whether the maximum spectral similarity RSp is equal to or greater than a predetermined abnormal spectrum threshold thRSp. If the determination in step S72 is "No," in step S76, the evaluation unit 330 generates, as secondary explanation information SEI, information indicating that the evaluation of the machine learning model 200 is normal.

[0094] If the determination in step S72 is "Yes," the evaluation unit 330 determines whether the maximum data similarity Da_max is equal to or greater than a predetermined abnormal data similarity threshold thDa. If the determination in step S73 is "Yes," that is, if the maximum spectral similarity RSp_max is equal to or greater than the abnormal spectrum threshold thRSp and the maximum data similarity Da_max is equal to or greater than the abnormal data similarity threshold thDa, the following event is assumed to have occurred. That is, if the evaluation data ED is abnormal data AD, the maximum data similarity Da_max is expected to be low. However, if the maximum data similarity Da_max is high, equal to or greater than the abnormal data similarity threshold thDa, this means that the abnormal data AD does not possess information indicating that it is abnormal. Therefore, in step S74, the evaluation unit 330 generates, as secondary explanation information SEI, information indicating that the information on the abnormal data AD is insufficient for class discrimination.

[0095] If the determination in step S73 is "No," that is, if the maximum spectral similarity RSp_max is equal to or greater than the abnormal spectrum threshold thRSp and the maximum data similarity Da_max is less than the abnormal data similarity threshold thDa, it is assumed that the following event has occurred. That is, an event similar to that occurring when step S35 is executed when the evaluation data ED is the training data TD or when step S55 is executed when the evaluation data ED is the verification data VD has occurred. Therefore, in step S75, the evaluation unit 330 generates, as secondary explanation information, information indicating that the machine learning model 200 may lack the ability to correctly classify the abnormal data AD, that is, the ability to classify it as an unknown class.

[0096] The various thresholds used in the evaluation process described above with reference to Figures 16 to 19 may be set by the user or may be set in advance. Furthermore, the various thresholds may be determined using the indices of the various histograms representing the various distributions described above, such as the standard deviation σ, ±2σ, or ±3σ. This standard deviation σ may be the standard deviation of the representative unimodal distribution identified by the EM algorithm.

[0097] A-7. Solution: The following describes how to deal with the generated secondary explanation information SEI. When steps S35 and S55 are executed and information indicating that the machine learning model 200 may not be able to correctly classify the evaluation data ED is generated as secondary explanation information SEI, the following measures may be taken. <Solution Method 1A> Review the network configuration of machine learning model 200. In the target method 1A, the network configuration is revised, for example, by increasing the number of vector neuron layers, etc. Also, in the target method 1A, for example, the number of layers in the network of the machine learning model 200 is increased, or the specific layer from which the feature spectrum Sp is obtained is changed.

[0098] When step S75 is executed and secondary explanation information SEI is generated indicating that the machine learning model 200 may not be able to correctly classify the evaluation data ED, the following countermeasures may be considered. <Solution 1B> The learning data TD is modified to be learning data more suitable for learning for class discrimination, and the machine learning model 200 is trained. In countermeasure method 1B, for example, data processing is performed on the input data IM of the training data TD to delete elements that show simple features. Also, for example, the prior label LB is further subdivided and associated with the input data IM. Note that countermeasure method 1B may include the same countermeasure as countermeasure method 1A.

[0099] When step S59 is executed and information indicating that overlearning of the machine learning model 200 will occur is generated, the following countermeasures can be considered. <Solution 1C> Review the learning parameters of machine learning model 200. In the subject method 1C, for example, the number of epochs is reduced, or the batch size in mini-batch learning is reduced.

[0100] Furthermore, in step S34, step S54, or step S74, if information is generated indicating that the information in the evaluation data ED is insufficient as information required for class discrimination, the following countermeasures can be considered. <Solution 1D> Change the data resolution of the input data IM. For example, increasing the data resolution of characteristic regions for class discrimination. <Solution 1E> Preprocessing such as average difference is introduced to the original data of the input data IM. <Solution 1F> Review the conditions for obtaining input data IM. For example, the distance between the target object and the imaging device is changed, or light is irradiated onto the target object.

[0101] Furthermore, in step S34 or step S54, if information indicating that there is defective data among the plurality of evaluation data ED is generated, the following countermeasures can be considered. <Solution 1G> Review the pre-labels LB of the training data TD. For example, if different a priori labels LB are associated with a plurality of pieces of input data IM having similar characteristics, the same new a priori labels LB are associated with the plurality of pieces of input data IM. <Solution Method 1H> Reduce the training data TD. For example, if different a priori labels LB are associated with multiple input data IM having similar characteristics, only the input data IM associated with one a priori label LB is retained, and the remaining input data IM are deleted.

[0102] If information indicating that outlier data exists is generated in step S39 or step S64, the following countermeasures can be considered. <Solution 1I> Expand the training data TD. For example, input data IM having characteristics similar to those of outlier data is added as training data TD. <Solution 1J> The outlier data is removed from the training data TD. <Solution 1K> Preprocessing is performed on the training data TD and validation data VD. Examples of pre-processing include smoothing and normalization when it is assumed that outlier data is determined to be present due to noise.

[0103] If the first learning assessment information is generated in step S38, the following measures may be taken. <Solution 1L> Expand the training data TD. For example, input data IM for reducing variability is newly added as learning data TD. <Solution 1M> The input data IM that is causing the large variation is deleted from the training data TD. <Solution 1N> Preprocessing is performed on the training data TD. Examples of pre-processing include smoothing and normalization when it is assumed that the variations are large due to noise.

[0104] In step S62, if the first verification and evaluation information is generated, the following measures can be taken. <Solution 1O> Expand the training data TD. New learning data TD is added so that the difference in features between each piece of evaluation input data IM and the input data IM contained in the learning data TD is reduced. <Solution Method 1> Preprocessing is performed on the training data TD. Examples of pre-processing include smoothing and normalization.

[0105] In step S74, if information indicating that the information of the abnormal data AD is insufficient as information required for class discrimination is generated, the following countermeasures can be considered. <Solution 1Q> Change a specific layer. For example, if the features that affect class discrimination appear in the fine shape, a lower layer is used as the specific layer. For example, the specific layer is changed from the ConvVN2 layer 250 to the ConvVN1 layer 240.

[0106] The evaluation unit 330 may display the contents of the above-mentioned countermeasures 1A to 1K on the display unit 150. This allows the user to easily understand the countermeasures regardless of their level of experience.

[0107] A-8.Specific examples of how to deal with this: A-8-1. First example: A plurality of training data TD were input as evaluation data ED to the trained machine learning model 200, and class discrimination was performed using activation values ​​corresponding to the judgment values ​​of each class output from the ClassVN layer 260. As a result, the accuracy rate of the class discrimination was lower than the desired value. In this example, the ClassVN layer 260 is the specific layer. The evaluation unit 330 generated secondary explanation information SEI using the primary explanation information FEI. In this case, the secondary explanation information SEI was generated in step S55 shown in FIG. 18 as information indicating that the machine learning model 200 may not be able to correctly classify the training data TD used as the evaluation data ED. Therefore, the number of vector neurons in the machine learning model 200 was increased, and training of the machine learning model 200 was performed again. As a result, the accuracy rate was higher than the desired value.

[0108] A-8-2. Second specific example: A trained machine learning model 200 was input with multiple validation data VD as evaluation data ED, and class discrimination was performed using activation values ​​corresponding to the judgment values ​​for each class output from the ClassVN layer 260. As a result, the accuracy rate of class discrimination was lower than desired. In this example, the ClassVN layer 260 is a specific layer. The evaluation unit 330 generated secondary explanation information SEI using primary explanation information FEI. In this case, first verification evaluation information was generated in step S62 shown in FIG. 18. Then, analysis of the training data TD and the validation data VD based on this secondary explanation information SEI confirmed that the validation data VD had different characteristics from the training data TD. Specifically, it was found that the input data IM for the validation data VD contained noise due to differences in the settings of the device used to acquire the training data TD and the input data IM for the validation data VD. Therefore, a smoothing process was performed as preprocessing on the input data IM for the validation data VD. The verification data VD after the smoothing process was input to the machine learning model 200 and an evaluation process was performed, resulting in the generation of information indicating normality as secondary explanation information. The accuracy rate of the class discrimination of the machine learning model 200 was calculated using the verification data VD after the smoothing process, and the accuracy rate exceeded the desired value.

[0109] A-8-3. Third example: An evaluation process was performed on the trained machine learning model 200 by inputting abnormal data AD, which was expected to be classified as an unknown class, as evaluation data ED. Classification was performed using activation values ​​corresponding to the judgment values ​​for each class output from the ClassVN layer 260. In this example, the ClassVN layer 260 was used as an identification layer. The evaluation unit 330 generated secondary explanation information SEI using the primary explanation information FEI. In this case, the secondary explanation information SEI was generated in step S74 shown in FIG. 19 by indicating that the information in the abnormal data AD was insufficient for classification. Analysis of the abnormal data AD based on this information revealed that the area of ​​data representing mutations in the target object, which is a factor in classifying the target object as an unknown class, was relatively small compared to the area indicated by the abnormal data AD. Therefore, the identification layer used to generate the primary explanation information FEI was changed from the ClassVN layer 260 to the ConvVN2 layer 250. After changing the specific layer, primary explanation information FEI was generated based on the output of the ConvVN2 layer 250, and secondary explanation information SEI was generated from this primary explanation information FEI. In this case, the secondary explanation information SEI indicated "normal." The class accuracy rate, which is the percentage of multiple abnormal data AD classified as an unknown class, also increased. The class accuracy rate can also be evaluated using spectral similarity RSp. In this case, changing the layer that generates the primary explanation information FEI may also change the calculated class accuracy rate. In this case, it is preferable to input the training data TD and the validation data VD into the machine learning model 200 to reconfirm whether the class accuracy rate is above the desired value.

[0110] A-9. How to calculate the output vector of each layer of a machine learning model: The method for calculating the output of each layer in the machine learning model 200 shown in FIG. 2 is as follows.

[0111] Each node in the PrimeVN layer 230 regards the scalar output of the 1x1x32 nodes in the Conv layer 220 as a 32-dimensional vector and obtains the vector output of that node by multiplying this vector by a transformation matrix. This transformation matrix is ​​an element of a kernel with a surface size of 1x1, and is updated by learning the machine learning model 200. Note that the processing of the Conv layer 220 and the PrimeVN layer 230 can also be integrated into one primary vector neuron layer.

[0112] When the PrimeVN layer 230 is referred to as the "lower layer L" and the ConvVN1 layer 240 adjacent to it on the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following equation.

number

[0113] As the normalization function F(X), for example, the following formula (E3a) or (E3b) can be used.

number

[0114] In the above equation (E3a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function | j On the other hand, in equation (E3b), the sum vector u j Norm of |u j | is the norm |u j Activation value a by dividing by the sum of | j It should be noted that a function other than equation (E3a) or (E3b) may be used as the normalization function F(X).

[0115] The ordinal number i in the above equation (E2) is the output vector M of the jth node in the upper layer L+1. L+1 j The integer n is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine . Thus, the integer n is given by n = Nk × Nc (E5) Here, Nk is the surface size of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 230. In the example of FIG. 2, Nk=9 and Nc=16, so n=144.

[0116] One kernel used to calculate the output vector of the ConvVN1 layer 240 has a surface size of 3 × 3 and a depth of 16 channels in the lower layer, with 3 × 3 × 16 = 144 elements. Each of these elements is a prediction matrix W L ij In addition, 12 sets of this kernel are required to generate output vectors for 12 channels of the ConvVN1 layer 240. Therefore, the prediction matrix W of the kernel used to obtain the output vector of the ConvVN1 layer 240 is L ij The number of prediction matrices W is 144 × 12 = 1728. L ij is updated by learning of the machine learning model 200.

[0117] As can be seen from the above equations (E1) to (E4), the output vector M of each node in the upper layer L+1 L+1 j is calculated by the following calculation: (a) Output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is a linear combination of j Seeking (c) Sum vector u j Norm of |u j The activation value a is normalized by normalizing | j Seeking (d) Sum vector u j norm |u j Divide by | and then use the activation value a j Multiply by.

[0118] In addition, the activation value a j is the norm |u jis the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in equations (E3), (E3a), (E3b), and (4) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of j is only used in the above equations (E3) and (E4), and does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 so that it outputs

[0119] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above formulas (E1) to (E4) used in a vector neural network are different from the calculations used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v on the right side of the above formula (E2) ij are multiplied by weights, and the weights are searched by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M is calculated by calculating the above-mentioned equations (E1) to (E4) once in order. L+1 j Therefore, there is no need to repeat dynamic routing, which has the advantage of allowing faster calculations. In addition, the vector neural network of this embodiment has the advantage that it requires less memory for calculations than a capsule network, and according to experiments by the inventors of this disclosure, it only requires about 1 / 2 to 1 / 3 of the memory required.

[0120] Vector neural networks are similar to capsule networks in that they use nodes that use vectors as input and output. Therefore, they share the advantages of using vector neurons with capsule networks. Furthermore, the multiple layers 210-250 are similar to conventional convolutional neural networks in that the higher layers represent features of larger areas and the lower layers represent features of smaller areas. Here, "feature" refers to a characteristic part contained in the input data to the neural network. Vector neural networks and capsule networks are superior to conventional convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of a node's output vector represents the probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the relative positions of the respective features. Alternatively, the vector direction of the output vectors of the two nodes can be said to represent the variation of the feature. For example, for a node corresponding to the "eye" feature, the direction of the output vector can represent variations such as the narrowness of the eyes or the way they are lifted. In conventional convolutional neural networks, it is said that spatial information of features is lost due to the pooling process. As a result, vector neural networks and capsule networks have the advantage of being superior to conventional convolutional neural networks in terms of the performance of identifying input data.

[0121] The advantages of vector neural networks can also be considered as follows. In other words, the advantage of vector neural networks is that the output vectors of nodes represent the features of input data as coordinates in continuous space. Therefore, output vectors can be evaluated such that the closer the vector directions, the more similar the features. Another advantage is that even if the features contained in the input data are not covered by the training data, they can be determined by interpolation. On the other hand, conventional convolutional neural networks have the disadvantage that the features of input data cannot be represented as coordinates in continuous space due to the chaotic compression caused by the pooling process.

[0122] The outputs of each node in the ConvVN2 layer 250 and the ClassVN layer 260 are similarly determined using the above-mentioned equations (E1) to (E4), and therefore detailed explanations are omitted. The ClassVN layer 260, which is the top layer, has a resolution of 1×1 and M channels.

[0123] The output of the ClassVN layer 260 is converted into a plurality of decision values ​​Class1 to Class2 for known classes. These decision values ​​are usually normalized by a softmax function. Specifically, for example, the decision value for each class can be obtained by performing the following operation: calculating the vector length of the output vector from the output vector of each node of the ClassVN layer 260, and then normalizing the vector length of each node by a softmax function. As described above, the activation value a obtained by the above formula (E3) is j is the output vector M L+1 j The activation value a at each node in the ClassVN layer 260 is a value corresponding to the vector length of j may be output and used as the judgment value for each class.

[0124] In the above-described embodiment, a vector neural network that determines an output vector by calculating the above equations (E1) to (E4) was used as the machine learning model 200, but instead, a capsule network disclosed in U.S. Pat. No. 5,210,798 or WO 2009 / 083553 may be used.

[0125] According to the above embodiment, secondary explanation information SEI indicating an evaluation of the trained machine learning model 200 can be generated and output using primary explanation information FEI including spectral similarity information IRSp and data similarity information IDa. This allows the trained machine learning model to be evaluated without differences between users. Furthermore, based on the evaluation of the machine learning model 200, improvements such as increasing the accuracy rate of the machine learning model 200 can be efficiently performed.

[0126] B. Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0127] (1) According to a first aspect of the present disclosure, there is provided a method for evaluating a trained machine learning model. The machine learning model is a vector neural network model having multiple vector neuron layers, and is trained using multiple pieces of training data each having input data and a prior label associated with the input data. This evaluation method includes: (a) inputting the evaluation data into the trained machine learning model to generate primary explanatory information used for evaluating the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanatory information to generate secondary explanatory information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanatory information. The step (a) includes: (a1) inputting the evaluation data into the trained machine learning model to obtain a feature spectrum from the output of a specific layer of the trained machine learning model; (a2) calculating the feature spectrum and The method includes the steps of: (a) calculating, for each of a plurality of known feature spectra included in a group of known feature spectra obtained from the output of the specific layer by inputting the plurality of training data again into the trained machine learning model, spectral similarity, which is the similarity between the plurality of training data and a known feature spectrum included in the group of known feature spectra obtained from the output of the specific layer; (a3) ​​calculating data similarity, which is the similarity between the input data and the evaluation data; and (a4) generating the primary explanation information including spectral similarity information on the spectral similarity and data similarity information on the data similarity. According to this aspect, secondary explanation information indicating an evaluation of the trained machine learning model can be generated and output using the primary explanation information including the spectral similarity information and the data similarity information, thereby enabling the trained machine learning model to be evaluated without differences between users.

[0128] (2) In the above aspect, in the case where at least one of (i) the learning data and (ii) verification data that has not been used in training the machine learning model and has the input data and the a priori label associated with the input data is used as the evaluation data in the step (a), the step (a2) includes: a step of calculating, for each of the plurality of the a priori known feature spectra, an own-class spectral similarity which is the spectral similarity between the feature spectrum and a known feature spectrum of the own-class that is of the same class as the evaluation class indicated by the a priori label associated with the evaluation data among the group of known feature spectra; and a step of calculating, for each of the plurality of other-class known feature spectra, an other-class spectral similarity which is the spectral similarity between the feature spectrum and a known feature spectrum of the other class that is of a class different from the evaluation class among the group of known feature spectra; Step (a3) ​​may include: calculating an own-class maximum data similarity, which is the similarity between the input data associated with the own-class known feature spectrum, which is a source of calculation of the own-class maximum spectral similarity, which indicates the maximum value among the plurality of own-class spectral similarities, and the evaluation data; and calculating an other-class maximum data similarity, which is the similarity between the input data associated with the other-class known feature spectrum, which is a source of calculation of the other-class maximum spectral similarity, which indicates the maximum value among the plurality of other-class spectral similarities, and the evaluation data. Step (a4) may include generating the primary explanation information including own-class spectral similarity information related to the own-class maximum spectral similarity, own-class data similarity information related to the own-class maximum data similarity, other-class spectral similarity information related to the other-class maximum spectral similarity, and other-class data similarity information related to the other-class maximum data similarity. According to this aspect, secondary explanation information, which indicates a more detailed evaluation of the trained machine learning model, can be generated and output using primary explanation information including a wider variety of information. This allows efficient improvement of the machine learning model based on the evaluation of the machine learning model.

[0129] (3) In the above-described embodiment, when the learning data is used as the evaluation data in step (a), step (b) may include (b1) generating the secondary explanation information using a first learning comparison result of a value indicated by the other-class spectral similarity information and a predetermined first other-class spectral similarity threshold, and a second learning comparison result of a value indicated by the other-class data similarity information and a predetermined first other-class data similarity threshold. According to this embodiment, the secondary explanation information can be generated using the first learning comparison result and the second learning comparison result.

[0130] (4) In the above embodiment, step (b1) may include generating, as the secondary explanation information, information indicating at least one of the following: that the training data contains inappropriate defective data and that the training data lacks information necessary for class discrimination, when the value indicated by the other-class spectral similarity information is equal to or greater than the first other-class spectral similarity threshold and the value indicated by the other-class data similarity information is equal to or greater than the first other-class data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0131] (5) In the above embodiment, step (b1) may include generating, as the secondary explanation information, information indicating that the machine learning model may lack the ability to correctly classify the evaluation data when the value indicated by the other-class spectral similarity information is equal to or greater than the first other-class spectral similarity threshold and the value indicated by the other-class data similarity information is less than the first other-class data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0132] (6) In the above-described embodiment, when a plurality of pieces of training data are used as the evaluation data in step (a), step (b) may include (b2) generating, as the secondary explanation information, at least one of first learning evaluation information indicating that the input data contained in the plurality of training data used as the evaluation data has a large variance, and second learning evaluation information indicating that the input data contained in the plurality of training data used as the evaluation data has outlier data, if the value indicated by the own-class spectral similarity information is less than a predetermined first own-class spectral similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0133] (7) In the above-described embodiment, step (b2) may generate the first learning evaluation information as the secondary explanation information when the number of the training data serving as the evaluation data satisfying the condition that the value indicated by the own-class spectral similarity information is less than the first own-class spectral similarity threshold is equal to or greater than a predetermined first data threshold, and may generate the second learning evaluation information as the secondary explanation information when the number of the training data serving as the evaluation data satisfying the condition that the value indicated by the own-class spectral similarity information is less than the first own-class spectral similarity threshold is less than the first data threshold. This embodiment allows the generation of secondary explanation information indicating a more specific evaluation. This allows the machine learning model, the evaluation data, and the training data to be more efficiently improved based on the evaluation of the machine learning model.

[0134] (8) In the above-described embodiment, when the verification data is used as the evaluation data in step (a), step (b) may include (b3) generating the secondary explanation information using a first verification comparison result between the value indicated by the other-class spectral similarity information and a predetermined second other-class spectral similarity threshold, and a second verification comparison result between the value indicated by the other-class data similarity information and the predetermined second other-class data similarity threshold. According to this embodiment, the secondary explanation information can be generated using the first verification comparison result and the second verification comparison result.

[0135] (9) In the above embodiment, step (b3) may include generating, as the secondary explanation information, information indicating at least one of the following: that the verification data contains inappropriate, incomplete data; and that the verification data lacks information necessary for class discrimination, if the value indicated by the other-class spectral similarity information is equal to or greater than the second other-class spectral similarity threshold and the value indicated by the other-class data similarity information is equal to or greater than the second other-class data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0136] (10) In the above embodiment, step (b3) may include generating, as the secondary explanation information, information indicating that the machine learning model may lack the ability to correctly classify the evaluation data when the value indicated by the other-class spectral similarity information is equal to or greater than the second other-class spectral similarity threshold and the value indicated by the other-class data similarity information is less than the second other-class data similarity threshold. According to this embodiment, secondary explanation information indicating a more specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0137] (11) In the above-described embodiment, when a plurality of pieces of verification data are used as the evaluation data in step (a), step (b) may include (b4) generating, as the secondary explanation information, at least one of first verification evaluation information indicating a large deviation in characteristics between the input data included in the plurality of verification data and the input data included in the training data, and second verification evaluation information indicating that outlier data exists among the plurality of pieces of input data included in the plurality of verification data, if the value indicated by the own-class spectral similarity information is less than a predetermined second own-class spectral similarity threshold and the value indicated by the own-class data similarity information is less than a predetermined own-class data similarity threshold. This embodiment enables the generation of secondary explanation information indicating a more specific evaluation. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0138] (12) In the above-described embodiment, step (b4) may generate the first verification evaluation information as the secondary explanation information when the number of verification data serving as the evaluation data that satisfies the conditions that the value indicated by the own-class spectral similarity information is less than the second own-class spectral similarity threshold and the value indicated by the own-class data similarity information is less than the own-class data similarity threshold is equal to or greater than a predetermined second data threshold, and may generate the second verification evaluation information as the secondary explanation information when the number of verification data serving as the evaluation data that satisfies the conditions that the value indicated by the own-class spectral similarity information is less than the second own-class spectral similarity threshold and the value indicated by the own-class data similarity information is less than the own-class data similarity threshold is less than the second data threshold. This embodiment allows the generation of secondary explanation information that indicates a more specific evaluation. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0139] (13) In the above embodiment, step (b3) may include generating, as the secondary explanation information, information indicating that overlearning of the machine learning model has occurred when the value indicated by the own-class spectral similarity information is less than the second own-class spectral similarity threshold and the value indicated by the own-class data similarity information is equal to or greater than the own-class data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, evaluation data, and training data based on the evaluation of the machine learning model.

[0140] (14) In the above-described embodiment, in step (a4), the own-class spectral similarity information may include information on at least one of a representative value of a distribution of the plurality of own-class spectral similarities and the maximum own-class spectral similarity, and the other-class spectral similarity information may include information on at least one of a representative value of a distribution of the plurality of other-class spectral similarities and the maximum other-class spectral similarity. According to this embodiment, the distribution of spectral similarities or the maximum spectral similarity can be used as the own-class spectral similarity information or the other-class spectral similarity information.

[0141] (15) In the above-described embodiment, when step (a) uses, as the evaluation data, abnormal data that is not associated with the a priori label and is assumed to be classified into an unknown class different from the class corresponding to the a priori label, step (a2) may include a step of identifying a maximum spectral similarity that is a maximum value among the spectral similarities calculated for the plurality of known feature spectra, step (a3) ​​may include a step of identifying a maximum data similarity that is a similarity between the input data associated with the known feature spectrum that is the source of calculation of the maximum spectral similarity specified in step (a2) and the abnormal data, and step (a4) may include a step of generating the primary explanation information that includes spectral similarity information regarding the spectral similarities and the maximum data similarity. According to this embodiment, secondary explanation information can be generated and output using the abnormal data.

[0142] (16) In the above-described embodiment, when the abnormal data is used as the evaluation data in step (a), step (b) may include generating, as the secondary explanation information, information indicating that the machine learning model may lack the ability to correctly classify the abnormal data if the maximum spectral similarity is equal to or greater than a predetermined abnormal spectrum threshold and the maximum data similarity is less than a predetermined abnormal data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0143] (17) In the above-described embodiment, when the abnormal data is used as the evaluation data in step (a), step (b) may include generating, as the secondary explanation information, information indicating that information on the abnormal data is insufficient as information required for class discrimination when the maximum spectral similarity is equal to or greater than a predetermined abnormal spectrum threshold and the maximum data similarity is equal to or greater than a predetermined abnormal data similarity threshold. According to this embodiment, secondary explanation information indicating a specific evaluation can be generated. This allows for more efficient improvement of the machine learning model, the evaluation data, and the training data based on the evaluation of the machine learning model.

[0144] (18) According to a second aspect of the present disclosure, there is provided an evaluation device for a trained machine learning model, the evaluation device including: a memory for storing the machine learning model, the machine learning model being a vector neural network model having a plurality of vector neuron layers, trained using a plurality of training data having input data and a prior label associated with the input data; and a processor, the processor executing: (a) inputting the evaluation data into the trained machine learning model to generate primary explanation information to be used for evaluation of the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanation information to generate secondary explanation information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanation information, the process (a) including: (a1) The method includes: (a1) inputting the evaluation data into a trained machine learning model and determining a feature spectrum from an output of a specific layer of the trained machine learning model; (a2) determining, for each of a plurality of known feature spectra included in a group of known feature spectra, a spectral similarity between the feature spectrum and a known feature spectrum included in a group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model; (a3) ​​determining a data similarity between the input data and the evaluation data; and (a4) generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity. According to this aspect, secondary explanation information indicating an evaluation of the trained machine learning model can be generated and output using the primary explanation information including the spectral similarity information and the data similarity information, thereby enabling the trained machine learning model to be evaluated without differences between users.

[0145] (19) According to a third aspect of the present disclosure, there is provided a computer program for causing a computer to execute evaluation of a trained machine learning model. The machine learning model is a vector neural network model having multiple vector neuron layers, and is trained using multiple pieces of training data each having input data and a prior label associated with the input data. The computer program causes a computer to execute the following functions: (a) inputting evaluation data into the trained machine learning model to generate primary explanatory information to be used in evaluating the machine learning model; (b) using values ​​indicated by each piece of information in the primary explanatory information to generate secondary explanatory information indicating an evaluation of the trained machine learning model; and (c) outputting the generated secondary explanatory information. The function (a) includes: (a1) inputting the evaluation data into the trained machine learning model to obtain a feature spectrum from the output of a specific layer of the trained machine learning model; and (a2) outputting the secondary explanatory information. The system has (a) a function for calculating, for each of a plurality of known feature spectra included in a group of known feature spectra, a spectral similarity between a feature spectrum and a known feature spectrum included in a group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model, (a3) ​​a function for calculating a data similarity between the input data and the evaluation data, and (a4) a function for generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity. According to this aspect, secondary explanation information indicating an evaluation of the trained machine learning model can be generated and output using the primary explanation information including the spectral similarity information and the data similarity information, thereby enabling the trained machine learning model to be evaluated without differences between users.

[0146] The present disclosure may be realized in various forms other than those described above, such as a non-transitory storage medium on which a computer program is recorded. [Explanation of symbols]

[0147] 5...Evaluation system, 100...Evaluation device, 110...Processor, 112...Learning execution unit, 113...Classification unit, 114...Evaluation processing unit, 120...Memory, 130...Interface circuit, 140...Input device, 150...Display unit, 200...Machine learning model, 210...Input layer, 220...Conv layer, 230...PrimeVN layer, 240...ConvVN1 layer, 250...ConvVN2 layer, 260...Output layer, 280...Intermediate layer, 310...Similarity calculation unit, 330...Evaluation unit, 400...Imaging device, FEI...Primary explanation information, IM...Input data, SEI...Secondary explanation information

Claims

1. A method for evaluating a trained machine learning model executed by a computer, comprising: the machine learning model is a vector neural network type model having a plurality of vector neuron layers, and is trained using a plurality of training data having input data and prior labels associated with the input data; The evaluation method includes: (a) inputting evaluation data into the trained machine learning model to generate primary explanatory information to be used in evaluating the machine learning model; (b) generating secondary explanation information indicating an evaluation of the trained machine learning model using values ​​indicated by each piece of information contained in the primary explanation information; (c) outputting the generated secondary explanatory information; The step (a) (a1) inputting the evaluation data into the trained machine learning model and obtaining a feature spectrum from an output of a specific layer of the trained machine learning model; (a2) calculating, for each of the plurality of known feature spectra in the group of known feature spectra, a spectral similarity between the feature spectrum and a known feature spectrum included in the group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model; (a3) calculating a data similarity between the input data and the evaluation data; (a4) generating the primary explanation information including spectral similarity information related to the spectral similarity and data similarity information related to the data similarity.

2. The evaluation method according to claim 1, In the step (a), when at least one of (i) the training data and (ii) validation data that has not been used in training the machine learning model and includes the input data and the prior label associated with the input data is used as the evaluation data, The step (a2) calculating, for each of the plurality of self-class known feature spectra, a self-class spectral similarity which is the spectral similarity between the feature spectrum and a self-class known feature spectrum of the group of known feature spectra that is the same class as the evaluation class indicated by the a priori label associated with the evaluation data; calculating, for each of the plurality of other-class known feature spectra, an other-class spectral similarity, which is the spectral similarity between the feature spectrum and an other-class known feature spectrum of a class different from the evaluation class among the group of known feature spectra, The step (a3) calculating a self-class maximum data similarity, which is the similarity between the input data associated with the self-class known feature spectrum, which is a calculation source of the self-class maximum spectrum similarity indicating the maximum value among the plurality of self-class spectrum similarities, and the evaluation data; calculating an other-class maximum data similarity, which is the similarity between the input data associated with the other-class known feature spectrum, which is a calculation source of an other-class maximum spectrum similarity indicating the maximum value among the plurality of other-class spectrum similarities, and the evaluation data; The step (a4) generating the primary explanation information including own-class spectral similarity information related to the own-class maximum spectral similarity, own-class data similarity information related to the own-class maximum data similarity, other-class spectral similarity information related to the other-class maximum spectral similarity, and other-class data similarity information related to the other-class maximum data similarity.

3. The evaluation method according to claim 2, In the step (a), when the learning data is used as the evaluation data, the step (b) (b1) An evaluation method including a step of generating the secondary explanation information using a first learning comparison result of a value indicated by the other-class spectral similarity information and a predetermined first other-class spectral similarity threshold, and a second learning comparison result of a value indicated by the other-class data similarity information and a predetermined first other-class data similarity threshold.

4. The evaluation method according to claim 3, The step (b1) generating, as the secondary explanatory information, information indicating at least one of the following: that the training data contains inappropriate defective data; and that the information in the training data is insufficient to be information necessary for class discrimination, when the value indicated by the other-class spectral similarity information is equal to or greater than the first other-class spectral similarity threshold and the value indicated by the other-class data similarity information is equal to or greater than the first other-class data similarity threshold.

5. The evaluation method according to claim 3 or 4, The step (b1) generating, as the secondary explanation information, information indicating that the machine learning model may lack the ability to correctly classify the evaluation data when the value indicated by the other-class spectral similarity information is equal to or greater than the first other-class spectral similarity threshold and the value indicated by the other-class data similarity information is less than the first other-class data similarity threshold.

6. The evaluation method according to any one of claims 3 to 5, In the step (a), when a plurality of pieces of learning data are used as the evaluation data, the step (b) (b2) When the value indicated by the own-class spectral similarity information is less than a predetermined first own-class spectral similarity threshold, the evaluation method includes a step of generating, as the secondary explanation information, at least one of first learning assessment information indicating that the input data contained in the plurality of learning data used as the evaluation data has a large variance, and second learning assessment information indicating that the input data contained in the plurality of learning data used as the evaluation data has outlier data.

7. The evaluation method according to claim 6, The step (b2) generating the first learning evaluation information as the secondary explanation information when the number of the learning data as the evaluation data that satisfies the condition that the value indicated by the own-class spectral similarity information is less than the first own-class spectral similarity threshold is equal to or greater than a predetermined first data threshold; an evaluation method in which, when the number of the learning data as the evaluation data that satisfies the condition that the value indicated by the own-class spectral similarity information is less than the first own-class spectral similarity threshold is less than the first data threshold, the second learning evaluation information is generated as the secondary explanation information.

8. The evaluation method according to any one of claims 2 to 7, In the step (a), when the verification data is used as the evaluation data, the step (b) (b3) An evaluation method including a step of generating the secondary explanation information using a first verification comparison result of the value indicated by the other-class spectral similarity information and a predetermined second other-class spectral similarity threshold, and a second verification comparison result of the value indicated by the other-class data similarity information and a predetermined second other-class data similarity threshold.

9. The evaluation method according to claim 8, The step (b3) generating, as the secondary explanation information, information indicating at least one of the following: that the verification data contains inappropriate defective data; and that information in the verification data is insufficient as information necessary for class discrimination, when the value indicated by the other-class spectral similarity information is equal to or greater than the second other-class spectral similarity threshold and the value indicated by the other-class data similarity information is equal to or greater than the second other-class data similarity threshold.

10. The evaluation method according to claim 8 or claim 9, The step (b3) generating, as the secondary explanation information, information indicating that the machine learning model may lack the ability to correctly classify the evaluation data, when the value indicated by the other-class spectral similarity information is equal to or greater than the second other-class spectral similarity threshold and the value indicated by the other-class data similarity information is less than the second other-class data similarity threshold.

11. The evaluation method according to any one of claims 8 to 10, In the step (a), when a plurality of pieces of verification data are used as the evaluation data, the step (b) (b4) When the value indicated by the own class spectrum similarity information is less than a predetermined second own class spectrum similarity threshold and the value indicated by the own class data similarity information is less than a predetermined own class data similarity threshold, the evaluation method includes a step of generating, as the secondary explanation information, at least one of: first verification evaluation information indicating a large deviation in features between the input data included in the plurality of verification data and the input data included in the training data; and second verification evaluation information indicating that outlier data exists among the plurality of input data included in the plurality of verification data.

12. The evaluation method according to claim 11, The step (b4) generating the first verification evaluation information as the secondary explanation information when the number of the verification data as the evaluation data that satisfies the conditions that the value indicated by the own class spectrum similarity information is less than the second own class spectrum similarity threshold and the value indicated by the own class data similarity information is less than the own class data similarity threshold is equal to or greater than a predetermined second data threshold; the evaluation method, wherein when the number of the verification data as the evaluation data that satisfies the conditions that the value indicated by the own class spectral similarity information is less than the second own class spectral similarity threshold and the value indicated by the own class data similarity information is less than the own class data similarity threshold is less than the second data threshold, the evaluation method generates the second verification evaluation information as the secondary explanation information.

13. The evaluation method according to claim 11 or 12, The step (b3) generating, as the secondary explanatory information, information indicating that overfitting of the machine learning model has occurred, when the value indicated by the own-class spectral similarity information is less than the second own-class spectral similarity threshold and the value indicated by the own-class data similarity information is equal to or greater than the own-class data similarity threshold.

14. The evaluation method according to any one of claims 2 to 13, In the step (a4), the own-class spectral similarity information includes at least one of information on a representative value of a distribution of the plurality of own-class spectral similarities and information on the own-class maximum spectral similarity; The evaluation method, wherein the other-class spectral similarity information includes at least one of information on a representative value of a distribution of the plurality of other-class spectral similarities and information on the maximum other-class spectral similarity.

15. The evaluation method according to any one of claims 1 to 14, In the step (a), when abnormal data that is not associated with the prior label and is assumed to be classified into an unknown class different from the class corresponding to the prior label is used as the evaluation data, The step (a2) specifying a maximum spectral similarity that is the maximum value among the spectral similarities calculated for each of the plurality of known characteristic spectra, The step (a3) calculating a maximum data similarity between the input data associated with the known characteristic spectrum, which is a calculation source of the maximum spectrum similarity specified in step (a2), and the abnormal data; The step (a4) generating the primary explanation information including spectral similarity information relating to the spectral similarity and the maximum data similarity.

16. The evaluation method according to claim 15, In the step (a), when the abnormal data is used as the evaluation data, the step (b) generating, as the secondary explanatory information, information indicating that the machine learning model may lack the ability to correctly classify the abnormal data, when the maximum spectral similarity is equal to or greater than a predetermined abnormal spectrum threshold and the maximum data similarity is less than a predetermined abnormal data similarity threshold.

17. The evaluation method according to claim 15 or 16, In the step (a), when the abnormal data is used as the evaluation data, the step (b) generating, as the secondary explanatory information, information indicating that information on the abnormal data is insufficient as information required for class discrimination, when the maximum spectrum similarity is equal to or greater than a predetermined abnormal spectrum threshold and the maximum data similarity is equal to or greater than a predetermined abnormal data similarity threshold.

18. An evaluation device for a trained machine learning model, a memory for storing the machine learning model, the machine learning model being a vector neural network type model having a plurality of vector neuron layers, and trained using a plurality of training data having input data and prior labels associated with the input data; a processor; The processor: (a) inputting evaluation data into the trained machine learning model to generate primary explanatory information to be used in evaluating the machine learning model; (b) generating secondary explanation information indicating an evaluation of the trained machine learning model using values ​​indicated by each piece of information contained in the primary explanation information; (c) a process of outputting the generated secondary explanatory information; The treatment (a) (a1) inputting the evaluation data into the trained machine learning model and obtaining a feature spectrum from an output of a specific layer of the trained machine learning model; (a2) calculating, for each of the plurality of known feature spectra included in the group of known feature spectra, a spectral similarity between the feature spectrum and a known feature spectrum included in the group of known feature spectra obtained from the output of the specific layer by again inputting the plurality of training data into the trained machine learning model; (a3) a process of calculating a data similarity between the input data and the evaluation data; (a4) generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity.

19. A computer program that causes a computer to evaluate a trained machine learning model, the machine learning model is a vector neural network type model having a plurality of vector neuron layers, and is trained using a plurality of training data having input data and prior labels associated with the input data; The computer program comprises: (a) a function of inputting evaluation data into the trained machine learning model and generating primary explanatory information to be used in evaluating the machine learning model; (b) a function of generating secondary explanation information indicating an evaluation of the trained machine learning model using values ​​indicated by each piece of information contained in the primary explanation information; (c) a function of outputting the generated secondary explanatory information; The function (a) is (a1) a function of inputting the evaluation data into the trained machine learning model and obtaining a feature spectrum from the output of a specific layer of the trained machine learning model; (a2) a function of calculating, for each of the plurality of known feature spectra in the group of known feature spectra, a spectral similarity between the feature spectrum and a known feature spectrum included in the group of known feature spectra obtained from the output of the specific layer by inputting the plurality of training data again into the trained machine learning model; and (a3) a function of calculating a data similarity between the input data and the evaluation data; (a4) A computer program having a function of generating the primary explanation information including spectral similarity information regarding the spectral similarity and data similarity information regarding the data similarity.

Images