Evaluation method, evaluation device, and computer program

Abstract

To provide an evaluation method that may require preparation of an individualized learning set using a machine learning model.SOLUTION: An evaluation method for evaluating target data comprises steps of: inputting a plurality of learning sets of generic learning data of a type different from the target data and a label corresponding to the generic learning data into a vector neural network type machine learning model having a plurality of vector neuron layers, and learning the machine learning model; acquiring a reference feature spectrum; acquiring a target feature spectrum; calculating a spectral similarity between the reference feature spectrum and the target feature spectrum; and evaluating the target data using the spectral similarity.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to techniques for evaluating subject data. [Background technology]

[0002] Conventionally, a vector neural network type machine learning model having multiple vector neuron layers has been known (Patent Document 1). In the trained machine learning model, a spectral similarity is calculated by comparing a known feature spectrum serving as an evaluation standard with a target feature spectrum of target data. The known feature spectrum and the target feature spectrum are obtained from the output of the vector neuron layer of the machine learning model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-56611 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional technology, the data of the learning set used for training the machine learning model is the same as the data on which the known characteristic spectrum is based for calculating the spectral similarity. Therefore, in the conventional technology, in order to train the machine learning model, it may be necessary to prepare the data on which the known characteristic spectrum is based, that is, a learning set according to an individual purpose for using the machine learning model. [Means for solving the problem]

[0005] According to a first embodiment of the present disclosure, there is provided an evaluation method for evaluating target data, which includes: (a) inputting a plurality of learning sets of generic learning data of a type different from the target data and labels corresponding to the generic learning data into a vector neural network type machine learning model having a plurality of vector neuron layers to train the machine learning model; (b) after the step (a), inputting reference data of the same type as the target data, the evaluation of which indicates a predetermined reference evaluation, into the trained machine learning model to obtain a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model; (c) after the step (a), inputting the target data to be evaluated into the trained machine learning model to obtain a target feature spectrum as the feature spectrum from an output of the specific layer; (d) calculating a spectral similarity that is a similarity between the reference feature spectrum and the target feature spectrum; and (e) evaluating the target data using the spectral similarity.

[0006] According to a second embodiment of the present disclosure, there is provided an evaluation device for evaluating target data. The evaluation device includes a learning execution unit for inputting a plurality of learning sets of generic learning data of a type different from the target data and labels corresponding to the generic learning data to a vector neural network type machine learning model having a plurality of vector neuron layers to learn the machine learning model, a first acquisition unit for inputting reference data of the same type as the target data and the evaluation indicating a reference evaluation to the trained machine learning model to acquire a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model, a second acquisition unit for inputting the target data to be evaluated to the trained machine learning model to acquire a target feature spectrum as the feature spectrum from an output of the specific layer, a calculation unit for calculating a spectral similarity which is a similarity between the reference feature spectrum and the target feature spectrum, and an evaluation unit for evaluating the target data using the spectral similarity.

[0007] According to a third aspect of the present disclosure, there is provided a computer program to be executed by a computer that evaluates target data. The computer program includes: (a) a function of inputting a plurality of learning sets of generic learning data of a type different from the target data and a label corresponding to the generic learning data into a vector neural network type machine learning model having a plurality of vector neuron layers to train the machine learning model; (b) a function of inputting reference data of the same type as the target data and the evaluation indicating a reference evaluation into the trained machine learning model after executing the function (a) to obtain a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model; (c) a function of inputting the target data to be evaluated into the trained machine learning model after executing the function (a) to obtain a target feature spectrum as the feature spectrum from an output of the specific layer; (d) a function of calculating a spectral similarity that is a similarity between the reference feature spectrum and the target feature spectrum; and (e) a function of evaluating the target data using the spectral similarity. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an evaluation system according to an embodiment. [Diagram 2] FIG. 1 is an explanatory diagram showing the configuration of a machine learning model. [Diagram 3] 1 is a flowchart showing the learning process of a machine learning model. [Figure 4] FIG. 13 is a diagram for explaining a learning set group. [Diagram 5] FIG. 4 is a diagram for explaining reference data. [Figure 6] FIG. [Figure 7] FIG. 4 is an explanatory diagram showing the configuration of a reference feature spectrum group. [Figure 8] 4 is a flowchart of an evaluation process executed by the evaluation device. [Figure 9] 11 is a flowchart showing details of step S130. [Figure 10]FIG. 4 is an explanatory diagram showing a first calculation method of the spectral similarity. [Figure 11] Conceptual diagram of calculation formula (c1). [Figure 12] FIG. 11 is an explanatory diagram showing a second calculation method of the spectral similarity. [Figure 13] FIG. 11 is an explanatory diagram showing a third calculation method. [Figure 14] FIG. 4 is a diagram showing an evaluation result of the evaluation device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A. Embodiment: FIG. 1 is a diagram showing an evaluation system 5 in an embodiment. This evaluation system 5 is a device for evaluating various target data IDE, for example, evaluating whether products such as equipment and parts are normal or evaluating the degree of fatigue of a target part of a human. In this embodiment, the target data IDE is a moving image captured of a robot in operation. The evaluation system 5 evaluates whether the operation of the target robot is normal by evaluating the moving image.

[0010] The evaluation system 5 includes an evaluation device 100, a sensor device 400, and a learning set group LSG. The sensor device 400 is a device for acquiring target data IDE to be evaluated and original evaluation data that is the source of the target data IDE. The sensor device 400 is, for example, an imaging device or an ultrasonic device that transmits ultrasonic waves and receives reflected waves. In this embodiment, the sensor device 400 is a camera that can capture moving images and still images. The sensor device 400 can communicate data with the evaluation device 100 by wire or wirelessly. The learning set group LSG is used for learning the machine learning model 200 described later. The learning set group LSG may be stored in, for example, an external storage device different from the evaluation device 100, or may be stored in the storage device 120 of the evaluation device 100. The external storage device can communicate data with the evaluation device 100 by wire or wirelessly. The learning set group LSG will be described in detail later.

[0011] The evaluation device 100 includes a processor 110, a storage device 120, an interface circuit 130, and an input device 140 and a display unit 150 connected to the interface circuit 130. The evaluation device 100 is, for example, a personal computer. The evaluation device 100 uses a trained machine learning model 200 stored in the storage device 120 to evaluate the target data IDE.

[0012] The processor 110 executes various programs stored in the storage device 120 to thereby implement a learning execution unit 112 , a data processing unit 113 , a spectrum acquisition unit 114 , and an evaluation processing unit 118 .

[0013] The learning execution unit 112 inputs a plurality of learning sets LS constituting the learning set group LSG to the machine learning model 200, and executes learning processing of the machine learning model 200. Details of the machine learning model 200 will be described later.

[0014] The data processing unit 113 performs data processing such as image processing on data captured and acquired by the sensor device 400 or the like. The data processing unit 113 can perform edge extraction processing, binarization processing, object extraction processing, and the like on each frame image of a moving image acquired by the sensor device 400, for example.

[0015] The spectrum acquisition unit 114 inputs an input data ID to the trained machine learning model 200, thereby acquiring a feature spectrum Sp from the output of a specific layer of the trained machine learning model 200. The spectrum acquisition unit 114 has a first acquisition unit 115 and a second acquisition unit 116. Details of the feature spectrum Sp will be described later.

[0016] The first acquisition unit 115 inputs the reference data IDS, which is an example of the input data ID, to the trained machine learning model 200, and acquires a reference feature spectrum KSp as a feature spectrum Sp from the output of a specific layer of the trained machine learning model 200. The first acquisition unit 115 stores the acquired reference feature spectrum KSp in the storage device 120. In this embodiment, a plurality of reference feature spectra KSp are stored in the storage device 120 as a reference feature spectrum group KSpG. The reference data IDS is of the same type as the target data IDE, which is another example of the input data ID. That is, the reference data IDS and the target data IDE are of the same type of target from which data is generated. In this embodiment, since the target data IDE is a moving image obtained by capturing the operation of a robot, the reference data IDS is also a moving image obtained by capturing the operation of the same type of robot. That is, in this embodiment, the reference data IDS and the target data IDE are moving images composed of a plurality of frame images arranged in a time series. The reference data IDS from which the reference feature spectrum KSp is generated is data indicating a predetermined reference evaluation. The predetermined reference evaluation may be any index that serves as a reference for evaluating the target data IDE, and in this embodiment, it is represented by a label LB indicating "normal," which is an example of the reference class. In other words, the reference evaluation in this embodiment indicates that the robot's operation is normal, and is an evaluation classified into the reference class.

[0017] The second acquisition unit 116 inputs the target data IDE to be evaluated to the trained machine learning model 200, and acquires the target feature spectrum ESp as the feature spectrum Sp from the output of the specific layer. The second acquisition unit 116 stores the acquired target feature spectrum ESp in the storage device 120.

[0018] The evaluation processing unit 118 evaluates the target data IDE. The evaluation processing unit 118 displays the evaluation result on the display unit 150. The evaluation processing unit 118 includes a calculation unit 117 and an evaluation unit 119.

[0019] The calculation unit 117 calculates a spectral similarity RSp, which is the similarity between the reference feature spectrum KSp and the target feature spectrum ESp. The method of calculating the spectral similarity RSp will be described later.

[0020] The evaluation unit 119 evaluates the target data IDE using the calculated spectral similarity RSp. For example, the evaluation unit 119 evaluates the target data IDE by classifying into two or more classes. In this embodiment, when the spectral similarity RSp is equal to or greater than a predetermined threshold, the evaluation unit 119 classifies the target data IDE into a reference class, that is, "normal." On the other hand, when the spectral similarity RSp is less than the threshold, the evaluation unit 119 classifies the target data IDE into a class different from the reference class. The different class is, for example, a class labeled with an "abnormal" label.

[0021] FIG. 2 is an explanatory diagram showing the configuration of the machine learning model 200. The machine learning model 200 includes, in order from the input data ID side, a convolution layer 220, a primary vector neuron layer 230 which is one of the intermediate layers, a first convolution vector neuron layer 240 which is one of the intermediate layers, a second convolution vector neuron layer 250 which is one of the intermediate layers, and a classification vector neuron layer 260 which is an output layer. Of these layers 220 to 260, the convolution layer 220 is the lowest layer, and the classification vector neuron layer 260 is the highest layer. As described above, the machine learning model 200 is a vector neural network type machine learning model having a plurality of vector neuron layers 230, 240, 250, and 260. In the following description, the layers 220, 230, 240, 250, and 260 are also referred to as the "Conv layer 220," the "PrimeVN layer 230," the "ConvVN1 layer 240," the "ConvVN2 layer 250," and the "ClassVN layer 260," respectively.

[0022] 2, two convolution vector neuron layers 240 and 250 are used, but the number of convolution vector neuron layers is arbitrary, and the convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0023] The configuration of each of the layers 220 to 260 can be described as follows. ·Conv layer 220: Conv[32,5,2] ·PrimeVN layer 230: PrimeVN[16,1,1] ·ConvVN1 layer 240:ConvVN1[12,3,2] ·ConvVN2 layer 250:ConvVN2[6,3,1] ·ClassVN layer 260:ClassVN[M,4,1] Vector dimension VD: VD=16 In the description of each layer, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the surface size of the kernel, and the stride. For example, the layer name of the Conv layer 220 is "Conv", the number of channels is 32, the surface size of the kernel is 5x5, and the stride is 2. In FIG. 2, these descriptions are shown under each layer. The hatched rectangles drawn in each layer represent the surface size of the kernel used when calculating the output vector of the adjacent upper layer. In this embodiment, the input data ID is, for example, a set of frame images in a unit period tm that constitutes a moving image, so the surface size of the kernel is also two-dimensional. Note that the parameter values ​​used in the description of each layer are examples and can be changed arbitrarily. If the input data ID is composed of 30 frame images of 29x29 pixels, the number of channels of the input data ID is 29x29x30. Furthermore, regardless of the number of channels of the input data ID, a constant size output is produced by the convolution process by the Conv layer 220.

[0024] The Conv layer 220 is a layer composed of scalar neurons. The other layers 230 to 260 are layers composed of vector neurons. A vector neuron is a neuron that takes a vector as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0025] In FIG. 2, for the Conv layer 220, a first axis x and a second axis y that define the planar coordinates of the node array, and a third axis z that represents the depth are shown. Also, it is shown that the sizes of the Conv layer 220 in the x, y, and z directions are 13, 13, and 32. The sizes in the x and y directions are called "resolution." The size in the z direction is the number of channels. These three axes x, y, and z are also used as coordinate axes indicating the position of each node in other layers. However, in FIG. 2, the illustration of these axes x, y, and z is omitted in layers other than the Conv layer 220.

[0026] As is well known, the resolution W1 after convolution is given by the following equation: W1 = Ceil{(W0 - Wk + 1) / S} (A1) Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function that rounds up X to the nearest integer. The resolution of each layer shown in FIG. 2 is an example in which the resolution of the input data ID is 29, and the actual resolution of each layer is changed appropriately depending on the size of the input data ID.

[0027] The ClassVN layer 260 has M channels. M is the number of classes determined by the machine learning model 200. In this embodiment, M is 10, and two class judgment values ​​Class_1 to Class_10 are output. The number of channels M of the ClassVN layer 260 can be set to any integer equal to or greater than 2.

[0028] FIG. 2 further illustrates a partial region Rn in each layer 220, 230, 240, 250, and 260. The subscript "n" of the partial region Rn is the code of each layer. For example, the partial region R220 indicates a partial region in the Conv layer 220. The "partial region Rn" is a region in each layer that is identified by a planar position (x, y) defined by the position of the first axis x and the position of the second axis y, and includes multiple channels along the third axis z. The partial region Rn has dimensions of "Width" x "Height" x "Depth" corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one "partial region Rn" is "1 x 1 x number of depths", that is, "1 x 1 x number of channels".

[0029] 2, for example, a feature spectrum Sp is obtained from the output of the ConvVN2 layer 250. In the present disclosure, the vector neuron layer used to calculate the spectral similarity RSp is also referred to as a "specific layer." The specific layer may be an intermediate layer other than the ConvVN2 layer 250, may be the ClassVN layer 260 which is an output layer, or may be two or more layers among the vector neuron layers 230, 240, 250, and 260.

[0030] Fig. 3 is a flowchart showing a learning process of the machine learning model 200. In the learning process, a plurality of learning sets LS are prepared in step S10. Fig. 4 is a diagram for explaining a learning set group LSG. The learning set group LSG is composed of a plurality of learning sets LS. Each learning set LS has general-purpose learning data LD and a label LB corresponding to the general-purpose learning data LD.

[0031] The general-purpose learning data LD is a type of data different from the target data IDE to be evaluated. The general-purpose learning data LD is general data that is widely used as learning data for the machine learning model 200, and in this embodiment, it is generated based on MNIST data, which is an image representing handwritten digits. Specifically, the general-purpose learning data LD is a moving image in which an image, which is MNIST data, is rotated in a predetermined rotation direction R for a predetermined time tp. The general-purpose learning data LD is a collection of M frame images FML at regular time intervals tv. "M" is an integer equal to or greater than 2. In other words, the general-purpose learning data LD is a moving image composed of a plurality of frame images arranged in a time series. In FIG. 4, for each general-purpose learning data LD, only one frame image FML is shown for convenience, but in reality, it is composed of M frame images FML. In this embodiment, the images that are MNIST data are handwritten digits "0" to "9", and a plurality of data with different outer shapes of the digits are prepared for each digit. The image rotation speed of the general-purpose learning data LD may be different or the same for each general-purpose learning data LD. Furthermore, the rotation direction R of the general-purpose learning data LD may be different or the same for each piece of general-purpose learning data LD.

[0032] The label LB indicates the number represented by the general-purpose learning data LD, and a different label is assigned to each number. In this embodiment, the labels "0" to "9" are associated with each of the general-purpose learning data LD representing the numbers "0" to "9."

[0033] As shown in FIG. 3, in step S20, the learning execution unit 112 inputs each learning set LS of the learning set group LSG to the machine learning model 200 to execute learning of the machine learning model 200. Specifically, the learning execution unit 112 executes learning of the machine learning model 200 so as to reproduce the correspondence between each piece of generic learning data LD and a label LB associated with each piece of generic learning data LD. Note that the generic learning data LD of the learning set group LSG is data-converted into a Numpy format that is a format suitable for the machine learning model 200. Note that other input data ID input to the machine learning model 200 is also data-converted into the Numpy format, similar to the generic learning data LD.

[0034] Next, in step S30, the first acquisition unit 115 inputs the reference data IDS to the trained machine learning model 200, and acquires a reference feature spectrum KSp from the output of a specific layer of the machine learning model 200. The acquired reference feature spectrum KSp is stored in the storage device 120. In this embodiment, the specific layer is the ConvVN2 layer 250.

[0035] FIG. 5 is a diagram for explaining the reference data IDS. The reference data IDS in this embodiment is a moving image generated based on an original reference moving image RD obtained by capturing the movement of a robot 900 as a reference object operating normally. The robot 900 has a base 901 and an arm 902 connected to the base 901. In the robot 900 operating normally, the arm 902 moves back and forth between a start point and an end point at a constant speed. The original reference moving image RD is a set of reference frame images FMK1 to FMKN captured during a predetermined reference time ts, during which the robot 900 operates normally. The predetermined reference time ts may be the same as or different from the time tp of the video of the general-purpose learning data LD, which is a moving image. In addition, the number of the reference frame images FMK1 to FMKN may be the same as or different from the number of the frame images FML of the general-purpose learning data LD. In this embodiment, the original reference moving image RD is composed of N reference frame images FMK1 to FMKN. "N" is an integer of 2 or more. When the N base frame images FMK1 to FMKN are used without distinction, the base frame image FMK is used.

[0036] The data processing unit 113 generates multiple processed reference frame images FMS by executing image processing to extract the moving reference object, the robot 900, more specifically the arm 902, from the reference frame image FMK using the multiple reference frame images FMK1-FMKN that constitute the original reference moving image RD. That is, the data processing unit 113 generates multiple, in this embodiment N, processed reference frame images FMS arranged in time series as reference data IDS. The data processing unit 113 calculates the average pixel value for each pixel of the multiple reference frame images FMS1-FMSN. Then, the data processing unit 113 calculates the absolute value of the difference between each pixel of the reference frame image FMK and the average value of the corresponding pixel for each of the multiple reference frame images FMK1-FMKN, and generates a set of the absolute values ​​of the differences at each pixel as the processed reference frame image FMS. As a result, the stationary base 901, objects 923, 924, and background are removed from the processed reference frame image FMS, and the moving reference object, the robot 900, more specifically the arm 902, is extracted. Note that the data processing for extracting the moving reference object from the reference frame image FMK is not limited to the above. For example, the data processing unit 113 may execute data processing such as pattern recognition processing and edge extraction processing on the reference frame image FMK to generate a processed reference frame image FMS from which the reference object has been extracted.

[0037] Furthermore, a plurality of reference data IDS may be prepared. In this embodiment, a plurality of reference data IDS is prepared. The plurality of reference data IDS may be generated based on each of divided data obtained by dividing a moving image of the robot 900 operating normally into a plurality of periods, or may be generated based on the moving image obtained by individually acquiring a moving image of the robot 900 operating normally. When a plurality of reference data IDS are generated based on divided data, the generation period of one reference data IDS and the generation period of another reference data may or may not overlap partially.

[0038] Fig. 6 is an explanatory diagram showing a feature spectrum Sp obtained by inputting any input data ID to the trained machine learning model 200. Fig. 6 is an explanatory diagram showing a reference feature spectrum KSp corresponding to one reference data IDS as a specific example.

[0039] The horizontal axis in FIG. 6 is the position of vector elements related to the output vectors of multiple nodes included in one partial region R250 of the ConvVN2 layer 250. The position of this vector element is expressed by a combination of the element number ND of the output vector in each node and the channel number NC. In this embodiment, the vector dimension is 16, that is, the number of elements of the output vector output by each node, so the element number ND of the output vector is 16, from 0 to 15. In addition, the number of channels in the ConvVN2 layer 250 is 6, so the channel number NC is 6, from 0 to 5. Therefore, the number of elements of the feature spectrum Sp is 16×6=96. The feature spectrum Sp composed of this 96 number of elements represents the feature spectrum Sp of the reference data IDS composed of N reference frame images FMS1 to FMSN. The feature spectrum Sp is obtained by arranging multiple element values ​​of the output vector of each vector neuron included in one partial region R250 across multiple channels along the third axis z.

[0040] The vertical axis of FIG. 6 indicates the feature value CV at each spectral position. In this example, the feature value CV is the value V of each element of the output vector. ND The feature value CV may be subjected to statistical processing such as centering to the average value 0. Note that the feature value CV is the value V of each element of the output vector. ND Alternatively, the normalization coefficient may be used as is. In the latter case, the number of feature values ​​CV included in the feature spectrum Sp is equal to the number of channels, that is, six. The normalization coefficient is a value corresponding to the vector length of the output vector of the node.

[0041] The number of feature spectra Sp obtained from the output of the ConvVN2 layer 250 for one input data ID is equal to the number of planar positions (x, y) of the ConvVN2 layer 250, that is, the number of subregions R250, and is therefore nine.

[0042] 7 is an explanatory diagram showing the configuration of the reference feature spectrum group KSpG. In this example, the reference feature spectrum group KSpG is shown as a collection of reference feature spectra KSp as known feature spectra acquired from the output of the ConvVN2 layer 250. Note that as the reference feature spectrum group KSpG, it is sufficient that those obtained from the output of at least one vector neuron layer are registered, and reference feature spectrum groups obtained from the output of the ConvVN1 layer 240 or the ClassVN layer 260 may be registered.

[0043] Each record of the reference feature spectrum group KSpG includes a parameter k indicating the order of the partial region Rn in the layer, a parameter q indicating a data number, a parameter tm indicating a unit period divided at regular intervals in the video, and a reference feature spectrum KSp. A collection of time-series frame images included in the unit period tm constitutes the reference data IDS. For example, multiple reference data IDS are generated by dividing a video having the same data number q into unit periods tm. The reference feature spectrum KSp is the same as the feature spectrum Sp in FIG. 6.

[0044] The parameter k of the partial region Rn takes a value indicating which of the multiple partial regions Rn included in a specific layer it is, i.e., which planar position (x, y) it is. For the ConvVN2 layer 250, the number of partial regions R250 is 16, so k = 1 to 16. The data number parameter q is a number for identifying the video image that is the source of the reference data IDS.

[0045] FIG. 8 is a flowchart of the evaluation process executed by the evaluation device 100. First, in step S100, the target data IDE is prepared. In this embodiment, a plurality of processed target frame images are generated by extracting the robot 900, which is the evaluation target, specifically the arm 902 of the robot 900, from a plurality of target frame images constituting the original target video. The plurality of evaluation frame images constitute an original target video arranged in time series acquired by the sensor device 400. This original target video is data obtained by capturing the movement of the robot 900 for a predetermined reference time ts. The predetermined reference time ts may be the same as or different from the reference time ts, which is the capture time of the original reference video RD that is the source data of the reference data IDS. The data processing unit 113 executes image processing to extract the arm 902 of the moving robot 900, which is the evaluation target, from the original target video in the same manner as the image processing to generate the reference data IDS from the original reference video RD. The data processing unit 113 generates multiple pieces of target data IDE consisting of multiple processed target frame images by extracting data for multiple unit periods tm from the video after image processing. As described above, the data processing unit 113 generates multiple processed target frame images from which the robot 900 has been extracted, using multiple target frame images that make up the original target video acquired by capturing the movements of the robot 900. In this way, the data processing unit 113 generates multiple processed target frame images arranged in chronological order as target data IDE.

[0046] Next, in step S110, the second acquisition unit 116 inputs the target data IDE to be evaluated to the trained machine learning model 200, and acquires the target feature spectrum ESp from the output of the ConvVN2 layer 250, which is a specific layer. The data configuration of the target feature spectrum ESp is the same as that of the reference feature spectrum KSp shown in Figs. 6 and 7.

[0047] Next, in step S120, calculation unit 117 calculates a spectral similarity RSp between reference feature spectrum KSp and target feature spectrum ESp. The target feature spectrum ESp has the same configuration as the feature spectrum Sp shown in Fig. 6. A method for calculating the spectral similarity RSp will be described in detail later.

[0048] Next, in step S130, the evaluation unit 119 performs an evaluation process on the target data IDE using the spectral similarity RSp.

[0049] FIG. 9 is a flowchart showing the details of step S130. In step S132, the evaluation unit 119 judges whether the spectral similarity RSp is equal to or greater than a threshold. If the spectral similarity RSp is equal to or greater than the threshold, the evaluation unit 119 classifies the target data IDE into a reference class in step S134. In this embodiment, as described above, the reference class is a label of "normal" indicating that the movement of the robot 900 is normal. On the other hand, if the spectral similarity RSp is less than the threshold, the evaluation unit 119 classifies the target data IDE into a class different from the reference class in step S136. In this embodiment, the different class is a label of "abnormal" indicating that the movement of the robot 900 is abnormal. In this embodiment, the evaluation process of step S130 is sequentially executed for each set of individual target feature spectra IESp corresponding to the multiple processed target frame images arranged in time series that constitute the target data IDE, that is, for each unit period tm. The evaluation unit 119 displays evaluation information indicating the result of the evaluation process on the display unit 150. For example, when a label determination of "abnormal" is made, the evaluation information includes information indicating "abnormal" and the time t when the target frame image that is the basis of the processed target frame image that has been classified as "abnormal" was acquired, i.e., time information when the abnormality occurred, and is displayed on the display unit 150. The evaluation information may also be color information according to the magnitude of the spectral similarity RSp. For example, the evaluation unit 119 outputs a color approaching red as the spectral similarity RSp increases to the display unit. In this way, the evaluation unit 119 can easily classify the target data IDE for each unit period tm by determining whether the spectral similarity RSp is equal to or greater than a threshold value.

[0050] Next, an example of a calculation method of the spectral similarity RSp will be described. FIG. 10 is an explanatory diagram showing a first calculation method M1 of the spectral similarity RSp. In the first calculation method M1, first, a local spectral similarity S(j, k, tm) is calculated for each partial region Rn from the output of the ConvVN2 layer 250, which is a specific layer. In this embodiment, the calculation source of the spectral similarity RSp is a video image composed of a plurality of frame images. Therefore, the calculation unit 117 calculates the spectral similarity RSp between the feature spectra Sp generated based on the input data ID, which is a set of frame images arranged in a chronological order with consecutive frame image numbers NFM, for each certain number of frame images. The certain number is a number according to the time interval Δt for performing the evaluation process of step S130 shown in FIG. 9. When the frame rate of the video image is 60 fps and the time interval Δt is 0.5 seconds, the certain number is 30. That is, each of the target data IDE and the reference data IDS is a set of 30 frame images with consecutive frame image numbers NFM. The target data IDE and the reference data IDS are input as input data ID to the machine learning model 200, and the spectral similarity RSp between the generated feature spectra Sp is calculated. It is preferable that the number of consecutive frame image numbers NFM is the same for the target data IDE and the reference data IDS from which the spectral similarity RSp is calculated. Furthermore, for the target data IDE and the reference data IDS from which the spectral similarity RSp is calculated, the frame image numbers NFM of the frame images constituting the target data IDE and the frame image numbers NFM of the frame images constituting the reference data IDS may be the same or different. An example of a method for calculating the spectral similarity RSp will be described below.

[0051] In the first calculation method M1, the local spectral similarity S(j,k,tm) is calculated using the following formula: S(j,k,tm)=max[G{ESp(j,k,tm), KSp(j,k=all,q=all,tm=all)}] (c1) j is a parameter indicating a particular layer. k is a parameter indicating the subregion Rn. q is a parameter indicating a data number. tm is a parameter indicating unit periods in a moving image, which are divided at regular intervals. The duration of each unit period is the same. FIG. 10 shows the case where the unit period tm is "1". G{a,b} is a function that calculates the spectral similarity between a and b. ESp(j, k, tm) is the target feature spectrum ESp obtained from the output of a specific subregion Rn of a specific layer j, which corresponds to a set of multiple frame images acquired in a unit period indicated by the parameter tm of the target data IDE, which is a moving image. max[X] is a logical operation that takes the maximum value of X. KSp(j, k, q=all, tm=all) is the reference feature spectrum KSp corresponding to a set of multiple frame images acquired for each unit period, among all reference feature spectra KSp of data number q obtained from the output of a specific subregion Rn of a specific layer j. As the function G{a,b} for calculating the local spectral similarity, for example, an equation for calculating cosine similarity or an equation for calculating similarity according to distance can be used.

[0052] 11 is a conceptual diagram of the above calculation formula (c1). In the calculation method of the above formula (c1), the feature spectra Sp corresponding to each of all unit periods tm0 to tm(N-1) of the reference data IDS are used as comparison targets for the spectral similarity RSp of the feature spectrum Sp corresponding to the unit period tm of the target data IDE, for example, the unit period tm1. Also, the reference data IDS of all data numbers q are used as comparison targets for the spectral similarity RSp of the feature spectrum Sp corresponding to the unit period tm of the target data IDE, for example, the unit period tm1.

[0053] 12 is an explanatory diagram showing a second calculation method M2 of the spectral similarity RSp. In the second calculation method M2, the local spectral similarity S(j,k nf,ff) is calculated using the following equation. S(j,k,tm)=max[G{ESp(j,k,tm), KSp(j,k,q=all,tm=all)}] (c2)

[0054] The above-mentioned first calculation method M1 uses the reference feature spectrum KSp (j, k=all, q=all, tm=all) in all partial regions k of a specific layer j, whereas the second calculation method M2 uses only the reference feature spectrum KSp for the partial region k that is the same as the partial region k of the individual object feature spectrum IESp. The other methods in the second calculation method M2 are the same as those in the first calculation method M1.

[0055] 13 is an explanatory diagram showing a third calculation method M3. In the third calculation method M3, the spectral similarity RSp is calculated using the following equation. RSp(j,k,tm)=max[G{ESp(j,k=all,tm), KSp(j,k=all,q=all,tm=all)}] (c2) That is, each of a plurality of target feature spectra ESp obtained from the output of all partial regions Rn generated from a specific layer j in target data IDE during a certain unit period tm is compared with each of reference feature spectra KSp obtained from the output of all partial regions Rn for each of all reference data IDS to calculate a plurality of individual spectral similarities S. Then, the maximum value of the calculated individual spectral similarities S is calculated as the spectral similarity RSp.

[0056] Regarding the above first calculation method M1 to third calculation method M3, the spectral similarity RSp may be calculated by comparing the target data IDE and the reference data IDS in the same unit period tm to calculate the individual spectral similarity S.

[0057] FIG. 14 is a diagram showing an evaluation result of the evaluation device 100 of this embodiment. 10,000 sets of learning sets LS, in which images of MNIST data are used as general-purpose learning data LD, were prepared, and the machine learning model 200 was trained. The reference data IDS is a video of the robot 900 operating normally. On the other hand, the target data IDE is a video of the robot 900 operating normally for a certain period until time t12, and operating abnormally for the remaining period after time t12. The period indicated by single hatching after time t12 in FIG. 14 is the period during which the robot 900 operates abnormally. The spectral similarity RSp is calculated at the end of each unit period. In this embodiment, for example, the spectral similarity RSp in the unit period from time t0 to time t1 is calculated immediately after time t1. The threshold value th in step S132 shown in FIG. 9 is set based on the normal spectral similarity, which is the spectral similarity RSp for each unit period during a certain period during which the robot 900 indicated by the target data IDE is operating normally. Specifically, the threshold value th was set to a value obtained by subtracting 3σ of the distribution of each normal spectral similarity from the average value of each normal spectral similarity.

[0058] Regarding the evaluation result of the target data IDE by the evaluation device 100, in the period up to time t12 when the robot 900 is operating normally, the spectral similarity RSp is equal to or greater than the threshold th, and the target data IDE is classified and determined to be in a reference class indicating normality. In addition, in the period after time t12 when the robot 900 is operating abnormally, the proportion of the spectral similarity RSp being less than the threshold th exceeds 90%. Therefore, in the period after time t12, the probability that the target data IDE is correctly classified and determined to be in a class indicating abnormality is high. Therefore, the accuracy of the evaluation result of the target data IDE using the machine learning model 200 trained using the general-purpose training data LD is high.

[0059] According to the above embodiment, the machine learning model 200 is trained using generic training data LD of a type different from the target data IDE to be evaluated, so that even if the type of the target data IDE is different and the purpose of using the machine learning model 200 is different, it is not necessary to prepare a training set for the machine learning model 200 for each purpose. Furthermore, according to this embodiment, the evaluation of the target data IDE is not performed using a judgment value corresponding to each class output from the output layer, but is performed using a feature spectrum Sp acquired from the output of a specific layer. As a result, even if the machine learning model 200 is trained using generic training data LD, the evaluation of the target data IDE can be performed with high accuracy.

[0060] Also, according to the above embodiment, the reference feature spectrum KSp is obtained from the reference data IDS obtained by extracting the arm 902 of the robot 900, which is the reference object, from the original reference video image RD. This makes it possible to obtain a feature spectrum Sp that better represents the features of the robot 900, which is the comparison source for calculating the spectral similarity RSp, particularly the normal operation of the robot 900. This makes it possible to further improve the evaluation accuracy of the target data IDE. Also, according to the above embodiment, the object feature spectrum ESp is obtained from the object data IDE obtained by extracting the robot 900, which is the object to be evaluated, from the original object video image, so it is possible to obtain a feature spectrum Sp that better represents the features of the robot 900, particularly the movement of the robot 900. This makes it possible to further improve the evaluation accuracy of the target data IDE.

[0061] B. Other embodiments: B-1. Alternative embodiment 1: In the above embodiment, the general-purpose learning data LD, the reference data IDS, and the target data IDE are moving images, but each data may be a still image or may be two-dimensional data with a physical quantity on the first axis and time on the second axis. The two-dimensional data is, for example, data showing a time change in voltage. In the above embodiment, the trained machine learning model 200 is used for the purpose of determining whether the operation of the robot 900 indicated by the target data IDE is normal, but the trained machine learning model 200 may be used for purposes other than this. A specific example will be described below.

[0062] The evaluation device 100 of the present disclosure can be applied to evaluation using exercise equipment. The exercise equipment can communicate data with the evaluation device 100 by wire or wirelessly. The exercise equipment is, for example, a treadmill, an exercise bike (registered trademark), or a muscle training machine. The treadmill is a machine in which a belt driven by a motor moves based on the user's settings, and the user can exercise by walking or running on the belt. The treadmill includes a treadmill body having a motor and a belt, an ultrasonic device which is an example of a sensor device 400 capable of measuring the muscle condition of the user, and a controller for controlling the treadmill body and the ultrasonic device. The muscle training machine can electrically change the load applied to the target part of the user to be trained in weight training.

[0063] The machine learning model 200 is trained using a plurality of training sets, each of which is a set of general-purpose training data LD, which is MNIST data that is a still image, and a label LB associated with the general-purpose training data LD.

[0064] The reference data IDS is, for example, data showing the state of the muscle of the target part of a subject other than the user, and is an ultrasound image of the muscle acquired by an ultrasound device. The reference evaluation associated with the reference data IDS indicates that the state of the muscle of the target part of the subject is good, that is, that the muscle of the target part is not fatigued.

[0065] The target data IDE is an ultrasound image of the muscle in the target part when the user is exercising using the exercise machine. The target data IDE is acquired and evaluated at regular intervals. The evaluation device 100 calculates a spectral similarity RSp between the reference feature spectrum KSp of the reference data IDS and the target feature spectrum ESp of the target data IDE. When the calculated spectral similarity RSp is equal to or greater than a threshold, the evaluation unit 119 performs a class classification indicating that the target part of the user is not fatigued. On the other hand, when the calculated spectral similarity RSp is less than the threshold, the evaluation unit 119 performs a class classification indicating that the target part of the user is fatigued, and performs a predetermined post-processing. As the post-processing, the display unit 150 displays warning information that prompts the user to reduce the load on the exercise machine or to stop exercising with the exercise machine, or transmits a load reduction command, which is a command to reduce the load or a command to stop the operation, to the exercise machine. When the load is reduced on the treadmill, the treadmill controller reduces the rotation speed of the motor.

[0066] Also, reference data IDS may be prepared for each of a plurality of levels of reference evaluation. For example, the fatigue level of the target part of the subject is classified into a plurality of levels, and the target data IDE is acquired for each fatigue level, which is the reference evaluation. For example, the fatigue level may be expressed in three levels, "low", "medium", and "high", for the reference evaluation, and a reference feature spectrum KSp may be acquired from the reference data IDS corresponding to each reference evaluation. The calculation unit 117 calculates the spectral similarity RSp between the reference feature spectrum KSp corresponding to each fatigue level and the target feature spectrum ESp of the target data IDE for each reference evaluation, i.e., for each fatigue level. The evaluation unit 119 identifies the reference feature spectrum KSp that is the source of calculation of the spectral similarity RSp showing the largest value among the multiple spectral similarities RSp calculated for each reference evaluation. The evaluation unit 119 evaluates the reference evaluation associated with the identified reference feature spectrum KSp, i.e., the fatigue level, as the fatigue level of the target part of the user. The evaluation unit 119 transmits a load setting command corresponding to the evaluated fatigue level to the exercise equipment.

[0067] In addition, when the target data IDE and the reference data IDS are not moving images but still images acquired at each time, or data indicating physical quantities such as voltage and current detected by the sensor device 400 for a certain period of time, the parameter tm is omitted in the above first to third calculation methods.

[0068] C. Other forms: The present disclosure is not limited to the above-mentioned embodiment, and can be realized in various forms without departing from the spirit of the present disclosure. For example, the present disclosure can be realized in the following aspects. The technical features in the above-mentioned embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined in order to solve some or all of the problems of the present disclosure, or to achieve some or all of the effects of the present disclosure. Furthermore, if the technical feature is not described as essential in this specification, it can be appropriately deleted.

[0069] (1) According to a first embodiment of the present disclosure, there is provided an evaluation method for evaluating target data, comprising: (a) inputting a plurality of learning sets of generic learning data of a type different from the target data and labels corresponding to the generic learning data into a vector neural network type machine learning model having a plurality of vector neuron layers to train the machine learning model; (b) after the step (a), inputting reference data of the same type as the target data, the evaluation of which indicates a predetermined reference evaluation, into the trained machine learning model to obtain a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model; (c) after the step (a), inputting the target data to be evaluated into the trained machine learning model to obtain a target feature spectrum as the feature spectrum from an output of the specific layer; (d) calculating a spectral similarity that is a similarity between the reference feature spectrum and the target feature spectrum; and (e) evaluating the target data using the spectral similarity. According to this embodiment, by training the machine learning model using generic training data of a type different from the target data to be evaluated, even if the type of target data is different and the purpose of using the machine learning model is different, there is no need to prepare a training set for the machine learning model for each purpose. Also, according to this embodiment, the evaluation of the target data is not performed using a judgment value corresponding to each class output from the output layer, but using a feature spectrum obtained from the output of a specific layer. As a result, even if the machine learning model is trained using generic training data, the evaluation of the target data can be performed with high accuracy.

[0070] (2) In the above embodiment, the step (e) may evaluate the target data by classifying the target data into two or more classes, the reference evaluation may be an evaluation in which the target data is classified into a reference class, and the step (e) may classify the target data into the reference class if the spectral similarity is equal to or greater than a predetermined threshold, and classify the target data into a class different from the reference class if the spectral similarity is less than the threshold. According to this embodiment, by determining whether the spectral similarity is equal to or greater than a threshold, the class classification of the target data can be facilitated.

[0071] (3) In the above embodiment, the plurality of vector neuron layers may have, in order from the target data side, a convolution vector neuron layer as an intermediate layer and a classification vector neuron layer as an output layer, and the specific layer may be the intermediate layer. According to this embodiment, the target data can be evaluated using the feature spectrum obtained from the output of the intermediate layer.

[0072] (4) In the above embodiment, the general-purpose learning data, the reference data, and the target data are each a video image composed of a plurality of frame images arranged in time series, and the evaluation method may further include a step of (f) generating a plurality of processed reference frame images arranged in time series as the reference data by generating a plurality of processed reference frame images in which the reference object is extracted using a plurality of reference frame images constituting an original reference video image obtained by capturing an image of the movement of the reference object. According to this embodiment, since the reference feature spectrum is obtained from the reference data in which the reference object is extracted from the original reference video image, a feature spectrum that more accurately represents the features of the reference object can be obtained. This can further improve the evaluation accuracy of the target data.

[0073] (5) In the above embodiment, the general-purpose learning data, the reference data, and the target data are each a moving image composed of a plurality of frame images arranged in a chronological order, and the evaluation method may further include a step of (g) generating a plurality of processed target frame images arranged in a chronological order as the target data by using a plurality of target frame images constituting an original target moving image obtained by capturing the movement of the evaluation object to generate a plurality of processed target frame images from which the evaluation object is extracted.

[0074] (6) According to a second embodiment of the present disclosure, an evaluation device for evaluating target data is provided. The evaluation device includes a learning execution unit that inputs a plurality of learning sets of generic learning data of a type different from the target data and labels corresponding to the generic learning data to a vector neural network type machine learning model having a plurality of vector neuron layers to learn the machine learning model, a first acquisition unit that inputs reference data of the same type as the target data and whose evaluation indicates a reference evaluation to the trained machine learning model to acquire a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model, a second acquisition unit that inputs the target data to be evaluated to the trained machine learning model to acquire a target feature spectrum as the feature spectrum from an output of the specific layer, a calculation unit for calculating a spectral similarity that is a similarity between the reference feature spectrum and the target feature spectrum, and an evaluation unit that evaluates the target data using the spectral similarity. According to this embodiment, by training the machine learning model using generic learning data of a type different from the target data to be evaluated, even if the types of target data are different and the purposes for using the machine learning model are different, it is not necessary to prepare a learning set for the machine learning model for each purpose. Furthermore, according to this embodiment, the evaluation of the target data is performed using the feature spectrum acquired from the output of a specific layer, rather than using the judgment values ​​corresponding to each class output from the output layer, which allows the target data to be evaluated with high accuracy even when the machine learning model is trained using general-purpose training data.

[0075] (7) According to a third aspect of the present disclosure, there is provided a computer program to be executed by a computer that evaluates target data. The computer program includes: (a) a function of inputting a plurality of learning sets of generic learning data of a type different from the target data and a label corresponding to the generic learning data into a vector neural network type machine learning model having a plurality of vector neuron layers to train the machine learning model; (b) a function of inputting reference data of the same type as the target data, the evaluation of which indicates a reference evaluation, into the trained machine learning model after executing the function (a), and acquiring a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model; (c) a function of inputting the target data to be evaluated into the trained machine learning model after executing the function (a), and acquiring a target feature spectrum as the feature spectrum from an output of the specific layer; (d) a function of calculating a spectral similarity that is a similarity between the reference feature spectrum and the target feature spectrum; and (e) a function of evaluating the target data using the spectral similarity. According to this embodiment, by training the machine learning model using generic training data of a type different from the target data to be evaluated, even if the type of target data is different and the purpose of using the machine learning model is different, there is no need to prepare a training set for the machine learning model for each purpose. Also, according to this embodiment, the evaluation of the target data is not performed using a judgment value corresponding to each class output from the output layer, but using a feature spectrum obtained from the output of a specific layer. As a result, even if the machine learning model is trained using generic training data, the evaluation of the target data can be performed with high accuracy.

[0076] The present disclosure may be realized in various forms other than those described above, for example, in the form of a non-transitory storage medium having a computer program recorded thereon. [Explanation of symbols]

[0077] ESp...target feature spectrum, FMK...reference frame image, FML...frame image, FMS...processed reference frame image, ID...data, IDE...target data, IDS...reference data, KSp...reference feature spectrum, KSpG...reference feature spectrum group, LB...label, LD...general-purpose learning data, LS...learning set, LSG...learning set group, Rn...subregion, R...rotation direction, RD...original reference video image, Sp...feature spectrum, x...first axis, y...second axis, z...third axis, 5...evaluation system, 100...evaluation device, 110...processor, 112...learning execution unit, 113...data processing unit, 114...spectrum acquisition unit, 115...first acquisition unit, 116...second acquisition unit, 117...calculation unit, 118...evaluation processing unit, 119...evaluation unit, 120...storage device, 130...interface circuit, 140...input device, 150...display unit, 200...machine learning model, 220...convolution layer, 230...primary vector neuron layer, 240...first convolution vector neuron layer, 250...second convolution vector neuron layer, 260...classification vector neuron layer, 400...sensor device, 900...robot, 901...base, 902...arm, 923, 924...object

Claims

1. An evaluation method in which a computer evaluates target data, (a) inputting a plurality of training sets of general-purpose training data of a type different from the target data and labels corresponding to the general-purpose training data into a vector neural network type machine learning model having a plurality of vector neuron layers, and training the machine learning model; (b) after step (a), inputting reference data of the same type as the target data, the evaluation of which indicates a predetermined reference evaluation, into a trained machine learning model, and acquiring a reference feature spectrum as a feature spectrum from the output of a specific layer of the trained machine learning model; (c) after the step (a), inputting the target data to be evaluated into the trained machine learning model and acquiring a target feature spectrum as the feature spectrum from the output of the specific layer; (d) calculating a spectral similarity between the reference feature spectrum and the target feature spectrum; (e) evaluating the target data using the spectral similarity.

2. The evaluation method according to claim 1, The step (e) includes evaluating the target data by classifying the target data into two or more classes; The reference evaluation is an evaluation classified into a reference class, The step (e) If the spectral similarity is equal to or greater than a predetermined threshold, classifying the target data into the reference class; When the spectral similarity is less than the threshold, the target data is classified into a class different from the reference class.

3. The evaluation method according to claim 1, The plurality of vector neuron layers include, in order from the target data side, a convolution vector neuron layer as an intermediate layer and a classification vector neuron layer as an output layer; The evaluation method, wherein the specific layer is the intermediate layer.

4. The evaluation method according to claim 1, the general-purpose learning data, the reference data, and the target data are each a moving image composed of a plurality of frame images arranged in time series, The evaluation method further includes: (f) An evaluation method comprising a step of generating a plurality of processed reference frame images arranged in chronological order as the reference data by using a plurality of reference frame images constituting an original reference moving image obtained by capturing the movement of a reference object to generate a plurality of processed reference frame images from which the reference object has been extracted.

5. The evaluation method according to claim 1, the general-purpose learning data, the reference data, and the target data are each a moving image composed of a plurality of frame images arranged in time series, The evaluation method further includes: (g) An evaluation method comprising a step of generating a plurality of processed target frame images arranged in chronological order as the target data by using a plurality of target frame images constituting an original target moving image obtained by capturing the movement of the evaluation target, to generate a plurality of processed target frame images from which the evaluation target is extracted.

6. An evaluation device that evaluates target data, a learning execution unit that inputs a plurality of learning sets of general-purpose learning data of a type different from the target data and labels corresponding to the general-purpose learning data into a vector neural network type machine learning model having a plurality of vector neuron layers, and learns the machine learning model; a first acquisition unit that inputs reference data of the same type as the target data, the evaluation of which indicates a reference evaluation, into a trained machine learning model, and acquires a reference feature spectrum as a feature spectrum from an output of a specific layer of the trained machine learning model; a second acquisition unit that inputs the target data to be evaluated to the trained machine learning model and acquires a target feature spectrum as the feature spectrum from an output of the specific layer; a spectral similarity calculation unit that calculates a similarity between the reference feature spectrum and the target feature spectrum; an evaluation unit that evaluates the target data using the spectral similarity.

7. A computer program to be executed by a computer that evaluates target data, (a) a function of inputting a plurality of learning sets of general-purpose learning data of a type different from the target data and labels corresponding to the general-purpose learning data into a vector neural network type machine learning model having a plurality of vector neuron layers, and training the machine learning model; (b) a function of, after executing the function (a), inputting reference data of the same type as the target data, in which the evaluation indicates a reference evaluation, into a trained machine learning model, and acquiring a reference feature spectrum as a feature spectrum from the output of a specific layer of the trained machine learning model; (c) a function of inputting the target data to be evaluated into the trained machine learning model after executing the function (a) and acquiring a target feature spectrum as the feature spectrum from the output of the specific layer; (d) a function of calculating a spectral similarity between the reference feature spectrum and the target feature spectrum; (e) a function of evaluating the target data using the spectral similarity.

Images