Method for extracting inappropriate and defective data from multiple learning data used in learning a machine learning model, information processing device, and computer program

Abstract

To provide a technique of extracting defective data contained in a plurality of pieces of training data.SOLUTION: A method includes the steps of: (b1) selecting a reference class from a plurality of classes; (b2) calculating a plurality of degrees of similarity between a feature spectrum corresponding to target training data and a plurality of the feature spectra belonging to the reference class; (b3) applying, to the plurality of degrees of similarity, a defectiveness function that is determined in advance, and calculating a defectiveness index with respect to the target training data; and (b4) determining whether or not the target training data is the defective data, based on a result of comparison between the defectiveness index and a threshold value.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to a method for extracting inappropriate and defective data from multiple pieces of training data used for training a machine learning model, an information processing device, and a computer program. [Background technology]

[0002] Patent Documents 1 and 2 disclose a vector neural network type machine learning model that uses vector neurons, called a capsule network. A vector neuron is a neuron whose input and output are vectors. A capsule network is a machine learning model that uses vector neurons called capsules as network nodes. A vector neural network type machine learning model such as a capsule network can be used for classification processing of input data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 5,210,798 [Patent Document 2] International Publication No. 2019 / 083553 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, training data used to train machine learning models may contain inappropriate, incomplete data such as outlier data and overlapping data. Outlier data is data with significantly different characteristics from normal training data sets in general. Overlapping data is data with characteristics significantly similar to normal training data from a different class. It is known that the presence of incomplete data in training data hinders the successful training and validation of machine learning models. Therefore, there has long been a need for technology that can extract incomplete data contained in multiple training data sets. [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a method for extracting inappropriate, defective data from a plurality of training data sets used to train a machine learning model that classifies input data into a plurality of classes. The machine learning model is configured as a vector neural network having a plurality of vector neuron layers. The method includes: (a) inputting the plurality of training data sets into the trained machine learning model, respectively, to obtain feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the plurality of training data sets into classes; and (b) selecting target training data from the plurality of training data sets and determining whether the target training data corresponds to the defective data. The step (b) includes: (b1) selecting a reference class from the plurality of classes; (b2) calculating multiple similarities between the feature spectrum for the target training data and multiple feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the multiple similarities; and (b4) determining whether the target training data corresponds to the deficiency data based on a comparison result between the deficiency index and a threshold value. In one embodiment, the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index. In another embodiment, the specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as multiple channels along a third axis that is oriented differently from the first and second axes, and the feature spectrum is one of (i) a first type of feature spectrum in which multiple element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the multiple channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the multiple channels along the third axis.

[0006] According to a second aspect of the present disclosure, there is provided an information processing device that executes a process of extracting inappropriate, defective data from a plurality of training data used to train a machine learning model that classifies input data into a plurality of classes. The information processing device includes a memory that stores a machine learning model configured as a vector neural network having a plurality of vector neuron layers, and a processor that executes calculations using the machine learning model. The processor executes the following processes: (a) inputting each of the plurality of training data into the trained machine learning model, obtaining feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the plurality of training data by class; and (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data. The process (b) includes: (b1) a process of selecting a reference class from the plurality of classes; (b2) a process of calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) a process of calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; and (b4) a process of determining whether the target training data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value. In one embodiment, the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index. In another embodiment, the specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as multiple channels along a third axis that is oriented differently from the first and second axes, and the feature spectrum is one of (i) a first type of feature spectrum in which multiple element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the multiple channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the multiple channels along the third axis.

[0007] According to a third aspect of the present disclosure, there is provided a computer program that causes a processor to execute a process of extracting inappropriate deficiency data from multiple training data sets used to train a machine learning model that classifies input data into multiple classes. The computer program causes the processor to execute the following processes: (a) inputting each of the multiple training data sets into the trained machine learning model, obtaining feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the multiple training data sets into classes; and (b) selecting target training data from the multiple training data sets and determining whether the target training data corresponds to the deficiency data. The process (b) includes: (b1) selecting a reference class from the multiple classes; (b2) calculating multiple similarities between the feature spectrum for the target training data and multiple feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the multiple similarities; and (b4) determining whether the target training data corresponds to the deficiency data based on a comparison result between the deficiency index and a threshold value. In one embodiment, the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index. In another embodiment, the specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as multiple channels along a third axis that is oriented differently from the first and second axes, and the feature spectrum is one of (i) a first type of feature spectrum in which multiple element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the multiple channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the multiple channels along the third axis. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a classification processing system according to an embodiment. [Figure 2] FIG. 1 is an explanatory diagram showing an example of the configuration of a machine learning model. [Figure 3] 10 is a flowchart showing the overall procedure of a process for extracting defective data. [Figure 4] FIG. 10 is an explanatory diagram showing an example of learning data. [Figure 5] FIG. [Figure 6] FIG. 2 is an explanatory diagram showing the configuration of a group of characteristic spectra. [Figure 7] 10 is a flowchart showing a procedure for extracting outlier data. [Figure 8] FIG. 10 is an explanatory diagram showing an example of the distribution of a similarity group Sq c,c related to normal training data. [Figure 9] FIG. 10 is an explanatory diagram showing an example of the distribution of a similarity group Sq c,c relating to outlier data. [Figure 10] 10 is a flowchart showing a procedure for extracting overlap data. [Figure 11] FIG. 10 is an explanatory diagram showing an example of the distribution of a similarity group Sq c,c′ related to normal training data. [Figure 12] FIG. 10 is an explanatory diagram showing an example of the distribution of a similarity group Sq c,c′ regarding overlap data. [Figure 13] FIG. 4 is an explanatory diagram showing a first calculation method of similarity. [Figure 14] FIG. 10 is an explanatory diagram showing a second similarity calculation method. [Figure 15] FIG. 10 is an explanatory diagram showing a third similarity calculation method. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. Implementation: FIG. 1 is a block diagram showing a classification processing system according to an embodiment. The classification processing system includes an information processing device 100 and a camera 400. The camera 400 is used to capture images as learning data. The camera 400 may be a camera that captures color images, or a camera that captures monochrome images or spectral images. In this embodiment, images captured by the camera 400 are used as learning data, but data other than images may also be used as learning data. In this case, a learning data acquisition device appropriate for the type of data is used instead of the camera 400.

[0010] In this disclosure, the phrase "training data" is used as a term that includes both training data and validation data. Training data is labeled data used to adjust the internal parameters of a machine learning model. Validation data is labeled data used to validate a trained machine learning model. However, in the embodiment described below, an example will be described in which training data is used as training data, and defective data is extracted or detected from the training data. "Defective data" may include outlier data and overlap data. Outlier data is data whose characteristics are significantly different from those of a normal training data set in general. Overlap data is data that has characteristics significantly similar to normal training data of a different class.

[0011] The information processing device 100 has a processor 110, a memory 120, an interface circuit 130, and an input device 140 and a display device 150 connected to the interface circuit 130. A camera 400 is also connected to the interface circuit 130. For example, but not limited to, the processor 110 not only has the function of executing the processes described in detail below, but also has the function of displaying on the display device 150 data obtained by the processes and data generated in the course of the processes.

[0012] The processor 110 functions as a learning execution unit 112 that executes learning of a machine learning model, and a deficiency data extraction unit 114 that executes processing to extract deficiency data from training data. The deficiency data extraction unit 114 includes a similarity calculation unit 310 and a deficiency index calculation unit 320. The learning execution unit 112 and the deficiency data extraction unit 114 are each implemented by the processor 110 executing a computer program stored in the memory 120. However, the learning execution unit 112 and the deficiency data extraction unit 114 may also be implemented as hardware circuits. The term "processor" in the present disclosure also includes such hardware circuits. Furthermore, one or more processors that execute the learning processing and the deficiency data extraction processing may be processors included in one or more remote computers connected via a network.

[0013] The memory 120 stores a machine learning model 200, a training data group LT, and a feature spectrum group GSp. An example configuration and operation of the machine learning model 200 will be described later. The training data group LT is a collection of labeled data used for training the machine learning model 200. In this embodiment, the training data group LT is a collection of image data as training data. The feature spectrum group GSp is a collection of feature spectra obtained by inputting training data to be processed into the trained machine learning model 200. The feature spectrum will be described later.

[0014] FIG. 2 is an explanatory diagram showing the configuration of a machine learning model 200. This machine learning model 200 has an input layer 210, a hidden layer 280, and an output layer 260. The hidden layer 280 includes a convolutional layer 220, a primary vector neuron layer 230, a first convolutional vector neuron layer 240, and a second convolutional vector neuron layer 250. The output layer 260 is also referred to as a "classification vector neuron layer 260." Of these layers, the input layer 210 is the lowest layer, and the output layer 260 is the highest layer. In the following description, the layers of the hidden layer 280 are referred to as a "Conv layer 220," a "PrimeVN layer 230," a "ConvVN1 layer 240," and a "ConvVN2 layer 250," respectively, and the output layer 260 is referred to as a "ClassVN layer 260."

[0015] 2, two convolution vector neuron layers 240 and 250 are used, but the number of convolution vector neuron layers is arbitrary, and a convolution vector neuron layer may be omitted. However, it is preferable to use one or more convolution vector neuron layers.

[0016] An image of 28×28 pixels is input to the input layer 210. The configuration of each layer other than the input layer 210 can be described as follows. ·Conv layer 220: Conv[32,5,2] ·PrimeVN layer 230: PrimeVN[16,1,1] ·ConvVN1 layer 240:ConvVN1[12,3,2] ·ConvVN2 layer 250:ConvVN2[6,3,1] ·ClassVN layer 260:ClassVN[M,3,1] Vector dimension VD: VD=16 In the description of each layer, the character string before the parentheses is the layer name, and the numbers in the parentheses are, in order, the number of channels, the kernel surface size, and the stride. For example, the layer name of the Conv layer 220 is "Conv," the number of channels is 32, the kernel surface size is 5x5, and the stride is 2. In Figure 2, these descriptions are shown below each layer. The hatched rectangles drawn in each layer represent the kernel surface size used when calculating the output vector of the adjacent higher layer. In this embodiment, since the input data is image data, the kernel surface size is also two-dimensional. Note that the parameter values ​​used in the description of each layer are merely examples and can be changed as desired.

[0017] The input layer 210 and the Conv layer 220 are layers composed of scalar neurons. The other layers 230 to 260 are layers composed of vector neurons. A vector neuron is a neuron that uses vectors as input and output. In the above description, the dimension of the output vector of each vector neuron is constant at 16. In the following, the term "node" is used as a superordinate concept of scalar neurons and vector neurons.

[0018] FIG. 2 shows the first axis x and second axis y that define the planar coordinates of the node array for the Conv layer 220, and the third axis z that represents depth. It also shows that the sizes of the Conv layer 220 in the x, y, and z directions are 12, 12, and 32. The sizes in the x and y directions are called "resolution." The size in the z direction is the number of channels. These three axes x, y, and z are also used in other layers as coordinate axes that indicate the position of each node. However, in FIG. 2, these axes x, y, and z are omitted from the illustration for layers other than the Conv layer 220.

[0019] As is well known, the resolution W1 after convolution is given by the following equation: W1=Ceil{(W0-Wk+1) / S} (A1) Here, W0 is the resolution before convolution, Wk is the surface size of the kernel, S is the stride, and Ceil{X} is a function that rounds up the decimal point of X. The resolution of each layer shown in FIG. 2 is an example in which the resolution of the input data is 28, and the actual resolution of each layer is changed appropriately depending on the size of the input data.

[0020] The ClassVN layer 260 has M channels. M is the number of classes determined by the machine learning model 200. In this embodiment, M is 2, and two class determination values ​​Class_1 and Class_2 are output. The number of channels M of the ClassVN layer 260 can be set to any integer equal to or greater than 2.

[0021] FIG. 2 also illustrates subregions Rn in each layer 220, 230, 240, 250, and 260. The subscript "n" in subregion Rn refers to the layer's code. For example, subregion R220 indicates a subregion in the Conv layer 220. A "subregion Rn" is a region in each layer that is identified by a planar position (x, y) defined by the position of the first axis x and the position of the second axis y, and that includes multiple channels along the third axis z. The subregion Rn has dimensions of "Width" × "Height" × "Depth," corresponding to the first axis x, the second axis y, and the third axis z. In this embodiment, the number of nodes included in one "subregion Rn" is "1 × 1 × depth count," i.e., "1 × 1 × channel count."

[0022] 2, a feature spectrum Sp, which will be described later, is calculated from the output of the ConvVN2 layer 250. The similarity calculation unit 310 uses this feature spectrum Sp to calculate the similarity between each training data and other training data. The defective data is extracted using this similarity.

[0023] In the present disclosure, the vector neuron layer used to calculate the similarity is also referred to as the "specific layer." As the specific layer, a vector neuron layer other than the ConvVN2 layer 250 may be used, and any number of vector neuron layers greater than or equal to one may be used. The configuration of the feature spectrum Sp and the method of calculating the similarity using the feature spectrum Sp will be described later.

[0024] 3 is a flowchart showing the overall procedure for the process of extracting defective data. In step S110, the learning execution unit 112 uses the learning data group LT to execute learning of the machine learning model 200. When learning is completed, the trained machine learning model 200 is stored in the memory 120.

[0025] FIG. 4 is an explanatory diagram showing an example of training data. The machine learning model 200 of this embodiment is configured to input an image showing the attachment state of a part to a product and output class judgment values ​​Class_1 and Class_2 indicating pass or fail. The training data used includes four types of data: pass data LT1, fail data LT2, outlier data LT3, and overlap data LT4. The pass data LT1 and fail data LT2 are normal training data, and the outlier data LT3 and overlap data LT4 are defective data.

[0026] The pass data LT1 is an image in which the mounting angle of the part is within a normal range. The fail data LT2 is an image in which the mounting angle of the part is within an abnormal range and retightening is required. The pass data LT1 was assigned a label "1". The fail data LT2 was assigned a label "2". In this embodiment, many sheets of pass data LT1 and fail data LT2 were prepared. In this disclosure, "class" and "label" are synonymous terms.

[0027] The outlier data LT3 is an image in which the mounting angle of the component is within the normal range, but the component is positioned off-center. The outlier data LT3 includes multiple images that have been assigned the same label "1" as the pass data LT1. The overlap data LT4 is an image with an odd mounting angle that can be classified as either pass data LT1 or fail data LT2. The overlap data LT4 includes multiple images that have been assigned the label "1" and multiple images that have been assigned the label "2."

[0028] In step S120, the learning execution unit 112 generates a feature spectrum set GSp by inputting multiple pieces of training data to be subjected to the process of extracting defective data into the trained machine learning model 200. The feature spectrum set GSp is a collection of feature spectra, which will be described below.

[0029] FIG. 5 is an explanatory diagram showing a feature spectrum Sp obtained by inputting arbitrary input data to the trained machine learning model 200. As shown in FIG. 2, in this embodiment, the feature spectrum Sp is created from the output of the ConvVN2 layer 250. The horizontal axis in FIG. 5 represents the position of a vector element in the output vector of multiple nodes included in one subregion R250 of the ConvVN2 layer 250. The position of this vector element is represented by a combination of the element number ND of the output vector at each node and the channel number NC. In this embodiment, since the vector dimension is 16 (the number of elements of the output vector output by each node), the element number ND of the output vector is 16, ranging from 0 to 15. Furthermore, since the ConvVN2 layer 250 has six channels, the channel number NC is 6, ranging from 0 to 5. In other words, this feature spectrum Sp is obtained by arranging multiple element values ​​of the output vector of each vector neuron included in one subregion R250 across multiple channels along the third axis z.

[0030] The vertical axis of Fig. 5 represents the feature value C at each spectral position. V In this example, the feature value C V is the value of each element of the output vector V NDFeature value C V For the feature value C, statistical processing such as centering to the average value 0 may be performed. V As the value of each element of the output vector V ND Alternatively, the normalization coefficient may be used as is. In the latter case, the feature value C included in the feature spectrum Sp may be used as is. V The number of is equal to the number of channels, which is 6. The normalization coefficient is a value corresponding to the vector length of the output vector of the node.

[0031] The number of feature spectra Sp obtained from the output of the ConvVN2 layer 250 for one piece of input data is nine, since it is equal to the number of planar positions (x, y) of the ConvVN2 layer 250, that is, the number of subregions R250.

[0032] In step S120, the learning execution unit 112 inputs learning data to be subjected to the process of extracting defective data into the trained machine learning model 200, calculates the feature spectrum Sp shown in FIG. 5, and registers it in the memory 120 as a feature spectrum group GSp. In this embodiment, since training data is subjected to the process of extracting defective data, the multiple learning data used in step S120 are the same as the multiple learning data used in step S110. Note that if verification data is subjected to the process of extracting defective data, the multiple learning data used in step S120 will be different from the multiple learning data used in step S110. The following four combinations are possible for the first learning data used in step S120 and the second learning data to be subjected to the process of extracting defective data. (1) Both the first learning data and the second learning data are training data. (2) The first learning data is training data, and the second learning data is validation data. (3) Both the first training data and the second training data are validation data. (4) The first learning data is validation data, and the second learning data is training data. In this embodiment, the first of these combinations is used.

[0033] 6 is an explanatory diagram showing the configuration of the feature spectrum set GSp. In this example, the feature spectrum set GSp obtained from the output of the ConvVN2 layer 250 is shown. Note that it is sufficient that the feature spectrum set GSp is obtained from the output of at least one vector neuron layer, and it is also possible to register a feature spectrum set GSp obtained from the output of the ConvVN1 layer 240 or the ClassVN layer 260.

[0034] Each record in the feature spectrum group GSp includes a parameter k indicating the order of the subregion Rn within the layer, a parameter c indicating the class, a parameter q indicating the data number, and a feature spectrum Sp. The feature spectrum Sp is the same as the feature spectrum Sp in Figure 5.

[0035] The parameter k of the subregion Rn takes a value indicating which of the multiple subregions Rn included in a specific layer the subregion Rn is, i.e., which planar position (x, y) it is. For the ConvVN2 layer 250, there are nine subregions R250, so k = 1 to 9. The parameter c representing the class takes a value indicating which of the M classes distinguishable by the machine learning model 200 the subregion is. In this embodiment, M = 2, so c = 1 to 2. The parameter q representing the data number indicates the sequential number of the training data belonging to each class, and takes a value from 1 to max1 for c = 1, and a value from 1 to max2 for c = 2. In this way, the feature spectrum Sp is associated with the class c and the data number q of the training data. Furthermore, the feature spectrum Sp is classified by class.

[0036] In step S130, the imperfect data extraction unit 114 extracts imperfect data from the plurality of training data sets using the characteristic spectrum group GSp. That is, the imperfect data extraction unit 114 extracts or detects outlier data LT3 and overlap data LT4 from the four types of training data shown in Fig. 4 using the characteristic spectrum Sp read from the memory 120. The detailed procedure of step S130 will be described later.

[0037] In step S140, the defective data extraction unit 114 executes a process to eliminate the defective data. For example, for outlier data, the process can eliminate the outlier data from the training data set, or perform a data augmentation process on the outlier data to increase the number of data points and thereby eliminate the outlier. For overlapping data, the process can eliminate the overlapping data from the training data set, or add a new class and assign the overlapping data to the new class.

[0038] In step S150, the learning execution unit 112 uses the training data set from which the defective data has been eliminated to re-train the machine learning model 200. By performing training using a training data set without defective data, it is possible to obtain a machine learning model 200 with good classification accuracy.

[0039] FIG. 7 is a flowchart showing the procedure for extracting outlier data, and shows the detailed procedure of step S130 in FIG. 3. In step S211, the defective data extraction unit 114 sets a parameter q indicating the data number of the target learning data to be judged and a parameter c indicating its class to q=1 and c=1. The parameters q and c correspond to the data number q and class c in the feature spectrum group GSp shown in FIG. 5. In the following, the target learning data will be referred to as "target learning data x q c " and call the class "target class c".

[0040] In step S212, the defective data extraction unit 114 sets a parameter c' indicating a reference class to c'=c. The "reference class" is the class of the target learning data x q c In the following, the reference class is called "reference class c'" and the training data belonging to the reference class is called "reference training data". In the similarity calculation described later, the target training data x q c A plurality of similarities are calculated between the feature spectrum Sp for the target class c and the feature spectrum Sp for a plurality of reference training data belonging to the reference class c'. When outlier data is extracted as defective data, the reference class c' is set to the same value as the target class c.

[0041] In step S213, the similarity calculation unit 310 calculates the target training data x q c and a set of reference training data X c The similarity group S between q c,c The symbol S indicating the similarity group is calculated. q c,c Among them, the subscript " q " means the data number q of the target training data, and the superscript " c,c The first "c" in "" represents the target class, and the second "c" represents the reference class. c means all the training data belonging to the reference class c'=c. In the example shown in FIG. 6, when the reference class c' is c'=1, the training data with data number q from 1 to max1 are included in the set X of reference training data. c The similarity group S q c,c is the target training data x q c The similarity S between the feature spectrum Sp for each reference training data and the feature spectrum Sp for each reference training data q c When c'=1, there is max1 reference training data, so the similarity set S q c,cis the max1 similarity S q c It includes the similarity S q c depends on the data number q' of the reference training data and the reference class c', but the similarity S q c In the symbols, these parameters q' and c' are omitted. The prime symbol "'" attached to the symbols of the parameters q' and c' indicates that they are the same as the parameters q and c shown in Figure 6 above, and that they relate to the reference training data.

[0042] In step S214, the defective data extraction unit 114 extracts the similarity group S q c,c By applying the deficiency function to q Calculate the deficiency index d q is the target training data x q c The imperfection function is an index that indicates the degree of imperfection of the similarity group S q c,c is used as input, and the deficiency index d q The imperfection function suitable for extracting outlier data is the similarity set S q c,c The distribution of and the similarity group S for outlier data q c,c This is determined taking into account the difference in distribution of

[0043] Figure 8 shows the similarity set S q c,c The horizontal axis represents the similarity S q c In the procedure of Figure 7, the reference class c' is the same as the target class c, so the similarity group S q c,c Similarity S included in q c is often close to 1.0. q cThe reason why there are some is that the target class c contains outlier data.

[0044] Figure 9 shows the similarity group S q c,c 1 is an explanatory diagram showing an example of the distribution of similarity group S q c,c Then, let us consider a number of similarities S q c tends to be significantly smaller than 1.0. The reason for this is that the target training data x q c If is an outlier, the target training data x q c This is because, has features that are different from the features common to normal training data belonging to the target class c.

[0045] As a deficiency function suitable for the process of extracting outlier data, any of the following can be used. (1) Incomplete function f1: The incomplete function f1 is the similarity group S q c,c The statistically representative value of the deficiency index d q The statistical representative value can be, for example, the average value. In some cases, such as in the overlap data extraction process described below, the maximum value can also be used as the statistical representative value used in the imperfection function f1. (2) Incomplete function f2: The incomplete function f2 is the similarity group S q c,c The representative value of the histogram is the deficiency index d q The representative value of the histogram can be the median or the mode of the histogram. q c,c The representative value of the histogram of the similarity group S q c,c Since the second imperfection function f2 is a kind of statistical representative value of the first imperfection function f1, the second imperfection function f2 corresponds to a lower concept of the first imperfection function f1. (3) Incomplete function f3: The incomplete function f3 is the similarity group S qc,c The histogram is divided into one or more unimodal distributions, and the representative value in the representative unimodal distribution selected according to predetermined selection conditions is used as the deficiency index d q The third deficiency function f3 corresponds to a lower level concept of the second deficiency function f2.

[0046] In Figure 9, the deficiency index d calculated using the third deficiency function f3 described above is q In this example, first, the similarity group S q c,c The histogram of Ud11 and Ud12 is fitted with a mixed Gaussian distribution using the EM algorithm (Expectation-Maximization Algorithm) to obtain multiple unimodal distributions Ud11 and Ud12. In addition, the following conditions C1 and C2 are used as selection conditions for selecting one representative unimodal distribution from the multiple unimodal distributions Ud11 and Ud12. <Condition C1> Similarity group S q c,c The ratio of the area of ​​one unimodal distribution to the total area of ​​the histogram is greater than or equal to the area threshold. <Condition C2> In the unimodal distribution that satisfies condition C1, the similarity S q c The average value of is the largest.

[0047] The area threshold in the above condition C1 is set to a value of, for example, about 5 to 10%. This condition C1 is set to the similarity group S for outlier data as shown in FIG. q c,c However, the similarity S is close to 1.0. q c Since it contains a small number of q c In the example of FIG. 9, of the two unimodal distributions Ud11 and Ud12, the second unimodal distribution Ud12 does not satisfy the above condition C1. Therefore, the first unimodal distribution Ud11 is selected as the representative unimodal distribution, and its representative value, the mode, is set as the deficiency index d q The target training data xq c If is an outlier, the deficiency index d q takes on a fairly small value.

[0048] As a selection condition for selecting one representative unimodal distribution from the plurality of unimodal distributions Ud11 and Ud12, conditions other than the above-mentioned conditions C1 and C2 may be used. For example, the one with the largest area among the plurality of unimodal distributions may be selected as the representative unimodal distribution. In this way, the similarity group S q c,c If we divide the histogram into one or more unimodal distributions and select one representative unimodal distribution from them, we can obtain a similarity group S q c,c Even if the histogram of contains multiple peaks, the appropriate deficiency index d q It is possible to obtain

[0049] In steps S215 to S217, the deficiency data extraction unit 114 extracts the deficiency index d q Depending on the comparison result with the first threshold Th1, the target training data x q c Determine whether d is an outlier or not. q If Th1, the target training data x q c is determined to be outlier data. <d q In this case, in step S217, the target training data x q c As shown in FIG. 9, the first threshold Th1 is set to the value of the target training data x q c The deficiency index d obtained when is an outlier data q is set to a value significantly larger than

[0050] In step S218, the deficiency data extraction unit 114 increments the target training data number q by 1. In step S219, the deficiency data extraction unit 114 determines whether the target training data number q exceeds its maximum value, that is, whether the processing of steps S213 to S217 has been completed for all training data of the target class c. If the processing for all training data of the target class c has not been completed, the process returns to step S213. On the other hand, if the processing for all training data of the target class c has been completed, the process proceeds to step S220.

[0051] In step S220, the deficiency data extraction unit 114 increments the target class c by 1 and sets the target learning data number q to 1. In step S221, the deficiency data extraction unit 114 determines whether the processing of steps S212 to S219 has been completed for all classes. If the processing for all classes has not been completed, the process returns to step S212. On the other hand, if the processing for all classes has been completed, the process of FIG. 7 ends.

[0052] As described above, by performing the processing according to the procedure in FIG. 7, outlier data can be extracted or detected from a plurality of training data.

[0053] FIG. 10 is a flowchart showing the procedure for extracting overlap data, showing the detailed procedure of step S130 in FIG. 3. The procedure in FIG. 10 is executed separately from the procedure in FIG. 7 described above. In this embodiment, the process for extracting outlier data according to the procedure in FIG. 7 and the process for extracting overlap data according to the procedure in FIG. 10 can each be executed as the process of step S130 in FIG. 3. The order in which these two extraction processes are executed is arbitrary. However, it is also possible to execute only one of the process in FIG. 7 and the process in FIG. 10.

[0054] Steps S311 to S321 in Fig. 10 roughly correspond to steps S211 to S221 in Fig. 7. More specifically, steps S311, S318 to S321 are the same as steps S211, S218 to S221, and steps S312 to S317 are different from steps S212 to S217. The contents of steps S312 to S317 will be described below.

[0055] In step S312, the deficiency data extraction unit 114 sets the reference class c' to {all classes other than c}, where "c" is the target class. In the example shown in FIG. 6, if c=1, then c'=2. Note that if the number of classes is three or more, the reference class c' will include multiple classes.

[0056] In step S313, the similarity calculation unit 310 calculates the target training data x q c and a set of reference training data X c’ The similarity group S between q c,c’ In the example shown in FIG. 6, when c=1 and c'=2, max2 training data belonging to class 2 are included in the set of reference training data X c’ Therefore, the similarity group S q c,c’ is the max2 similarity S q c’ Contains:

[0057] In step S314, the defective data extraction unit 114 extracts the similarity group S q c,c’ By applying the deficiency function to q The imperfection function suitable for extracting overlapping data is the similarity group S q c,c’ The distribution of and the similarity group S for overlapping data q c,c’ It is determined taking into account the difference from the distribution of

[0058] Figure 11 shows the similarity set S for normal training data. qc,c’ 1 is an explanatory diagram showing an example of the distribution of the similarity group S q c,c’ Similarity S included in q c Many of the similarities S are much smaller than 1.0. q c The reason there are several is that the reference class c' contains overlapping data.

[0059] Figure 12 shows the similarity group S q c,c’ 1 is an explanatory diagram showing an example of the distribution of the similarity group S q c,c’ Then, let us consider a number of similarities S q c takes a value close to 1.0. The reason for this is that the target training data x q c If the overlap data is the target training data x q c This is because, has characteristics similar to the characteristics of classes other than the target class c.

[0060] As the imperfection functions suitable for the overlap extraction process, it is possible to use the imperfection functions f1 to f3 that are almost the same as those described in the outlier data extraction process. That is, as the imperfection functions, the similarity group S q c,c’ The statistically representative value of the deficiency index d q In the example of FIG. 12, the third deficiency function f3 described above is used, and the second unimodal distribution Ud22 of the two unimodal distributions Ud21 and Ud22 is selected as the representative unimodal distribution. In addition, the mode, which is the representative value of this representative unimodal distribution Ud22, is used as the deficiency index d q It should be noted that the imperfection function used in the overlap extraction process and the imperfection function used in the outlier data extraction process may be different functions.

[0061] In steps S315 to S317, the defect data extraction unit 114 determines whether the target learning data x q corresponds to overlapping data according to the comparison result between the defect index d q c and the second threshold Th2. Specifically, when Th2 ≤ d q , in step S316, it is determined that the target learning data x q c is overlapping data. On the other hand, when d q < Th2, in step S317, it is determined that the target learning data x q c is normal learning data. As shown in FIG. 12, the second threshold Th2 is set to a value sufficiently smaller than the defect index d q c obtained when the target learning data x q is overlapping data. Note that the first threshold Th1 used for the outlier data extraction process and the second threshold Th2 used for the overlapping data extraction process may be set to the same value or different values.

[0062] By performing the process according to the procedure of FIG. 10 described above, overlapping data can be extracted or detected from a plurality of learning data.

[0063] In the procedure of FIG. 10, the reference class c' is set to {all classes other than the target class c}, and the defect index d q is obtained only once for {all classes other than the target class c}. Instead, the reference class c' may be set to {one class other than the target class c}, and the defect index d q is obtained for each individual reference class c'. In the latter case, when the defect index d q is greater than or equal to the second threshold Th2 for at least one class other than the target class c, it is determined that the target learning data x q c is overlapping data. Also, for all classes other than the target class c, the defect index d qis less than the second threshold Th2, the target training data x q c is determined to be non-overlapping data.

[0064] As described above, in the above embodiment, the deficiency index d q Using this, defective data can be extracted from the training data.

[0065] B. Similarity calculation method: As a method for calculating the similarity, for example, any of the following methods can be adopted. (1) Target training data x q c The feature spectrum Sp and the reference training data x q’ c’ A first calculation method M1 for calculating the similarity without considering the correspondence of the partial region Rn in the feature spectrum Sp of (2) Target training data x q c The feature spectrum Sp and the reference training data x q’ c’ A second calculation method M2 for calculating the similarity between corresponding subregions Rn of the feature spectrum Sp (3) A third calculation method M3 for calculating the similarity without considering the partial region Rn at all Below, methods for calculating similarity from the output of the ConvVN2 layer 250 according to these calculation methods M1, M2, and M3 will be described in order.

[0066] 13 is an explanatory diagram showing a first similarity calculation method M1. In the first calculation method M1, first, a local similarity SL for each subregion k is calculated from the output of the ConvVN2 layer 250, which is a specific layer. q c (k) is calculated according to the formula described later. In the machine learning model 200 of FIG. 2, the number of subregions R250 in the ConvVN2 layer 250 is 9, so the parameter k of the subregions takes values ​​from 1 to 9. Note that FIG. 13 shows an example where c=1 and c'=2. These local similarities SL q cFrom (k), three types of similarity S shown on the right side of Figure 13 are q c Either of the following is calculated.

[0067] In the first calculation method M1, the local similarity SL q c (k) is calculated using the following formula: SL q c (k) = max[G{Sp(k, c, q), Sp(k'=all, c', q')}] (B1) where: k, k' are parameters indicating the subregion Rn, c and c' are parameters indicating the target class and the reference class, q, q' are parameters indicating the data numbers of the target training data and the reference training data. G{a,b} is a function that calculates the similarity between a and b. Sp(k, c, q) is the target training data x q c The feature spectrum obtained from the output of a specific subregion k of a specific layer according to Sp(k'=all, c', q') is the reference training data x q’ c’ The feature spectrum obtained from the output of all subregions k' of a particular layer according to max[X] is a logical operation that takes the maximum value of X. As the function G{a, b} for calculating the similarity, for example, an equation for calculating cosine similarity or an equation for calculating similarity according to distance can be used.

[0068] The three types of similarity S shown on the right side of Figure 13 q c is the local similarity SL for multiple subregions k. q c The maximum, average, or minimum value of (k) is obtained. The calculation of the maximum, average, or minimum value to be used is preset by the user experimentally or empirically.

[0069] As described above, in the first similarity calculation method M1, (1) Target training data x q c The feature spectrum Sp(k, c, q) obtained from the output of a specific subregion k of a specific layer according to q’ c’ The local similarity SL is the similarity between the feature spectrum Sp(k'=all, c', q') obtained from the output of all subregions k' of a specific layer according to q c Find (k), (2) Local similarity SL for multiple subregions k q c The similarity S is calculated by taking the maximum, average, or minimum value of (k). q c Ask for. According to the first calculation method M1, the similarity S can be calculated by a relatively simple calculation and procedure. q c can be obtained.

[0070] 14 is an explanatory diagram showing a second similarity calculation method M2. In the second calculation method M2, the local similarity SL is calculated using the following equation instead of the above-mentioned equation (B1): q c Calculate (k). SL q c (k) = G{Sp(k, c, q), Sp(k'=k, c', q')} (B2) where: Sp(k'=k, c', q') is the reference training data x q’ c’ is the feature spectrum obtained from the output of a specific subregion k'=k of a specific layer according to

[0071] In the first calculation method M1 described above, the reference training data x q’ c’ In the first calculation method, the feature spectrum Sp(k'=all, c', q') obtained from the output of all subregions k' of a specific layer is used, whereas in the second calculation method M2, the target training data xq c The second calculation method M2 uses only the feature spectrum Sp(k'=k, c', q') for the subregion k'=k that is the same as the subregion k of the feature spectrum Sp(k, c, q) obtained according to the above. The other steps in the second calculation method M2 are the same as those in the first calculation method M1.

[0072] In the second similarity calculation method M2, (1) Target training data x q c The feature spectrum Sp(k, c, q) obtained from the output of a specific subregion k of a specific layer according to q’ c’ The local similarity SL is the similarity between the feature spectrum Sp(k'=all, c', q') obtained from the output of the corresponding subregion k'=k of a specific layer according to q c Find (k), (2) Local similarity SL for multiple subregions k q c The similarity S is calculated by taking the maximum, average, or minimum value of (k). q c Ask for. The second calculation method M2 also calculates the similarity S by relatively simple calculations and procedures. q c can be obtained.

[0073] 15 is an explanatory diagram showing a third similarity calculation method M3. In the third similarity calculation method M3, the local similarity SL q c (k) is calculated from the output of the specific layer, ConvVN2 layer 250. q c is calculated.

[0074] The similarity S obtained by the third calculation method M3 q c is calculated using the following formula: S q c= max[G{Sp(k=all, c, q), Sp(k'=all, c', q')}} (B3) where: Sp(k=all, c, q) is the target training data x q c The feature spectrum obtained from the output of all subregions k of a particular layer according to Sp(k'=all, c', q') is the reference training data x q’ c’ is the feature spectrum obtained from the outputs of all subregions k' of a specific layer according to

[0075] As described above, in the third similarity calculation method M3, (1) Target training data x q c The feature spectrum Sp(k=all, c, q) obtained from the output of all subregions k of a specific layer according to q’ c’ The similarity S between the feature spectrum Sp(k'=all, c', q') obtained from the output of all subregions k' of a specific layer according to q c Ask for. According to the third calculation method M3, the similarity S can be calculated by simpler calculations and procedures. q c can be obtained.

[0076] The above-mentioned three calculation methods M1 to M3 are all methods for calculating similarity using the output of one specific layer. However, the similarity can be calculated using one or more of the vector neuron layers 240, 250, and 260 shown in Fig. 2 as specific layers. When multiple specific layers are used, it is preferable to use, for example, the minimum value of multiple similarities obtained from the multiple specific layers as the final similarity.

[0077] C. How to calculate the output vector of each layer of the machine learning model: The calculation method for the output of each layer shown in Figure 2 is as follows.

[0078] Each node in the PrimeVN layer 230 regards the scalar output of the 1x1x32 nodes in the Conv layer 220 as a 32-dimensional vector and obtains the vector output of that node by multiplying this vector by a transformation matrix. This transformation matrix is ​​an element of a kernel with a surface size of 1x1, and is updated by learning the machine learning model 200. Note that the processing of the Conv layer 220 and the PrimeVN layer 230 can also be integrated into one primary vector neuron layer.

[0079] When the PrimeVN layer 230 is referred to as the "lower layer L" and the ConvVN1 layer 240 adjacent to it on the upper side is referred to as the "upper layer L+1", the output of each node in the upper layer L+1 is determined using the following equation.

number

[0080] As the normalization function F(X), for example, the following formula (E3a) or (E3b) can be used.

number

[0081] In the above equation (E3a), the sum vector u j Norm of |u j The activation value a is normalized by the softmax function | j On the other hand, in equation (E3b), the sum vector u j Norm of |u j | is the norm |u j Activation value a by dividing by the sum of | j It should be noted that a function other than equation (E3a) or (E3b) may be used as the normalization function F(X).

[0082] The ordinal number i in the above equation (E2) is the output vector M of the jth node in the upper layer L+1. L+1 j The integer n is assigned for convenience to the nodes in the lower layer L used to determine the output vector M L+1 j is the number of nodes in the lower layer L used to determine . Thus, the integer n is given by n = Nk × Nc (E5) Here, Nk is the surface size of the kernel, and Nc is the number of channels in the lower layer, the PrimeVN layer 230. In the example of FIG. 2, Nk=9 and Nc=16, so n=144.

[0083] One kernel used to calculate the output vector of the ConvVN1 layer 240 has a surface size of 3 × 3 and a depth of 16 channels in the lower layer, with 3 × 3 × 16 = 144 elements. Each of these elements is a prediction matrix W L ij In addition, 12 sets of this kernel are required to generate output vectors for 12 channels of the ConvVN1 layer 240. Therefore, the prediction matrix W of the kernel used to obtain the output vector of the ConvVN1 layer 240 is L ij The number of prediction matrices W is 144 × 12 = 1728. L ij is updated by learning of the machine learning model 200.

[0084] As can be seen from the above equations (E1) to (E4), the output vector M of each node in the upper layer L+1 L+1 j is calculated by the following calculation: (a) Output vector M of each node in the lower layer L L i The prediction matrix W L ij Multiplying by the predicted vector v ij Seeking (b) Prediction vector v obtained from each node in the lower layer L ij The sum vector u is a linear combination of j Seeking (c) Sum vector u j Norm of |u j The activation value a is normalized by normalizing | j Seeking (d) Sum vector u j norm |u j Divide by | and then use the activation value a j Multiply by.

[0085] In addition, the activation value a j is the norm |u jis the normalization factor obtained by normalizing |. Therefore, the activation value a j can be considered as an index showing the relative output strength of each node among all nodes in the upper layer L+1. The norm used in equations (E3), (E3a), (E3b), and (4) is typically the L2 norm, which represents the vector length. In this case, the activation value a j is the output vector M L+1 j The activation value a corresponds to the vector length of j is only used in the above equations (E3) and (E4), and does not need to be output from the node. However, the activation value a j It is also possible to configure the upper layer L+1 so that it outputs

[0086] The configuration of a vector neural network is almost the same as that of a capsule network, and the vector neurons of a vector neural network correspond to the capsules of a capsule network. However, the calculations according to the above formulas (E1) to (E4) used in a vector neural network are different from the calculations used in a capsule network. The biggest difference between the two is that in a capsule network, the predicted vector v on the right side of the above formula (E2) ij are multiplied by weights, and the weights are searched by repeating dynamic routing multiple times. On the other hand, in the vector neural network of this embodiment, the output vector M is calculated by calculating the above-mentioned equations (E1) to (E4) once in order. L+1 j Therefore, there is no need to repeat dynamic routing, which has the advantage of allowing faster calculations. In addition, the vector neural network of this embodiment has the advantage that it requires less memory for calculations than a capsule network, and according to experiments by the inventors of this disclosure, it only requires about 1 / 2 to 1 / 3 of the memory required.

[0087] Vector neural networks are similar to capsule networks in that they use nodes that use vectors as input and output. Therefore, they share the advantages of using vector neurons with capsule networks. Furthermore, the multiple layers 220-260 are similar to conventional convolutional neural networks in that the higher layers represent features of larger areas and the lower layers represent features of smaller areas. Here, "feature" refers to a characteristic part contained in the input data to the neural network. Vector neural networks and capsule networks are superior to conventional convolutional neural networks in that the output vector of a node contains spatial information representing the spatial information of the feature represented by that node. That is, the vector length of a node's output vector represents the probability of the feature represented by that node, and the vector direction represents spatial information such as the direction and scale of the feature. Therefore, the vector direction of the output vectors of two nodes belonging to the same layer represents the relative positions of the respective features. Alternatively, the vector direction of the output vectors of the two nodes can be said to represent the variation of the feature. For example, for a node corresponding to the "eye" feature, the direction of the output vector can represent variations such as the narrowness of the eyes or the way they are lifted. In conventional convolutional neural networks, it is said that spatial information of features is lost due to the pooling process. As a result, vector neural networks and capsule networks have the advantage of being superior to conventional convolutional neural networks in terms of the performance of identifying input data.

[0088] The advantages of vector neural networks can also be considered as follows. In other words, the advantage of vector neural networks is that the output vectors of nodes represent the features of input data as coordinates in continuous space. Therefore, output vectors can be evaluated such that the closer the vector directions, the more similar the features. Another advantage is that even if the features contained in the input data are not covered by the training data, they can be determined by interpolation. On the other hand, conventional convolutional neural networks have the disadvantage that the features of input data cannot be represented as coordinates in continuous space due to the chaotic compression caused by the pooling process.

[0089] The outputs of each node in the ConvVN2 layer 250 and the ClassVN layer 260 are similarly determined using the above-mentioned equations (E1) to (E4), and therefore detailed explanations are omitted. The ClassVN layer 260, which is the top layer, has a resolution of 1x1 and M channels.

[0090] The output of the ClassVN layer 260 is converted into a plurality of class decision values ​​Class_1 to Class_2 for a plurality of classes. These class decision values ​​are usually normalized by a softmax function. Specifically, for example, the decision value for each class can be obtained by performing the following operation: calculating the vector length of the output vector from the output vector of each node of the ClassVN layer 260, and then normalizing the vector length of each node by a softmax function. As described above, the activation value a obtained by the above formula (E3) is j is the output vector M L+1 j The activation value a at each node in the ClassVN layer 260 is a value corresponding to the vector length of j may be output and used as the judgment value for each class.

[0091] In the above-described embodiment, a vector neural network that determines an output vector by calculating the above equations (E1) to (E4) was used as the machine learning model 200, but instead, a capsule network disclosed in U.S. Pat. No. 5,210,798 or WO 2009 / 083553 may be used.

[0092] Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0093] (1) According to a first aspect of the present disclosure, there is provided a method for extracting inappropriate, defective data from a plurality of training data sets used to train a machine learning model that classifies input data into a plurality of classes. The machine learning model is configured as a vector neural network having a plurality of vector neuron layers. The method includes: (a) inputting the plurality of training data sets into the trained machine learning model, respectively, to obtain feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the plurality of training data sets into classes; and (b) selecting target training data from the plurality of training data sets and determining whether the target training data corresponds to the defective data. The step (b) includes: (b1) selecting a reference class from the plurality of classes; (b2) calculating multiple similarities between the feature spectrum for the target training data and multiple feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the multiple similarities; and (b4) determining whether the target training data corresponds to the deficiency data based on a comparison result between the deficiency index and a threshold value. According to this method, it is possible to extract defective data from the training data using the defect index calculated according to the similarity.

[0094] (2) In the above method, the deficiency function may be a function that determines a statistically representative value of the plurality of similarities as the deficiency index. This method allows for the determination of an appropriate deficiency index.

[0095] (3) In the above method, the deficiency function may be a function that determines an average value or a maximum value of the plurality of similarities as the deficiency index. This method allows for the determination of an appropriate deficiency index.

[0096] (4) In the above method, the deficiency function may be a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index. This method allows for the determination of an appropriate deficiency index.

[0097] (5) In the above method, the step (b3) may include a step of dividing the histogram of the plurality of similarities into one or more unimodal distributions, and a step of determining, as the deficiency index, a representative value in a representative unimodal distribution selected from the one or more unimodal distributions according to predetermined selection conditions. According to this method, an appropriate deficiency index can be obtained from a histogram having multiple peaks.

[0098] (6) In the above method, the selection conditions may include a first condition that the ratio of the area of ​​one unimodal distribution to the total area of ​​the histogram is equal to or greater than an area threshold, and a second condition that the average value of the similarity is the largest among the unimodal distributions that satisfy the first condition. This method allows the selection of an appropriate unimodal distribution for determining the deficiency index.

[0099] (7) In the above method, the deficiency data may include outlier data, the reference class may be the same class as the target class to which the target learning data belongs, and step (b4) may include a step of determining that the target learning data is the outlier data if the deficiency index is equal to or less than the threshold, and determining that the target learning data is not the outlier data if the deficiency index exceeds the threshold. According to this method, outlier data can be extracted as defective data.

[0100] (8) In the above method, the deficiency data may include overlap data that approximates learning data of another class different from the class to which the deficiency data belongs, the reference class may be a class different from the target class to which the target learning data belongs, and step (b4) may include a step of determining that the target learning data is the overlap data if the deficiency index is equal to or greater than the threshold, and determining that the target learning data is not the overlap data if the deficiency index is less than the threshold. According to this method, overlapping data can be extracted as defective data.

[0101] (9) In the above method, the specific layer has a configuration in which vector neurons arranged on a plane defined by two axes, a first axis and a second axis, are arranged as multiple channels along a third axis that is oriented differently from the first and second axes. The feature spectrum may be any one of (i) a first type of feature spectrum in which multiple element values ​​of an output vector of a vector neuron at one planar position in the specific layer are arranged across the multiple channels along the third axis, (ii) a second type of feature spectrum obtained by multiplying each element value of the first type of feature spectrum by an activation value corresponding to the vector length of the output vector, and (iii) a third type of feature spectrum in which the activation values ​​at one planar position in the specific layer are arranged across the multiple channels along the third axis. According to this method, the characteristic spectrum can be easily obtained.

[0102] (10) According to a second aspect of the present disclosure, there is provided an information processing device that executes a process of extracting inappropriate defective data from a plurality of training data used for training a machine learning model that classifies input data into a plurality of classes. The information processing device includes a memory that stores a machine learning model configured as a vector neural network having a plurality of vector neuron layers, and a processor that executes calculations using the machine learning model. The processor executes the following processes: (a) inputting each of the plurality of training data into the trained machine learning model, obtaining feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the plurality of training data by class; and (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data. The process (b) includes: (b1) a process of selecting a reference class from the plurality of classes; (b2) a process of calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) a process of calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; and (b4) a process of determining whether the target training data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value.

[0103] (11) According to a third aspect of the present disclosure, there is provided a computer program that causes a processor to execute a process of extracting inappropriate deficiency data from a plurality of training data used for training a machine learning model that classifies input data into a plurality of classes. The computer program causes the processor to execute the following processes: (a) inputting each of the plurality of training data into the trained machine learning model, obtaining feature spectra obtained from the output of a specific layer of the machine learning model, and classifying the feature spectra for each of the plurality of training data into classes; and (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the deficiency data. The process (b) includes: (b1) selecting a reference class from the plurality of classes; (b2) calculating multiple similarities between the feature spectrum for the target training data and multiple feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the multiple similarities; and (b4) determining whether the target training data corresponds to the deficiency data based on a comparison result between the deficiency index and a threshold value.

[0104] The present disclosure can be realized in various forms other than those described above, such as a computer program for realizing the functions of the deficiency data extraction device, a non-transitory storage medium on which the computer program is recorded, etc. [Explanation of symbols]

[0105] 100...information processing device, 110...processor, 112...learning execution unit, 114...defect data extraction unit, 120...memory, 130...interface circuit, 140...input device, 150...display device, 200...machine learning model, 210...input layer, 220...convolutional layer, 230...primary vector neuron layer, 240...first convolutional vector neuron layer, 250...second convolutional vector neuron layer, 260...classification vector neuron layer, 280...hidden layer, 310...similarity calculation unit, 320...defect index calculation unit, 400...camera

Claims

1. A method for extracting inappropriate defective data from a plurality of training data used for training a machine learning model that classifies input data into a plurality of classes, the method comprising: The method comprises: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; Including, The step (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The method, wherein the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index.

2. 10. The method of claim 1, The step (b3) partitioning the histogram of the plurality of similarities into one or more unimodal distributions; determining, as the deficiency index, a representative value in a representative unimodal distribution selected from the one or more unimodal distributions in accordance with predetermined selection conditions; A method comprising:

3. 3. The method of claim 2, The selection condition is: a first condition that the ratio of the area of ​​one unimodal distribution to the total area of ​​the histogram is equal to or greater than an area threshold; a second condition that the average value of the similarity is the largest among the unimodal distributions that satisfy the first condition; A method comprising:

4. A method for extracting inappropriate defective data from a plurality of training data used for training a machine learning model that classifies input data into a plurality of classes, the method comprising: The method comprises: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; Including, The step (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, The characteristic spectrum is (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one plane position in the specific layer are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by an activation value corresponding to the vector length of the output vector; (iii) a third type of feature spectrum in which the activation values ​​at one planar position of the specific layer are arranged across the plurality of channels along the third axis; A method, which is one of the following:

5. 5. The method of claim 4, The method, wherein the deficiency function is a function that calculates an average or maximum value of the plurality of similarities as the deficiency index.

6. 5. The method of claim 4, The method, wherein the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index.

7. The method according to any one of claims 1 to 6, the defective data includes overlap data that is similar to learning data of another class different from the class to which the defective data belongs, the reference class is a class different from a target class to which the target training data belongs, The method, wherein step (b4) includes determining that the target learning data is the overlap data when the deficiency index is equal to or greater than the threshold, and determining that the target learning data is not the overlap data when the deficiency index is less than the threshold.

8. An information processing device that executes a process of extracting inappropriate defective data from a plurality of learning data used for training a machine learning model that classifies input data into a plurality of classes, a memory for storing a machine learning model configured as a vector neural network having a plurality of vector neuron layers; a processor that executes calculations using the machine learning model; Equipped with The processor: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; Run The process (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) a process of determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The information processing device, wherein the deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index.

9. A computer program that causes a processor to execute a process of extracting inappropriate defective data from a plurality of learning data used for training a machine learning model that classifies input data into a plurality of classes, The computer program comprises: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; causing the processor to execute The process (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) a process of determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The deficiency function is a function that determines a representative value in a histogram of the plurality of similarities as the deficiency index.

10. An information processing device that executes a process of extracting inappropriate defective data from a plurality of learning data used for training a machine learning model that classifies input data into a plurality of classes, a memory for storing a machine learning model configured as a vector neural network having a plurality of vector neuron layers; a processor that executes calculations using the machine learning model; Equipped with The processor: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; Run The process (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) a process of determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, The characteristic spectrum is (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one plane position in the specific layer are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by an activation value corresponding to the vector length of the output vector; (iii) a third type of feature spectrum in which the activation values ​​at one planar position of the specific layer are arranged across the plurality of channels along the third axis; An information processing device that is one of the above.

11. A computer program that causes a processor to execute a process of extracting inappropriate defective data from a plurality of learning data used for training a machine learning model that classifies input data into a plurality of classes, The computer program comprises: (a) inputting each of the plurality of training data into the trained machine learning model to obtain a feature spectrum obtained from an output of a specific layer of the machine learning model, and classifying the feature spectrum for each of the plurality of training data into classes; (b) selecting target training data from the plurality of training data and determining whether the target training data corresponds to the defective data; causing the processor to execute The process (b) (b1) selecting a reference class from the plurality of classes; (b2) calculating a plurality of similarities between the feature spectrum for the target training data and a plurality of the feature spectra belonging to the reference class; (b3) calculating a deficiency index for the target training data by applying a predetermined deficiency function to the plurality of similarities; (b4) a process of determining whether the target learning data corresponds to the deficiency data according to a comparison result between the deficiency index and a threshold value; Including, The specific layer has a configuration in which vector neurons arranged on a virtual plane defined by two axes, a first axis and a second axis, are arranged as a plurality of channels along a third axis in a direction different from the two axes, The characteristic spectrum is (i) a first type of feature spectrum in which a plurality of element values ​​of an output vector of a vector neuron at one plane position in the specific layer are arranged across the plurality of channels along the third axis; (ii) a second type feature spectrum obtained by multiplying each element value of the first type feature spectrum by an activation value corresponding to the vector length of the output vector; (iii) a third type of feature spectrum in which the activation values ​​at one planar position of the specific layer are arranged across the plurality of channels along the third axis; A computer program that is one of the following:

Images