Data oversampling method, data oversampling system, and data oversampling program

Abstract

To provide a data oversampling technique that can improve estimation results by a learned model using oversampled data.SOLUTION: A data oversampling method includes: a step in which a computer obtains M original data arrays of size N each consisting of binary elements; a step in which the computer generates an original data matrix of M rows and N columns based on the M original data arrays; a step in which the computer calculates an original data feature amount matrix of N rows and N columns based on a product of a transposed matrix of the original data matrix and the original data matrix; a step in which the computer executes generator matrix update processing one or more times; and a generating step in which the computer generates an output data array of size N consisting of binary data, as a solution to reduce a value of an objective function, based on an updated generator matrix.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a data oversampling method, a data oversampling system, and a data oversampling program. [Background technology]

[0002] It is well known that supervised machine learning is used to estimate or predict unknown data using existing data. In supervised machine learning, training is performed using data with correct labels as training data.

[0003] In supervised machine learning, a problem arises where learning cannot be performed correctly if there is an imbalance in the number of data points per label in the training data. Therefore, attempts are being made to solve this problem by increasing the amount of data with similar characteristics to the original data (oversampling) for labels with a small number of data points.

[0004] Oversampling is a method of correcting imbalances in data by increasing the amount of data representing a minority group. Methods of increasing the data include duplication and the creation of similar data.

[0005] Non-patent document 1 describes this problem. Non-patent document 2 also describes a method called SMOTE as an example of oversampling.

[0006] Non-patent document 3 describes annealing technology. Annealing technology, which utilizes the Ising model, is attracting attention as a technique for efficiently solving large-scale combinatorial optimization problems. Annealing technology is an approximate algorithm that can be used generally, not limited to specific computational problems, in combinatorial optimization algorithms. Its aim is not to find the exact optimal solution, but to derive a better solution in a realistic computation time.

[0007] The Ising model is a statistical mechanics model that describes the behavior of magnetic materials. It describes how the spin state is updated through the interaction of spins in a magnetic material to minimize energy, ultimately resulting in the minimum energy. In annealing technology, a combinatorial optimization problem of binary variables is mapped to the Ising model, and the solution to the original optimization problem is obtained by finding the state with the minimum energy. The Ising model can be converted to and from an unconstrained quadratic form optimization problem (QUBO), and the definitions of the Ising model and QUBO are often used interchangeably depending on the optimization problem being addressed. A device that realizes such annealing technology is called an annealing machine.

[0008] Combinatorial optimization computation using annealing techniques is a metaheuristic computation method, resulting in variability in search results. Taking advantage of this characteristic, research using annealing techniques for data sampling has recently attracted attention. One application of sampling machines is oversampling of small datasets to address the imbalance problem of training data in machine learning. [Prior art documents] [Non-patent literature]

[0009] [Non-Patent Document 1] Nathalie Japkowicz, "The class imbalance problem: Significance and strategies", Proc. of the Int'l Conf. on Artificial Intelligence, 2000. [Non-Patent Document 2] Nitesh V Chawla et al., "SMOTE: synthetic minority over-sampling technique," Journal of artificial intelligence research 16, pp. 321-357, 2002. [Non-Patent Document 3] T. Kadowaki, H. Nishimori, "Quantum annealing in the transverse ising model", PhysRevE, Vol.58, 5355-5363, 1998 [Overview of the Initiative] [Problems that the invention aims to solve]

[0010] However, conventional oversampling methods have the drawback that there is room for improvement in the estimation results obtained by trained models using oversampled data.

[0011] This invention was made to solve these problems and aims to provide a data oversampling technique that can improve the results of estimation by a trained model using oversampled data. [Means for solving the problem]

[0012] An example of a data oversampling method according to the present invention is: The computer obtains M original data arrays of size N, each consisting of binary elements. The computer generates an M-row, N-column original data matrix based on the M original data arrays, The computer calculates an N x N original data feature matrix based on the product of the transpose of the original data matrix and the original data matrix. A step in which a computer performs a generation matrix update process one or more times, wherein the generation matrix update process is: - A search step in which, based on a generation matrix of N rows and N columns consisting of real numbers, S sampling data arrays of size N consisting of binary data are generated as a solution to minimize the value of a given objective function. - A step of generating an S row N column sampling data matrix based on the S sampling data arrays, - calculating a sampling data feature matrix of N rows and N columns based on the product of the transposed matrix of the sampling data matrix and the sampling data matrix; - updating the generation matrix based on the difference between the original data feature matrix and the sampling data feature matrix; performing the generation matrix update process including the above steps one or more times; a generation step in which a computer generates an output data array of size N consisting of binary data as a solution for reducing the value of the objective function based on the updated generation matrix; comprising.

[0013] In one example, the search step and the generation step are performed using annealing processing. In one example, a step in which a computer generates a one-hot vector in which elements at different positions become 1 according to the value for each multi-valued element of the pre-replacement data array including multi-valued elements; a step in which a computer generates the original data array by replacing each multi-valued element in the pre-replacement data array with the corresponding one-hot vector; a step in which a computer generates different replacement values according to the positions of the elements with a value of 1 in the element column corresponding to the one-hot vector in the output data array; a step in which a computer generates a post-replacement output data array including multi-valued elements by replacing the one-hot vector in the output data array with the corresponding replacement value; further comprising.

[0014] An example of the data oversampling system according to the present invention executes the above method.

[0015] An example of the data oversampling program according to the present invention causes a computer to execute the above method.

Effect of the Invention

[0016] According to the present invention, the results of estimation by a trained model using oversampled data can be improved. For example, since it is possible to increase the amount of data that has a similar distribution to the original data to be oversampled, the problem of data imbalance in supervised machine learning can be solved. [Brief explanation of the drawing]

[0017] [Figure 1] An example configuration of the data oversampling system 10 according to Embodiment 1. [Figure 2] A flowchart illustrating an example of the operation of the data oversampling system 10 according to Embodiment 1. [Figure 3] An example of the original data matrix. [Figure 4] An example of an original data feature matrix obtained based on the original data matrix in Figure 3. [Figure 5] A flowchart illustrating step S5 in more detail in Figure 2. [Figure 6] An example of the execution result of data oversampling by the data oversampling system 10 according to Embodiment 1. [Figure 7] Another example of the execution result of data oversampling by the data oversampling system 10 according to Embodiment 1. [Figure 8] A flowchart illustrating an example of the operation of the data oversampling system according to Embodiment 2. [Figure 9] A specific example of the process in step S102. [Figure 10] Another specific example of the process in step S102. [Figure 11] A specific example of the process in step S105. [Figure 12] Another specific example of the process in step S105. [Figure 13] The data items used in the example execution result of Embodiment 2. [Figure 14] An example of the execution result when the processing according to Embodiment 2 is performed using the data array in Figure 13. [Figure 15] Examples of initial values ​​for the generator matrix available in Embodiment 2. [Modes for carrying out the invention]

[0018] One example of the present invention involves determining the parameters of the annealing process (e.g., the QUBO matrix) so that the distribution of the data output by annealing becomes close to the distribution of the original data that was oversampled.

[0019] After a sufficient number of parameter update processes, the data output by annealing will have a distribution similar to the data targeted for oversampling, and can therefore be used as oversampled data.

[0020] Hereinafter, embodiments of the present invention will be described based on the attached drawings. [Embodiment 1] Figure 1 shows an example configuration of a data oversampling system 10 according to Embodiment 1. The data oversampling system 10 performs data oversampling by executing the data oversampling method described herein. The data oversampling system 10 can be configured using, for example, one or more computers.

[0021] The data oversampling system 10 has a hardware configuration similar to that of a known computer, and includes, for example, arithmetic means 11 and storage means 12. The arithmetic means 11 includes, for example, a processor, and the storage means 12 includes, for example, storage media such as semiconductor memory devices and magnetic disk devices. Some or all of the storage media may be non-transitory storage media.

[0022] Furthermore, a computer may be equipped with input / output means. These input / output means may include, for example, input devices such as a keyboard and mouse, output devices such as a display and printer, and communication devices such as a network interface.

[0023] The storage means 12 may store a data oversampling program. The computer may perform the functions described in this embodiment by having the processor execute this data oversampling program. That is, the data oversampling system 10 may be realized by having the computer execute the data oversampling method described herein.

[0024] Figure 2 shows a flowchart illustrating an example of the operation of the data oversampling system 10 according to Embodiment 1. This flowchart represents an example of a data oversampling method.

[0025] First, the data oversampling system 10 obtains M original data arrays of size N, each consisting of binary elements, as the data to be oversampled (step S1). Each element of the original data array is, for example, {0,1}.

[0026] Next, the data oversampling system 10 generates an M-row, N-column original data matrix based on these M original data arrays (step S2). For example, the original data matrix can be generated by treating each original data array as a row vector and concatenating each row vector.

[0027] Figure 3 shows an example of an original data matrix. In this example, the original data matrix shown is generated from six original data arrays of size 5: original data array

[10011] , original data array

[0101] , original data array

[11010] , original data array

[1100] , original data array

[11000] , and original data array

[0011] .

[0028] Next, the data oversampling system 10 calculates an N x N matrix based on the product of the transpose of the original data matrix and the original data matrix (step S3). This matrix represents the features of the original data and will be referred to as the original data feature matrix below.

[0029] Figure 4 shows an example of the original data feature matrix obtained based on the original data matrix in Figure 3. In this example, the product of the transpose of the original data matrix and the original data matrix is ​​calculated, and then each element is divided by M.

[0030] The original data feature matrix represents the distribution of elements in the original data array, and the element in the i-th row and j-th column represents the probability that both the i-th element and the j-th element in the original data array are 1. In the example in Figure 4, both the element in the 1st row and 3rd column and the element in the 3rd row and 1st column are 0, but from the six original data arrays shown in Figure 3, it can be confirmed that there are no original data arrays in which both the 1st and 3rd elements are 1 at the same time.

[0031] Next, the data oversampling system 10 determines the initial values ​​of the matrix (generation matrix) for generating the sampled data array (step S4). The generation matrix is ​​an N x N matrix consisting of real numbers. The initial values ​​can be appropriately determined by a person skilled in the art, but for example, all elements can be set to 0. If the subsequent step S5 is performed by an annealing process, this generation matrix becomes the QUBO matrix as a parameter for the annealing process.

[0032] Next, the data oversampling system 10 performs a predetermined generation matrix update process one or more times (step S5).

[0033] Figure 5 shows a flowchart illustrating the details of step S5 in Figure 2. In the generation matrix update process, the data oversampling system 10 first generates S sampling data arrays of size N consisting of binary data (step S51, search step). The sampling data arrays have the same structure as the original data arrays. That is, the sampling data arrays are arrays of size N consisting of binary data, where each element is, for example, {0,1}.

[0034] The sampled data array is generated as a solution to minimize the value of a given objective function. More specifically, the sampled data array is generated as a solution that minimizes the value of the objective function. This solution may be an exact solution that precisely minimizes the value of the objective function, but generally it is an approximate solution that approximately minimizes the value of the objective function. The generation matrix update process including step S51 can be implemented, for example, as an annealing process.

[0035] The input to the objective function includes a generator matrix, and the output includes a sampled data array. The content of the objective function can be appropriately determined by a person skilled in the art based on prior art. As a specific example, the following objective function E(X) can be used. E(X) = Σ i Σ j Q ij X i X j However, Q is the generator matrix, X is the sampled data array, and i and j are the element indices.

[0036] When the generation matrix update process is performed by annealing, the element Q of a certain QUBO matrix Q mn This is the element X of the sampling data array X. m and X n This affects the probability that both values ​​are 1. Therefore, adjusting the QUBO matrix is ​​equivalent to adjusting the distribution of the sampled data sequence.

[0037] Next, the data oversampling system 10 generates an S-row, N-column sampling data matrix based on the S sampling data arrays generated in step S51 (step S52). For example, similar to step S2, the sampling data matrix can be generated by treating each sampling data array as a row vector and combining the row vectors.

[0038] Next, the data oversampling system 10 calculates an N-by-N matrix based on the product of the transposed matrix of the sampling data matrix and the sampling data matrix (step S53). In an example of the process, each element of the matrix is divided by S (i.e., the number of pieces of the generated sampling data array) during the calculation. This matrix is a matrix representing the characteristics of the sampling data, and hereinafter is referred to as the sampling data feature matrix. The structure of the sampling data feature matrix can be the same as the example of the original data feature matrix shown in FIG. 4.

[0039] The sampling data feature matrix represents the distribution of elements in the sampling data array, and the element in the i-th row and j-th column represents the probability that the i-th element and the j-th element both become 1 in the sampling data array. That is, an element P of a certain sampling data matrix P mn is the element Y of the sampling data array Y m and Y n both represent the probability of becoming 1.

[0040] Next, the data oversampling system 10 updates the generation matrix based on the difference between the original data feature matrix and the sampling data feature matrix (step S54). A specific calculation example is as follows. Q k+1 =Q k -η(Zmat - Xmat) where Q k and Q k+1 are the generation matrices after being updated in the k-th and k + 1-th executions of step S54, respectively, η is the learning rate (e.g., a positive constant), Zmat is the original data feature matrix, and Xmat is the sampling data feature matrix.

[0041] The data oversampling system 10 repeatedly executes steps S51 to S54 until a predetermined criterion is met (at least once). As an example of a criterion, steps S51 to S54 are repeatedly executed until the difference between Zmat and Xmat (loss function) becomes smaller than a predetermined threshold. The difference between Zmat and Xmat can be expressed, for example, by the mean squared error (hereinafter sometimes abbreviated as "MSE") based on each element. Alternatively, as another example of a criterion, steps S51 to S54 may be repeated a predetermined number of times. Through this process, the generation matrix update process can be repeated until the value of the loss function becomes sufficiently small. In this way, step S5 in Figure 2 is completed.

[0042] After step S5, the data oversampling system 10 generates and outputs an output data array based on the updated generation matrix (for example, the generation matrix at the end of step S5) (step S6, generation step). The output data array, like the sampled data array, is generated as a solution to minimize the value of a predetermined objective function. As a more specific example, the output data array is generated as a solution that minimizes the value of the objective function. This solution may be an exact solution that precisely minimizes the value of the objective function, but generally it is an approximate solution that approximately minimizes the value of the objective function. The specific calculation process in step S6 can be the same as in step S51.

[0043] The output data array has the same structure as the original data array and the sampled data array. That is, the output data array is an array of size N consisting of binary data, where each element is, for example, {0,1}. One or more output data arrays are produced.

[0044] This process generates an output data array with a distribution similar to the original data array, enabling oversampling of the original data array. Therefore, it can solve the problem of data imbalance in supervised machine learning.

[0045] In this embodiment, annealing is used to search for a solution, and in particular, steps S51 (search step) and S6 (generation step) are executed using annealing. As an alternative, any process that satisfies the following conditions 1 and 2 can be used instead of annealing. -Condition 1: Given an N x N matrix Q representing parameters as input, search for a solution X that minimizes an arbitrary objective function E(X). Each element of Q is a real number, and X is an N-dimensional array consisting of binary data {0,1}. -Condition 2: The search for solution X is a heuristic search. Note that the search is not necessarily limited to obtaining an exact solution. If an approximate solution is obtained, there will be variability in the results.

[0046] Below, we will describe a specific example of execution results using annealing processing as a performance evaluation of the data oversampling system 10 according to this embodiment.

[0047] Figure 6 shows an example of the results of data oversampling performed by the data oversampling system 10 according to Embodiment 1. Each map in Figure 6(a) to (f) represents a feature matrix. In each map in Figure 6, darker gray cells indicate that the value of the element corresponding to that cell is large.

[0048] Figure 6(a) shows an example of the original data feature matrix, representing the original data feature matrix, or Zmat, in Figure 4. This matrix is ​​the correct answer in the annealing process.

[0049] Figures 6(b), (c), (d), (e), and (f) show examples of the sampled data feature matrix after the generative matrix update process (step S5) has been executed 1, 100, 200, 500, and 1000 times, respectively, and represent Xmat. It can be seen that as the number of generative matrix update processes increases, the distribution approaches Zmat in Figure 6(a).

[0050] Figure 6(f), which shows the result after 1000 iterations, represents the distribution of the output data array. Comparing the correct answer in Figure 6(a) with the result in Figure 6(f), we can see that there is almost no difference. In other words, in this example, by repeating the generation matrix update process 1000 times, it is possible to generate an output data array that has almost the same distribution as the original data array.

[0051] Figure 7 shows another example of the results of data oversampling performed by the data oversampling system 10 according to Embodiment 1. This example uses the well-known MNIST image. An MNIST image is an image representing a handwritten digit, associated with a label representing that digit, with 1000 data points for each digit from 0 to 9. In this embodiment, data was used in which each pixel of each image was binarized to either white or black.

[0052] Of the digits 0 through 9, the data for digits 0 through 8 was kept at 1000, while the data for digit 9 was reduced to 100 to generate imbalanced data. This total of 9100 imbalanced data points was classified using the well-known classification method, LightGBM, and its accuracy was evaluated. Accuracy is expressed by Accuracy, Recall, Precision, and F-score. Since the definition and calculation methods of such accuracy are well-known, an explanation is omitted.

[0053] Figure 7(a) shows the results when training is performed using the imbalanced data as is, with an F-score of 0.738 for the digit 9, which is lower than the F-scores for the digits 0 through 8.

[0054] Figure 7(b) shows an example of a conventional method where data for the digit 9 is oversampled by randomly duplicating it, resulting in 1000 data points for training. The F-score for the digit 9 has improved to 0.824.

[0055] Figure 7(c) shows another example of the conventional method, where the data for the digit 9 was oversampled using SMOTE, resulting in 1000 data points for training. In this example, two original data points for the digit 9 were randomly selected, and new data was generated at their midpoint. This process was repeated. The F-score for the digit 9 improved to 0.827.

[0056] Figure 7(d) shows the result when the data for the digit 9 is oversampled using this embodiment, and the number of data points is increased to 1000 before training. The F-score for the digit 9 has improved to 0.907, which is particularly higher than the conventional method shown in Figures 7(b) and (c).

[0057] As described above, the data oversampling method, data oversampling system, and data oversampling program according to this embodiment enable appropriate oversampling, thereby effectively resolving imbalances in the number of data points in the original data array. This allows for more appropriate learning and improves the estimation results of the trained model using the oversampled data. In this way, the problem of data number imbalance in supervised machine learning can be solved.

[0058] [Embodiment 2] Embodiment 2 modifies Embodiment 1 to enable the handling of multi-level data. Embodiment 2 will be described below, but parts common to Embodiment 1 may be omitted from the explanation.

[0059] Figure 8 shows a flowchart illustrating an example of the operation of the data oversampling system according to Embodiment 2. This flowchart represents an example of a data oversampling method. Note that the process in step 104 of Figure 8 can be performed by steps S1 to S6 of Figure 2 (Embodiment 1).

[0060] First, the data oversampling system obtains an array containing multi-valued elements as the data to be oversampled (step S101). The multi-valued elements may be data with three or more values, or quantitative data. This array is the array that will be subjected to the replacement process described later, and will be referred to as the "data array before replacement" below.

[0061] Next, the data oversampling system generates a one-hot vector for each multi-valued element of the pre-replacement data array, where elements at different positions become 1 depending on their value (step S102).

[0062] Figure 9 shows a specific example of the processing in step S102. In this example, the multi-valued element is a three-valued data, for example, a categorical variable that represents which of the three categories the element corresponds to. If the value of the categorical variable is 0, a one-hot vector [1,0,0] is generated with the first element being 1. If the value of the categorical variable is 1, a one-hot vector [0,1,0] is generated with the second element being 1. If the value of the categorical variable is 2, a one-hot vector [0,0,1] is generated with the third element being 1. In this way, one-hot vectors are generated based on the multi-valued element.

[0063] Figure 10 shows another specific example of the processing in step S102. In this example, the multi-valued elements are quantitative data between 0 and 1.5. First, it is determined which of the different intervals each element belongs to. In the example in Figure 10, it is determined which of the following intervals it belongs to: the first interval "0 or more and less than 0.5", the second interval "0.5 or more and less than 1", and the third interval "1 or more (in this case, 1.5 or less)". Each interval does not overlap with the others and covers the entire range of the quantitative data.

[0064] Next, the value of the categorical variable is determined according to the result of the judgment. For example, if the value of the quantitative variable is 0.1, it belongs to the first interval, so the value of the categorical variable is 0. If the value of the quantitative variable is 0.7, it belongs to the second interval, so the value of the categorical variable is 1. If the value of the quantitative variable is 1.4, it belongs to the third interval, so the value of the categorical variable is 3. The processing after the value of the categorical variable is determined is the same as in the example in Figure 9.

[0065] After step S102, the data oversampling system generates a new data array consisting of binary elements by replacing each multi-valued element in the pre-replacement data array with the corresponding one-hot vector (step S103). The data array generated here corresponds to the original data array in Embodiment 1.

[0066] Next, the data oversampling system performs the same data oversampling process as in Embodiment 1 on the data array generated in step S103 (step S104). This generates an output data array obtained by oversampling.

[0067] Next, the data oversampling system generates different substitution values ​​in the sequence of elements in the output data array corresponding to the one-hot vector described above, depending on the position of the element whose value is 1 (step S105).

[0068] Figure 11 shows a specific example of the process in step S105. In this example, the multi-valued element before substitution is a 3-valued data (categorical variable), meaning that one one-hot vector corresponds to a sequence of three elements. If the first element of the one-hot vector is 1, the value of the categorical variable to be substituted becomes 0; if the second element of the one-hot vector is 1, the value of the categorical variable to be substituted becomes 1; and if the third element of the one-hot vector is 1, the value of the categorical variable to be substituted becomes 2.

[0069] The processing in step S105 is the reverse of the processing in step S103 described above. Which part of the output data array corresponds to the one-hot vector (i.e., which element and how many consecutive elements should constitute the one-hot vector) can be determined based on the processing in step S103 described above.

[0070] Furthermore, the handling of cases where the sequence of elements is not a one-hot vector (for example, when two or more elements are 1) can be appropriately determined by a person skilled in the art. For example, a function to convert any binary vector into a one-hot vector may be defined in advance.

[0071] Figure 12 shows another specific example of the process in step S105. In this example, the multivalued elements before substitution are quantitative data between 0 and 1.5, and in step S103, one quantitative data is substituted with a three-dimensional one-hot vector. If the first element of the one-hot vector is 1, the value of the categorical variable becomes 0, and the corresponding interval is the first interval "0 or more and less than 0.5". If the second element of the one-hot vector is 1, the value of the categorical variable becomes 1, and the corresponding interval is the first interval "0.5 or more and less than 1". If the third element of the one-hot vector is 1, the value of the categorical variable becomes 2, and the corresponding interval is the first interval "1 or more (in this case, 1.5 or less)".

[0072] Then, a single value within the relevant interval is generated as a replacement value for the quantitative variable. The generation of a value within the interval is performed, for example, by randomly selecting a value within that interval. For example, if the value of the categorical variable is 1, a replacement value between 0 and 0.5 (in the example in Figure 12) is randomly generated (0.4 is generated). If the value of the categorical variable is 2, a replacement value between 0.5 and 1 (0.7 is generated) is randomly generated (in the example in Figure 12) is randomly generated (1 and 1.5 is generated) (in the example in Figure 12) is randomly generated.

[0073] After step S105, the data oversampling system generates a new sequence by replacing the one-hot vectors in the output data sequence with the corresponding substitution values ​​(step S106). The sequence generated here is a sequence containing multivalued elements (which may also be quantitative data), and is hereafter referred to as the "substituted output data sequence".

[0074] The substituted output data array contains a multi-valued element corresponding to each of the multi-valued elements in the pre-substituted data array, and can be said to have the same structure as the pre-substituted data array. Thus, according to the data oversampling method, data oversampling system, and data oversampling program of Embodiment 2, appropriate oversampling can be performed even on data arrays containing multi-valued elements.

[0075] The following describes an example of performing data oversampling using the data oversampling system according to Embodiment 2.

[0076] Figure 13 shows the data items used in the example execution result of Embodiment 2. This data is a dataset representing the quality of wine provided by imbalanced-learn. The dataset contains 4715 data sequences for low-quality wines and 183 data sequences for high-quality wines, for a total of 4898 data sequences.

[0077] The level of quality is represented by the "quality" item in the bottom row of Figure 13. "Quality" is a binary data value of {0,1}, where 0 indicates low wine quality and 1 indicates high wine quality. As mentioned above, the ratio of data arrays where "quality" is 0 to data arrays where "quality" is 1 is 4715:183 ≈ 26:1, indicating unbalanced data.

[0078] Figure 14 shows an example of the execution result when the processing according to Embodiment 2 was performed using the data array in Figure 13. The value of quality was used as a label, and learning was performed to estimate quality from the values ​​of other variables. The ratio of training data to test data during verification was approximately 7:3. The "imbalanced state" is the case when the above-mentioned imbalanced data was used as is for training, and the F-score was 0.293.

[0079] "Conventional Example: Random," "Conventional Example: SMOTE," and "Embodiment 2" all describe the case where data with quality 1 was oversampled to 4715 entries, which is the same number as data with quality 0, before training.

[0080] "Conventional Example: Random" shows the case where oversampling was performed by random replication, similar to Figure 7(b). In this case, the F-score improved to 0.338. "Conventional Example: SMOTE" shows the case where oversampling was performed using SMOTE, similar to Figure 7(c). In this case, the F-score improved to 0.373.

[0081] "Embodiment 2" describes the case where oversampling is performed according to this embodiment. In this example, each variable representing quantitative data was fitted into 50 intervals, that is, each variable was converted into a 50-dimensional one-hot vector. Since there are 11 variables, the original data array to be oversampled became an array of size 550. In the results according to this embodiment, the F-score was improved to 0.419, which is particularly higher than any of the conventional methods. Furthermore, the recall was also higher than any of the conventional methods.

[0082] Thus, according to the data oversampling method, data oversampling system, and data oversampling program of this embodiment, appropriate oversampling can be performed even on data containing multi-valued elements. In this way, the problem of data number imbalance in supervised machine learning can be solved.

[0083] In Embodiment 2, to more reliably ensure that the values ​​of elements corresponding to one-hot vectors in the original data array are output as one-hot vectors in the output data array, a specific matrix can be used as the initial value of the generation matrix.

[0084] Figure 15 shows an example of initial values ​​for the generator matrix available in Embodiment 2. This example is for the case where the data array to be replaced consists of two variables, each represented by a three-dimensional one-hot vector. Since the original data array has 6 dimensions, the generator matrix (QUBO matrix) has a size of 6 × 6. In the original data array and the output data array, the first to third elements correspond to one variable, and the fourth to sixth elements correspond to the other variable.

[0085] In this example, if, for example, multiple elements (1st to 3rd) in the output data array are simultaneously 1, the conditions for a one-hot vector are not met. In other words, for the 1st to 3rd elements to constitute a one-hot vector, it is necessary to suppress the generation of an output data array in which multiple of these elements are simultaneously 1. The same applies to the 4th to 6th elements (and so on).

[0086] To achieve this, a generator matrix can be used such that the value of the objective function E(X) increases when multiple elements of the first to third elements of the output data array are simultaneously 1. An example of an initial value that gives the generator matrix this property is shown in Figure 15.

[0087] In this example, if multiple elements from the first to third positions of the output data array (or sampling data array) are simultaneously 1, the value of the objective function E(X) increases by 100, becoming extremely large. Therefore, the generation of such arrays can be suppressed. By setting the initial value of the generation matrix to such a matrix, one-hot vectors can be appropriately generated in the final output data array as well.

[0088] In the example in Figure 15, the matrix elements are set to 0 or 100, but the value "100" can be any other positive value. It is preferable to use a value that has a significant impact on the evaluation of the objective function E(X).

[0089] Thus, in Embodiment 2, the initial value of the generator matrix can be set to increase the value of the objective function when multiple elements that constitute a single one-hot vector in the original data array are simultaneously 1.

[0090] Those skilled in the art may optionally add, modify, or delete components within the scope of the present invention in each of the embodiments described above. For example, the data oversampling system 10 may consist of multiple computers connected via a communication network. [Explanation of Symbols]

[0091] 10…Data oversampling system 11...Arithmetic means 12...Memory means

Claims

1. A data oversampling method, The aforementioned method, The computer obtains M original data arrays of size N, each consisting of binary elements. The computer generates an M-row, N-column original data matrix based on the M original data arrays, The computer calculates an N x N original data feature matrix based on the product of the transpose of the original data matrix and the original data matrix. A step in which a computer performs a generation matrix update process one or more times, wherein the generation matrix update process is: - A search step in which, based on a generation matrix consisting of N x N real elements, S samples of size N consisting of binary data are generated as a solution to minimize the value of a predetermined objective function. - A step of generating an S row N column sampling data matrix based on the S sampling data arrays, - A step of calculating an N x N sampling data feature matrix based on the product of the transpose matrix of the sampling data matrix and the sampling data matrix, - A step of updating the generation matrix based on the difference between the original data feature matrix and the sampled data feature matrix, The steps include performing a generation matrix update process one or more times, The generation step involves a computer generating an output data array of size N consisting of binary data as a solution to reduce the value of the objective function, based on the updated generation matrix. A data oversampling method comprising the following features.

2. The method according to claim 1, wherein the search step and the generation step are performed using an annealing process.

3. The computer generates a one-hot vector for each multi-valued element in the pre-replacement data array, where the elements at different positions become 1 depending on the value of the multi-valued element. The computer generates the original data array by replacing each multi-valued element in the pre-replacement data array with the corresponding one-hot vector, The computer generates different substitution values ​​in the sequence of elements in the output data array corresponding to the one-hot vector, depending on the position of the element whose value is 1. The computer generates a substituted output data array containing multi-valued elements by replacing the one-hot vectors in the output data array with the corresponding substitution values, The method according to claim 1, further comprising:

4. A data oversampling system that performs the method according to any one of claims 1 to 3.

5. A data oversampling program that causes a computer to perform the method according to any one of claims 1 to 3.

Images