Invention title: Information processing device, information processing method, and program

Abstract

An artificial case generation device, wherein an input means acquires an actual case composed of a feature quantity. An artificial case generation means generates a plurality of artificial cases from the actual case. An artificial case selection means selects, from the plurality of generated artificial cases, an artificial case in which prediction by a machine learning model results in uncertainty. An output means outputs the selected artificial case.

Description

[Technical field]

[0001] This disclosure relates to creating training sets for use in machine learning. [Background technology]

[0002] When the number of training cases used in machine learning is insufficient, artificially generated cases (hereinafter referred to as "artificial cases") may be used as training cases. For example, Non-Patent Document 1 discloses a method for generating artificial cases similar to real cases close to the decision boundary. Non-Patent Documents 2 and 3 disclose methods for generating artificial cases. [Prior art documents] [Patent documents]

[0003] [Non-Patent Document 1] Ertekin (2013). Adaptive oversampling for imbalanced data classification. In Information Sciences and Systems 2013 - Proceedings of the 28th International Symposium on Computer and Information Sciences (ISCIS), pp. 261-269. [Non-Patent Document 2] Bucilua, C., Caruana, R. and Niculescu-Mizil, A.: Model Compression, Proc. ACM SIGKDD, pp. 535-541 (2006). [Non-Patent Document 3] Chawla, NV, Bowyer, KW, Hall, LO, and Kegelmeyer, WP: SMOTE: Synthetic minority over-sampling technique, Journal of Artificial Intelligent Research, 16, 321-357 (2002). Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the above-mentioned techniques, the generated artificial examples do not necessarily contribute to improving the predictive performance of the machine learning model.

[0005] One objective of the present disclosure is to provide an information processing device capable of generating artificial examples that contribute to improving the predictive performance of a machine learning model. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided an information processing device, An input means for acquiring actual examples consisting of feature quantities; an artificial example generating means for generating a plurality of artificial examples from the real examples; an artificial example selection means for selecting an artificial example for which a prediction of the machine learning model is uncertain from the plurality of generated artificial examples; an output means for outputting the selected artificial examples; Equipped with The artificial example selection means selects a plurality of artificial examples such that the real examples existing in close proximity in the feature space are different.

[0007] In another aspect of the present disclosure, a computer-implemented information processing method includes: Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected such that the real examples existing nearby in the feature space are different; The selected synthetic examples are output.

[0008] In yet another aspect of the disclosure, a program includes: Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected such that the real examples existing nearby in the feature space are different; The computer is caused to execute a process of outputting the selected synthetic examples. Effect of the Invention

[0009] According to the present disclosure, it is possible to generate artificial examples that contribute to improving the predictive performance of a machine learning model. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining a basic method for generating artificial examples. [Diagram 2] FIG. 1 is a diagram for explaining a method for generating artificial examples according to an embodiment. [Diagram 3] 11A and 11B are diagrams illustrating the effect of the present embodiment in comparison with the basic method. [Figure 4] 1 is a block diagram showing a hardware configuration of an artificial case generation device according to a first embodiment. FIG. [Diagram 5] 1 is a block diagram showing a functional configuration of an artificial case generation device according to a first embodiment. [Figure 6] FIG. 10 is a diagram for explaining an example of a method for selecting an artificial example. [Figure 7] FIG. 11 is a diagram for explaining another example of a method for selecting artificial examples. [Figure 8] FIG. 11 is a diagram for explaining another example of a method for selecting artificial examples. [Figure 9]This is a schematic diagram of a Query by Committee, an example of active learning. [Figure 10] 1 shows a schematic diagram of how active learning is used to select real examples. [Figure 11] 13 is a flowchart of an artificial example generation process. [Figure 12] FIG. 11 is a block diagram showing a functional configuration of an information processing apparatus according to a second embodiment. [Figure 13] 10 is a flowchart of a process performed by an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the drawings. <Explanation of the principle> The principle of the method according to the embodiment will be described below. (Basic method) First, an example of a method for creating training examples used in machine learning will be described as a basic method. In machine learning, the accuracy of the resulting machine learning model can be improved by adding not only real examples that have actually been observed, but also artificial examples that are similar to the real examples to the training examples. However, it is difficult to efficiently improve the accuracy of the machine learning model by randomly adding artificial examples. Therefore, in the basic method, a real example for which the prediction of the machine learning model is uncertain, i.e., a real example that is difficult to predict, is selected, and multiple artificial examples similar to the real example are generated and added to the training examples. By repeating this process, the number of training examples is increased, and the prediction accuracy of the machine learning model is improved.

[0012] Figure 1(A) is a diagram that explains the basic method. Assume that a support vector machine (SVM) is used as a machine learning model to perform two-class classification. Figure 1(A) is a diagram in which cases are arranged in a feature space. As shown in the figure, each real case is classified into classes C1 and C2 using a decision boundary. Here, real cases that are close to the decision boundary in the feature space can be considered as cases with uncertain predictions.

[0013] The basic method first acquires a real case close to the decision space, and generates a predetermined number (v) of artificial cases similar to the acquired real case. In the example of FIG. 1(A), a real case 80 close to the decision boundary is acquired as a real case with an uncertain prediction, and artificial cases 80a to 80c similar to the real case 80 are generated. The artificial cases are generated by combining the real case with an uncertain prediction with other real cases that are similar to it. For example, the artificial cases can be generated using the following formula:

[0014]

number

[0015] Next, the basic method adds the generated v artificial examples to the training examples to reconstruct the SVM. The basic method then obtains real examples with uncertain predictions based on the reconstructed SVM and generates similar artificial examples. After repeating this process a certain number of times, the basic method outputs the generated artificial examples.

[0016] (Issues with the basic method) However, the artificial examples obtained by the above basic method do not necessarily improve the prediction accuracy of the machine learning model. This is because the basic method has the following two main problems.

[0017] The first problem is that artificial examples generated from uncertain real examples are not necessarily uncertain in the same way. FIG. 1(B) shows an example of artificial examples generated using the basic method. In the example of FIG. 1(B), a real example 80 close to the decision boundary is adopted as a real example whose prediction is uncertain, and five artificial examples similar to this real example 80 are generated. Of these, the artificial example 80d is close to the decision boundary and is considered to be an uncertain example like the real example 80. However, the artificial example 80e and the like are far from the decision boundary in the feature space and are not necessarily uncertain examples. Such artificial examples do not contribute to improving the prediction performance of the machine learning model.

[0018] The second problem is that using multiple artificial examples created from the same real example as training examples results in redundancy. Since v artificial examples generated from the same real example by the basic method are similar to each other, the larger the number v of artificial examples is, the more similar artificial examples are added to the training examples, which contributes less to improving prediction performance. In addition, the addition of similar artificial examples may cause the distribution of training examples to deviate from the distribution of the original real examples, which may have a negative impact on prediction accuracy. In this regard, the second problem can be suppressed by reducing the number v of artificial examples, but this will exacerbate the first problem mentioned above. In other words, if the number v of artificial examples is large, the possibility of accidentally adding good artificial examples increases, but if the number v is small, the possibility of only artificial examples that do not contribute to improving performance being added.

[0019] (Method of the embodiment) In view of the above problems, the method of the embodiment executes the following process. (Process 1) Select real cases in some way to generate multiple artificial cases. (Process 2) From the generated artificial examples, artificial examples with uncertain predictions are selected and added as training examples.

[0020] FIG. 2 is a diagram for explaining the method of the embodiment. FIG. 2 is a diagram in which cases are arranged in a feature space, similar to FIGS. 1(A) and (B). In the example of FIG. 2, the method of the embodiment selects a real case 80 and generates five artificial cases based on the real case 80. Next, the method of the embodiment excludes artificial cases far from the decision boundary (artificial cases within a rectangle 81) from the five generated artificial cases, and adopts only an artificial case 80d close to the decision boundary. That is, the artificial cases within the rectangle 81 are excluded because their predictions cannot necessarily be said to be uncertain, and the artificial case 80d close to the decision boundary is adopted as an example whose prediction is uncertain.

[0021] According to this method, artificial cases with less uncertain predictions are not added to the training cases, and only artificial cases with less uncertain predictions are added to the training cases. This solves the above problem 1. Furthermore, by excluding artificial cases with less uncertain predictions, similar artificial cases are not added to the training cases, and the above problem 2 is solved. Generally, artificial cases are generated by combining cases, so the cost of generating artificial cases is low. In contrast, the computational cost of machine learning due to an increase in training cases is high. Therefore, as in the method of the embodiment, the computational cost of machine learning is reduced and it is more efficient to first generate a large number of artificial cases and then select only good cases to add to the training cases.

[0022] (Effects of the embodiment) Fig. 3 is an explanatory diagram of the effect of this embodiment compared with the basic method. Fig. 3(A) shows examples generated by the basic method, and Fig. 3(B) shows examples generated by the method of the embodiment. In the basic method, after selecting a real example with an uncertain prediction, multiple artificial examples are repeatedly generated from that example. For this reason, in the basic method, as shown in Fig. 3(A), there is a tendency for an excessive number of artificial examples to be generated in similar locations in the feature space.

[0023] In contrast, the method of the embodiment selects artificial examples with uncertain predictions from the generated artificial examples, so that, as shown in FIG. 3B, cases are not excessively generated in similar locations in the feature space, and cases can be added to locations where the predictions of the machine learning model are uncertain. Therefore, it is possible to generate artificial examples that improve the prediction accuracy of the model from a small number of real cases. As a result, it is also possible to generate artificial examples that maintain the distribution of the original real cases and efficiently improve the prediction accuracy of the model.

[0024] First Embodiment Next, a description will be given of the artificial example generation device 100 according to the first embodiment. The artificial example generation device 100 generates artificial examples to be added to training examples based on real examples.

[0025] [Hardware configuration] 4 is a block diagram showing a hardware configuration of the artificial case generation device according to the first embodiment. As shown in the figure, the artificial case generation device 100 includes an interface (I / F) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

[0026] The interface 11 inputs and outputs data to and from an external device. Specifically, the interface 11 acquires actual cases from the outside.

[0027] The processor 12 is a computer such as a CPU (Central Processing Unit) and controls the entire artificial case generation device 100 by executing a program prepared in advance. The processor 12 may be a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array). The processor 12 executes an artificial case generation process, which will be described later.

[0028] The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 is also used as a working memory while the processor 12 is executing various processes.

[0029] The recording medium 14 is a non-volatile, non-transient recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the artificial case generation device 100. The recording medium 14 records various programs executed by the processor 12. When the artificial case generation device 100 executes various processes, the programs recorded on the recording medium 14 are loaded into the memory 13 and executed by the processor 12. The DB 15 stores real cases input via the interface 11 and artificial cases generated based on the real cases.

[0030] [Function configuration] 5 is a block diagram showing the functional configuration of the artificial case generation device 100 according to the first embodiment. The artificial case generation device 100 includes an input unit 21, an artificial case generation unit 22, an artificial case selection unit 23, and an output unit 24.

[0031] The input unit 21 acquires a plurality of real cases and outputs them to the artificial case generation unit 22. The artificial case generation unit 22 selects a real case from the input plurality of real cases by some method. The method of selecting a real case will be described later. Then, the artificial case generation unit 22 generates a plurality of artificial cases using the selected real cases and outputs them to the artificial case selection unit 23. Note that the processing executed by the artificial case generation unit 22 corresponds to the process 1 described above.

[0032] The artificial example selection unit 23 selects an artificial example with an uncertain prediction from the multiple generated artificial examples, and outputs the selected artificial example to the output unit 24. A method for selecting an artificial example with an uncertain prediction will be described in detail later. The process executed by the artificial example selection unit 23 corresponds to the above-mentioned process 2. Then, the output unit 24 adds the input artificial example to the training examples used for training the machine learning model.

[0033] [Artificial example selection section] Next, a detailed description will be given of the artificial example selection unit 23. The artificial example selection unit 23 selects artificial examples to be added as training examples from the plurality of artificial examples generated by the artificial example generation unit 22.

[0034] (1) Method for selecting artificial examples First, a method for selecting an artificial example by the artificial example selection unit 23 will be described. (Method 1) In method 1, the artificial example selection unit 23 selects "artificial examples with uncertain predictions" as described with reference to Fig. 2. For example, the artificial example selection unit 23 selects, from among a plurality of artificial examples, an artificial example that is closest to the decision boundary or an artificial example that is within a predetermined distance from the decision boundary.

[0035] (Method 2) In method 2, the artificial example selection unit 23 does not simply select artificial examples with uncertain predictions, but selects "a plurality of artificial examples with uncertain predictions and dissimilar to each other." This makes it possible to add dissimilar artificial examples without selecting similar but redundant artificial examples, thereby improving the efficiency of learning and more effectively solving the above-mentioned problem 2. Specifically, any of the following three methods is used as method 2.

[0036] (Method 2-1) In method 2-1, the artificial case selection unit 23 calculates the similarity between artificial cases and selects artificial cases such that the artificial cases are not similar to each other. FIG. 6 is a diagram for explaining method 2-1. First, in step S11, the input unit 21 acquires multiple real cases. Next, in step S12, the artificial case generation unit 22 generates multiple artificial cases from each real case. Next, in step S13, the artificial case selection unit 23 calculates the prediction uncertainty for the multiple generated artificial cases and selects artificial cases with uncertain predictions, i.e., artificial cases with high uncertainty.

[0037] Next, in step S14, the artificial example selection unit 23 selects artificial examples with high uncertainty from among a plurality of artificial examples with uncertain predictions so that the artificial examples are not similar to each other. Specifically, the artificial example selection unit 23 calculates the similarity between the artificial examples, and does not select an artificial example that is highly similar to an artificial example that has already been selected. In this way, artificial examples that are not similar to each other are selected. Then, in step S15, the output unit 24 adds the selected artificial example to the training examples.

[0038] (Method 2-2) In method 2-2, the artificial example selection unit 23 selects artificial examples such that the real examples closest to the acquired artificial example do not match each other. FIG. 7 is a diagram for explaining method 2-2. First, in step S21, the input unit 21 acquires multiple real examples. Next, in step S22, the artificial example generation unit 22 generates multiple artificial examples from each real example. Next, in step S23, the artificial example selection unit 23 calculates the prediction uncertainty for the multiple generated artificial examples, and selects artificial examples with uncertain predictions, i.e., artificial examples with high uncertainty.

[0039] Next, in step S24, the artificial case selection unit 23 selects an artificial case with high uncertainty from among the multiple artificial cases with uncertain predictions, such that the real cases with the closest distance do not match. Specifically, for each of the artificial cases with high uncertainty, the artificial case selection unit 23 determines the real case with the closest distance in the feature space (hereinafter referred to as the "nearest real case"), and selects multiple artificial cases such that the nearest real cases are different from each other. For example, the artificial case selection unit 23 selects one artificial case from each of the multiple artificial cases that have the same real case as the nearest real case. In this way, artificial cases that are not similar to each other are selected. Then, in step S25, the output unit 24 adds the selected artificial case to the training cases.

[0040] In this case, the artificial example selection unit 23 may use the Euclidean distance as the distance between the artificial examples and the real examples, or may use a distance other than the Euclidean distance, or may use a similarity such as cosine similarity.

[0041] In addition, instead of selecting artificial cases such that the nearest real cases do not match as described above, the artificial case selection unit 23 may select artificial cases such that a predetermined number (M, where M≦K) of the nearest real cases do not match.

[0042] (Method 2-3) In method 2-3, the artificial case selection unit 23 selects artificial cases such that the real cases that are the generation sources do not match. Specifically, when the artificial case generation unit 22 generates multiple artificial cases from real cases, the artificial case selection unit 23 pairs each artificial case with the real case that is the generation source. Next, the artificial case selection unit 23 calculates the uncertainty of each artificial case and acquires artificial cases in descending order of uncertainty. At this time, the artificial case selection unit 23 does not acquire an artificial case that is paired with the same real case as an artificial case that has already been acquired, that is, an artificial case that has the same real case as an artificial case that has already been acquired as the generation source. This prevents multiple artificial cases that have the same real case as the generation source from being selected at the same time. In this way, the artificial case selection unit 23 acquires a certain number of artificial cases. Then, the output unit 24 adds the selected artificial case to the training cases.

[0043] FIG. 8 is a diagram for explaining method 2-3. As shown in the figure, assume that there are real case A and real case B, and three artificial cases 82 to 84 are generated from real case A. The artificial case 84 is closer to real case B than to real case A. Therefore, when method 2-2 is applied, the artificial case 83 closest to real case A and the artificial case 84 closest to real case B are selected. In contrast, in method 2-3, although the artificial case 84 is closer to real case B than to real case A, it is paired with real case A because it has real case A as its generation source. Therefore, of the artificial cases 82 to 84 whose generation source is real case A, the one with the highest uncertainty is selected.

[0044] (2) Method for selecting cases with uncertain predictions Next, a method for selecting cases with uncertain predictions will be described in detail. In this embodiment, active learning is used as an index for selecting cases with uncertain predictions. Active learning is a method for finding cases that cannot be predicted well by a current machine learning model and having an oracle assign a label to them. The accuracy of the machine learning model can be improved by adding cases labeled by the oracle and re-learning. The oracle may be a human or a machine learning model.

[0045] In this embodiment, the artificial case selection unit 23 selects an artificial case whose prediction is determined to be uncertain when evaluated using the criteria used in active learning as an artificial case whose prediction is uncertain. In other words, the artificial case selection unit 23 selects an artificial case that is the subject of a query to the oracle when evaluated using an active learning method (hereinafter also referred to as a "query case") as an artificial case whose prediction is uncertain. Each specific active learning method will be explained below. Note that active learning methods other than the following three may also be used.

[0046] (Query by committee) A query by committee can be used as a method of active learning. Figure 9 is an overview of a query by committee. In a query by committee, multiple models are generated from training cases. The types of models can be different. A committee is made up of multiple models, and the prediction results of each model for the training cases are obtained. Cases where the prediction results of the multiple models belonging to the committee differ are treated as query cases.

[0047] For example, when using vote entropy, which is one method of query by committee, the query case can be determined using the vote entropy value. In vote entropy, the case with the largest entropy of the voting results by multiple classifiers (i.e., the case with the most split votes) is determined as the query case. Specifically, the query case is the case x^ given by the following formula. Note that in this specification, for convenience, the letter "x" with a "^" above it is written as "x^".

[0048]

number

[0049] The value in the parentheses in formula (2) is the vote entropy value. Therefore, when using the vote entropy, the artificial example selection unit 23 may determine that an artificial example having a vote entropy value equal to or greater than a certain value is an artificial example with an uncertain prediction.

[0050] (Uncertanity sampling) As another active learning method, uncertainty sampling can be used. Specifically, least confidence in uncertainty sampling can be used as an index of the uncertainty of the prediction. In this case, as shown in the following formula, the example x^ with the smallest probability of the "maximum probability label" is taken as the query example.

[0051]

number

[0052] In addition, margin sampling in uncertainty sampling can be used as an index of the uncertainty of the prediction. In this case, the query example is the example x^ for which the difference between the probability of the "first most probable label" and the probability of the "second most probable label" is the smallest, as shown in the following formula.

[0053]

number

[0054] [Artificial case generation unit] Next, the artificial example generation unit 22 will be described in detail. (1) How to select actual cases First, a method for selecting real cases that are the source of generating artificial cases will be described. Basically, the artificial case generation unit 22 may select real cases by some method. Therefore, for example, the artificial case generation unit 22 may generate artificial cases using all real cases, or may generate artificial cases using real cases randomly selected from all real cases.

[0055] However, since the artificial example selection unit 23 selects artificial examples with uncertain predictions from among the generated artificial examples as artificial examples to be added to the training examples, it is desirable that the real cases from which the artificial examples are generated are real cases from which artificial examples with uncertain predictions are likely to be generated. From this perspective, the above-mentioned active learning can also be used for selecting the real cases. That is, the artificial example generation unit 22 selects real cases with uncertain predictions from among the multiple real cases using an active learning technique, and generates multiple artificial examples using the selected real cases.

[0056] FIG. 10 shows a schematic diagram of a method of using active learning to select real cases. First, in step S31, the input unit 21 acquires a plurality of real cases. Next, in step S32, the artificial case generation unit 22 selects a real case with an uncertain prediction by active learning. At this time, the method by which the artificial case generation unit 22 selects a real case with an uncertain prediction from a plurality of real cases is basically the same as the method by which the artificial case selection unit 23 selects an artificial case with an uncertain prediction from a plurality of artificial cases. That is, the artificial case generation unit 22 selects a real case with an uncertain prediction by using any of the active learning methods described above. As a result, as shown in FIG. 10, some real cases may not be selected as a source for generating artificial cases.

[0057] Next, in step S33, the artificial case generation unit 22 generates artificial cases from the selected real cases. The generated artificial cases are output to the artificial case selection unit 23. Then, in step S34, the artificial case selection unit 23 selects artificial cases with uncertain predictions from the input artificial cases. In this case, the active learning technique is used twice: when the artificial case generation unit 22 selects real cases, and when the artificial case selection unit 23 selects artificial cases with uncertain predictions.

[0058] (2) Method for generating artificial examples Next, a method for generating artificial cases by the artificial case generation unit 22 will be described. The artificial case generation unit 22 generates artificial cases by synthesizing a real case, which is a source of generation, with another real case. In one method, the artificial case generation unit 22 can generate artificial cases using the above-mentioned formula (1). The artificial case generation unit 22 can also use an artificial case generation method such as MUNGE shown in Non-Patent Document 2 or SMOTE shown in Non-Patent Document 3.

[0059] [Artificial case generation processing] Next, an artificial case generation process performed by the artificial case generation device 100 will be described. Fig. 11 is a flowchart of the artificial case generation process. This process is realized by the processor 12 shown in Fig. 4 executing a program prepared in advance and operating as each element shown in Fig. 5.

[0060] First, the input unit 21 acquires real cases (step S41). Next, the artificial case generation unit 22 generates artificial cases based on the acquired real cases (step S42). At this time, as described above, the artificial case generation unit 22 may use all real cases as real cases from which the artificial cases are generated, may use randomly selected real cases, or may use real cases with uncertain predictions selected by an active learning technique. Furthermore, the artificial case generation unit 22 may use equation (1) or the MUNGE or SMOTE technique as a method for generating artificial cases. The artificial case generation unit 22 outputs the generated artificial cases to the artificial case selection unit 23.

[0061] Next, the artificial case selection unit 23 selects an artificial case with an uncertain prediction from the input artificial cases (step S43). At this time, the artificial case selection unit 23 selects an artificial case by any one of the methods 1, 2-1, 2-2, and 2-3 as described above. The artificial case selection unit 23 outputs the selected artificial case to the output unit 24. Next, the output unit 24 outputs the input artificial case, i.e., the artificial case selected by the artificial case selection unit 23, as a training case (step S44).

[0062] Next, the artificial case generation device 100 determines whether or not the end condition is met (step S45). For example, the artificial case generation device 100 determines that the end condition is met when a required number of artificial cases are obtained. If the end condition is not met (step S45: No), the process returns to step S41, and steps S41 to S45 are repeated. On the other hand, if the end condition is met (step S45: Yes), the process ends.

[0063] [Assigning labels to artificial examples] In the above embodiment, the artificial example generation device 100 outputs unlabeled artificial examples. Instead, however, it may output labeled artificial examples. For example, the output unit 24 may assign labels to the artificial examples input from the artificial example selection unit 23 and output labeled artificial examples. In this case, the output unit 24 may assign the same labels as the real examples that were the generation sources to the input artificial examples. Alternatively, the output unit 24 may assign labels that are assigned by a machine learning model prepared in advance to the input artificial examples. Note that labels may be assigned to the artificial examples by humans and output as labeled artificial examples.

[0064] <Second embodiment> 12 is a block diagram showing the functional configuration of an information processing apparatus according to the second embodiment. The information processing apparatus 70 includes an input unit 71, an artificial example generating unit 72, an artificial example selecting unit 73, and an output unit 74.

[0065] 13 is a flowchart of processing by the information processing device 70 of the second embodiment. First, the input means 71 acquires real cases consisting of feature quantities (step S71). Next, the artificial case generation means 72 generates a plurality of artificial cases from the real cases (step S72). Next, the artificial case selection means 73 selects an artificial case where the prediction of the machine learning model becomes uncertain from the plurality of generated artificial cases (step S73). Then, the output means 74 outputs the selected artificial case (step S74).

[0066] According to the information processing device 70 of the second embodiment, it is possible to generate artificial examples that contribute to improving the prediction performance of a machine learning model.

[0067] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0068] (Appendix 1) An input means for acquiring actual examples consisting of feature quantities; an artificial example generating means for generating a plurality of artificial examples from the real examples; an artificial example selection means for selecting an artificial example for which a prediction of the machine learning model is uncertain from the plurality of generated artificial examples; an output means for outputting the selected artificial examples; An information processing device comprising:

[0069] (Appendix 2) 2. The information processing device according to claim 1, wherein the artificial example selection means selects the plurality of artificial examples such that the selected artificial examples are different from one another.

[0070] (Appendix 3) 3. The information processing device according to claim 1, wherein the artificial example selection means selects the plurality of artificial examples such that real examples existing in close proximity in a feature space are different.

[0071] (Appendix 4) 3. The information processing apparatus according to claim 1, wherein the artificial example selection means selects the plurality of artificial examples such that real examples serving as sources for generating the respective artificial examples are different.

[0072] (Appendix 5) 5. The information processing device according to claim 1, wherein the artificial example generation means generates the artificial examples by using all input real examples.

[0073] (Appendix 6) 5. The information processing device according to claim 1, wherein the artificial example generation means generates the artificial examples using a plurality of real examples randomly selected from input real examples.

[0074] (Appendix 7) 5. The information processing device according to claim 1, wherein the artificial example generation means selects an actual example from a plurality of input real examples for which a prediction of a machine learning model is uncertain, and generates the plurality of artificial examples using the selected actual example.

[0075] (Appendix 8) The information processing device according to any one of claims 1 to 7, wherein the output means assigns a label to the selected artificial example and outputs the label.

[0076] (Appendix 9) Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected, An information processing method for outputting selected artificial examples.

[0077] (Appendix 10) Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected, A recording medium storing a program for causing a computer to execute a process for outputting selected artificial examples.

[0078] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-mentioned embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]

[0079] 11 Interface 12 processors 13. Memory 14 Recording media 15 Database (DB) 21 Input section 22 Artificial case generation section 23 Artificial Example Selection Section 24 Output section 100 Artificial case generation device

Claims

1. An input means for acquiring actual examples consisting of feature quantities; an artificial example generating means for generating a plurality of artificial examples from the real examples; an artificial example selection means for selecting an artificial example for which a prediction of the machine learning model is uncertain from the plurality of generated artificial examples; an output means for outputting the selected artificial examples; Equipped with The artificial example selection means is an information processing device that selects artificial examples such that real examples existing in the vicinity of each other in a feature space are different.

2. 2. The information processing apparatus according to claim 1, wherein the artificial example generating means generates the artificial examples by using all input real examples.

3. 2. The information processing apparatus according to claim 1, wherein the artificial example generating means generates the artificial examples by using a plurality of real examples randomly selected from input real examples.

4. The information processing device according to claim 1 , wherein the artificial example generation means selects an actual example for which the prediction of the machine learning model is uncertain from among a plurality of input real examples, and generates the plurality of artificial examples using the selected actual example.

5. The information processing apparatus according to claim 1 , wherein the output means outputs the selected artificial example with a label added thereto.

6. 1. A computer-implemented information processing method, comprising: Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected such that the real examples existing nearby in the feature space are different; An information processing method for outputting selected artificial examples.

7. Acquire real examples consisting of features, generating a plurality of artificial examples from the real examples; From the multiple artificial examples generated, artificial examples for which the prediction of the machine learning model is uncertain are selected such that the real examples existing nearby in the feature space are different; A program that causes a computer to execute a process of outputting selected artificial examples.

Images