Learning apparatus, learning method, and learning program

Abstract

To prevent learning from being stagnating due to many similar samples selected as candidate for labeling.SOLUTION: An analysis unit determines a confidence level of a result estimated by a machine learning model for each of a plurality of unknown samples, which are input samples to which corresponding output samples are not assigned. A selection unit selects an unknown sample that requires assignment of an output sample for training the machine learning model, from among the unknown samples. The selection unit selects the unknown sample based on the confidence level, or selects the unknown sample regardless of confidence level.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a learning device, a learning method, and a learning program. [Background technology]

[0002] In supervised learning of machine learning, a method called active learning is known, in which a training dataset with high learning efficiency is selected for learning. Generally, creating output samples for input samples (labeling) is costly, and there is a need to improve the accuracy of machine learning models while reducing the labeling work. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7350587 Summary of the Invention [Problem to be solved by the invention]

[0004] One method for selecting input samples for which labeling is required is the uncertainty sampling method, which selects samples for which the computational results in a machine learning model have low reliability and then determines the labeling for the selected samples. However, the uncertainty sampling method has the risk of selecting samples that are similar to each other, which can cause learning to stagnate. The problem to be solved by the present invention is to provide a learning device, a learning method, and a learning program that can prevent learning from stagnating due to only similar samples being selected as candidates for labeling. [Means for solving the problem]

[0005] A learning device according to an embodiment is a learning device that trains a machine learning model based on a training dataset that is a combination of input samples and output samples. The learning device includes an analysis unit and a selection unit. The analysis unit calculates the reliability of an estimation result obtained by the machine learning model for each of a plurality of unknown samples, which are input samples to which no corresponding output sample is assigned. The selection unit selects, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model. The selection unit selects the unknown sample based on the reliability or regardless of the reliability. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic block diagram illustrating a hardware configuration of a learning device according to at least one embodiment. [Figure 2] FIG. 2 is a schematic block diagram showing the software configuration of the learning device according to the first embodiment. [Figure 3] FIG. 4 is a diagram showing an example of a random rate determination function according to the first embodiment. [Figure 4] 4 is a flowchart showing an active learning process performed by the learning device according to the first embodiment. [Figure 5] FIG. 10 is a schematic block diagram showing the software configuration of a learning device according to a second embodiment. [Figure 6] 10 is a flowchart showing an active learning process performed by a learning device according to a second embodiment. [Figure 7] 10 shows an example of a random rate selection screen according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] A learning device, a learning method, and a learning program according to an embodiment will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the hardware configuration of a learning device according to at least one embodiment. The learning device 100 updates the parameters of a machine learning model through active learning. The machine learning model handled by the learning device 100 may be any machine learning model to which supervised learning is applied. Examples of machine learning models include neural networks, support vector machines, decision trees, and logistic regression. The machine learning model may also be a classification model that classifies input data into several classes, or a regression model that calculates continuous values ​​from input data. The learning device 100 according to the embodiment performs learning processing for a classification model in which the machine learning model is configured as a neural network.

[0008] The learning device 100 selects an input sample to be used for training a machine learning model from among multiple input samples to which no corresponding output sample has been assigned, and requests the user, who acts as an oracle, to assign an output sample corresponding to the selected input sample. The learning device 100 trains the machine learning model using a combination of the selected input sample and the output sample assigned by the user as a training dataset. Hereinafter, regardless of whether it is a classification model or a regression model, the output sample is also referred to as a "label," and the act of the user selecting an output sample corresponding to an input sample is also referred to as "labeling."

[0009] As shown in FIG. 1, the learning device 100 includes a processor 110, a main memory 120, a storage 130, and an interface 140. The processor 110 reads a learning program for training a machine learning model from the storage 130, loads it into the main memory 120, and executes processing in accordance with the learning program. The processor 110 also allocates a necessary storage area in the main memory 120 in accordance with the learning program. Examples of the processor 110 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a microprocessor.

[0010] The learning program may be for realizing some of the functions to be performed by the learning device 100. For example, the learning program may be combined with other programs already stored in storage or implemented in other devices to achieve the functions. In other embodiments, the learning device 100 may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor 110 may be realized by the integrated circuit. Such an integrated circuit is also an example of a processor. In other embodiments, the learning device 100 may be virtualized on one or more computers.

[0011] Examples of storage 130 include a magnetic disk, a magneto-optical disk, an optical disk, and a semiconductor memory. Storage 130 may be an internal medium directly connected to the bus of learning device 100, or an external medium connected to learning device 100 via interface 140 or a communication line. When a learning program is distributed to learning device 100 via a communication line, learning device 100 that receives the program may load the learning program into main memory 120 and execute the above-described processing. In at least one embodiment, storage 130 is a non-transitory tangible storage medium.

[0012] An output device and an input device may be connected to the interface 140. The output device presents information such as the input sample that is the target of the query and the estimated classification destination to the user. Examples of the output device include display devices such as a CRT (Cathode Ray Tube) display, a liquid crystal display, and an organic EL (Electro-Luminescence) display, and audio output devices such as a speaker. Examples of the input device include a mouse, a keyboard, a touch panel, and a microphone. The interface 140 may be connected to a network. For example, the interface 140 may be connected to a terminal device, such as a smartphone, operated by a user via a network.

[0013] (First embodiment) FIG. 2 is a schematic block diagram showing the software configuration of the learning device 100 according to the first embodiment. The processor 110 of the learning device 100 according to the first embodiment executes a learning program, and thereby includes a learning unit 111, an analysis unit 112, a degree determination unit 113, a selection unit 114, a label acquisition unit 115, and a data update unit 116. In addition, storage areas for a model database 131, a known database 132, and an unknown database 133 are secured in the storage 130.

[0014] The model database 131 stores the network structure of the machine learning model, the weights, biases, optimization functions, and other information. The model database 131 also stores information such as the output value of the loss function during machine learning, the accuracy rate for a training dataset, and the accuracy rate for a validation dataset for verifying performance.

[0015] The known database 132 stores known samples, which are labeled input samples. For example, when a machine learning model classifies image data of products into product types, the known database 132 stores output samples (labels) in association with the image data, which are input samples, and information such as the product type as the correct classification destination. The known database 132 may store combinations of input samples and output samples in association with information on the correct classification destination and ID information that uniquely identifies the input sample. A validation dataset, which is a combination of input samples and output samples for confirming the level of learning of the machine learning model, may also be recorded in the known database 132.

[0016] The unknown database 133 stores unknown samples, which are input samples to which no label has been assigned. In addition to the input samples, the unknown database 133 may also include ID information that uniquely identifies the input samples as table data. In other words, the input samples stored in the unknown database 133 are not associated with output samples (labels) that contain information such as product types that are the correct classification destinations.

[0017] The learning unit 111 uses known samples stored in the known database 132 as a training dataset to train the machine learning model stored in the model database 131. The learning unit 111 records data of the machine learning model after training in the model database 131. The learning unit 111 may record in the model database 131 the output of the loss function used during training and the accuracy rate of the machine learning model for the training dataset and the validation dataset.

[0018] The analysis unit 112 calculates the reliability of the estimation result obtained by the machine learning model. The analysis unit 112 estimates output values ​​for unknown samples stored in the unknown database 133 using the machine learning model updated by the learning unit 111. For example, if the machine learning model is a class classification model for image data using a convolutional neural network, the analysis unit 112 may treat the probability of the classification destination obtained from the output layer of the machine learning model as the reliability, or may treat the entropy value of the output layer as the reliability. For example, if the machine learning model is a support vector machine, the analysis unit 112 may calculate the distance from the decision boundary as the reliability. For example, if the machine learning model is a regression model that outputs a confidence interval, the analysis unit 112 may calculate the inverse of the width of the confidence interval as the reliability. For example, if the machine learning model is an ensemble model, the analysis unit 112 may calculate the KL divergence of multiple small learners as the reliability.

[0019] The degree determination unit 113 determines the degree of learning of the machine learning model. The degree determination unit 113 according to the first embodiment determines that the higher the accuracy rate for the validation dataset recorded in the known database 132, the more advanced the learning is. Note that the degree determination unit 113 according to other embodiments may determine the degree of learning based on the calculated value of the loss function during the previous learning, the accuracy rate for the learning dataset, the number of iterations, etc.

[0020] The selection unit 114 selects an input sample to be used for training the machine learning model from among the multiple unknown samples. The selection unit 114 can select the input sample based on a predetermined query strategy in active learning, based on the reliability calculated by the analysis unit 112. Alternatively, the selection unit 114 can randomly select an input sample from among the multiple unknown samples, regardless of the reliability. The selection unit 114 determines the ratio (random ratio) of input samples to be selected based on the reliability and input samples to be selected randomly, depending on the level of learning of the machine learning model. FIG. 3 is a diagram illustrating an example of a random ratio determination function according to the first embodiment. When the accuracy rate calculated by the level determination unit 113 is higher than a first threshold (e.g., 0.8), i.e., when the level of learning is high, the selection unit 114 according to the first embodiment selects input samples to be used in the learning dataset based on the reliability. Furthermore, when the accuracy rate calculated by the level determination unit 113 is lower than the first threshold and the gradient of the accuracy rate exceeds a second threshold (e.g., 0.1), i.e., when learning is progressing smoothly, the selection unit 114 selects input samples to be used in the learning dataset based on the reliability. On the other hand, if the accuracy rate calculated by the degree determination unit 113 is lower than the first threshold and the gradient of the accuracy rate is lower than the second threshold, i.e., if learning is stagnating, the selection unit 114 randomly selects some input samples regardless of their reliability according to a random rate.

[0021] The selection unit 114 may determine the random rate according to the learning progress, such as the number of iterations of the machine learning model. Fig. 3 is a diagram showing an example of the relationship between the learning progress and the random rate according to the first embodiment. For example, the selection unit 114 may determine in advance a relationship in which the random rate monotonically decreases with the learning progress, and determine the random rate based on the learning progress. Furthermore, the selection unit 114 may determine the random rate according to the gradient of the correct answer rate. For example, the selection unit 114 may increase the random rate as the gradient of the correct answer rate decreases. For example, the selection unit 114 may decrease the random rate when the random rate in the previous learning is high and the gradient of the correct answer rate is low.

[0022] The label acquisition unit 115 presents a labeling request (query) for the input sample selected by the selection unit 114 to the user, and acquires the label input by the user in response to the labeling request.

[0023] The data updating unit 116 associates the labels acquired by the label acquiring unit 115 with the corresponding input samples and records them in the known database 132. On the other hand, the data updating unit 116 deletes the labeled input samples from the unknown database 133.

[0024] 4 is a flowchart showing the active learning process performed by the learning device 100 according to the first embodiment. The learning device 100 performs learning of a machine learning model in the following procedure.

[0025] First, the learning unit 111 updates the parameters of the machine learning model stored in the model database 131 using a learning dataset stored in the known database 132 (step S1). The learning unit 111 records the updated machine learning model in the model database 131. The degree determination unit 113 calculates the accuracy rate of the updated machine learning model using a verification dataset stored in the known database 132 (step S2). The degree determination unit 113 records the calculated accuracy rate in the model database 131 in association with the number of iterations.

[0026] The learning unit 111 determines whether the learning termination condition of the machine learning model is satisfied (step S3). If the learning termination condition is satisfied (step S3: YES), the learning device 100 terminates the learning process. If the learning termination condition is not satisfied (step S3: NO), the analysis unit 112 acquires an arbitrary number of unknown samples from the unknown database 133 and calculates estimation results and their reliability for the unknown samples using the machine learning model updated in step S2 (step S4). The analysis unit 112 calculates estimation results and their reliability for unknown samples that are greater than the number of samples (N) in the newly created learning dataset. Note that the analysis unit 112 may calculate estimation results and reliability for all unknown samples.

[0027] The selection unit 114 determines whether the accuracy rate of the machine learning model calculated in step S2 exceeds a first threshold (step S5). If the accuracy rate exceeds the first threshold (step S5: YES), the selection unit 114 determines that the level of learning is high, and selects N input samples based on the reliability calculated in step S4 (step S6). For example, the selection unit 114 selects N input samples in ascending order of reliability.

[0028] On the other hand, if the accuracy rate does not exceed the first threshold (step S5: NO), the selection unit 114 calculates the gradient of the accuracy rate (step S7). For example, the selection unit 114 calculates the difference between the accuracy rate calculated in step S2 and the previous accuracy rate recorded in the model database 131 as the gradient of the accuracy rate. The selection unit 114 determines whether the gradient of the accuracy rate exceeds a second threshold (step S8). If the gradient of the accuracy rate exceeds the second threshold (step S8: YES), the selection unit 114 determines that learning is progressing smoothly, and selects N input samples based on the reliability calculated in step S4 (step S6).

[0029] If the accuracy rate is not higher than the first threshold and the gradient of the accuracy rate does not exceed the second threshold (step S8: NO), the selector 114 determines that learning has stagnated. The selector 114 randomly acquires R×N input samples, which are N input samples multiplied by a predetermined random rate R, from the unknown database 133 (step S9). The selector 114 also selects (1−R)×N input samples based on the reliability calculated in step S4 (step S10).

[0030] Next, the label acquisition unit 115 presents a labeling request for the N unknown samples selected by the selection unit 114 to the user, and acquires labels corresponding to each unknown sample (step S11). The labeling request may be displayed on a display connected to the interface 140, or may be displayed on the user's terminal via a network connected to the interface 140. Label data may be acquired from an input device such as a keyboard connected to the interface 140, or may be acquired from the user's terminal via a network connected to the interface 140.

[0031] The data updating unit 116 associates the labels acquired by the label acquiring unit 115 with the corresponding input samples and records them in the known database 132 (step S12). On the other hand, the data updating unit 116 deletes the labeled input samples from the unknown database 133 (step S13). The learning device 100 returns the process to step S1 and performs the learning process of the machine learning model.

[0032] As described above, the learning device 100 according to the first embodiment has the following functions. The analysis unit 112 calculates the reliability of the estimation result by the machine learning model for each of a plurality of unknown samples, which are input samples to which no corresponding output sample is assigned. The selection unit 114 selects, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model. At this time, the selection unit 114 selects the unknown sample based on the reliability, or selects the unknown sample regardless of the reliability. By selecting unknown samples that require labeling based on their reliability, the learning device 100 can preferentially incorporate input samples with low computational reliability into the training dataset, thereby improving the learning efficiency and performance of the machine learning model. Furthermore, if the progress of learning in the machine learning model stagnates during repeated iterations of active learning due to the selection of training data based on reliability, the stagnation of learning can be prevented by randomly selecting some input samples to be incorporated into the training dataset regardless of their reliability.

[0033] (Second embodiment) The learning device 100 according to the first embodiment determines whether to select unknown samples based on reliability or randomly depending on the level of learning. In contrast, the learning device 100 according to the second embodiment allows a user, who is an oracle, to determine whether to select unknown samples based on reliability or randomly.

[0034] 5 is a schematic block diagram showing the software configuration of a learning device 100 according to the second embodiment. The learning device 100 according to the second embodiment includes an information output unit 117 and an instruction input unit 118 instead of the degree determination unit 113. The information output unit 117 presents to the user information related to the estimation results of unknown samples selected based on the reliability. After presenting the estimation results, the instruction input unit 118 accepts input of a random rate from the user.

[0035] 6 is a flowchart showing the active learning process performed by the learning device 100 according to the second embodiment. The learning device 100 performs learning of a machine learning model in the following procedure.

[0036] First, the learning unit 111 updates the parameters of the machine learning model stored in the model database 131 using the learning dataset stored in the known database 132 (step S51). The learning unit 111 records the updated machine learning model in the model database 131. The degree determination unit 113 calculates the accuracy rate of the updated machine learning model using the verification dataset stored in the known database 132 (step S52). The degree determination unit 113 records the calculated accuracy rate in the model database 131 in association with the number of iterations.

[0037] The learning unit 111 determines whether the learning termination condition of the machine learning model is satisfied (step S53). If the learning termination condition is satisfied (step S53: YES), the learning device 100 terminates the learning process. If the learning termination condition is not satisfied (step S53: NO), the analysis unit 112 acquires an arbitrary number of unknown samples from the unknown database 133 and calculates estimation results and their reliability for the unknown samples using the machine learning model updated in step S52 (step S54).

[0038] Next, the selection unit 114 selects N input samples based on the reliability calculated in step S54 (step S55). The information output unit 117 presents the user with a random rate selection screen including information such as the tendency of the estimation results of the N input samples selected in step S54 (step S56). FIG. 7 is an example of a random rate selection screen D according to the second embodiment. The random rate selection screen D shown in FIG. 7 includes statistical information D1 indicating the classification results and proportions of the input samples selected based on the reliability by the machine learning model, an instruction D2 prompting the user to select a random rate, an input form D3 for the random rate R, and a confirm button D4.

[0039] The user checks the statistical information D1 on the random rate selection screen D and determines the random rate R. For example, if the statistical information D1 shows that the input samples are biased toward a specific class, the user may consider increasing the random rate R. The instruction input unit 118 accepts input of the random rate R from the user via the random rate selection screen D (step S57). The selection unit 114 randomly acquires R×N input samples, which are obtained by multiplying N input samples by the random rate R input in step S57, from the unknown database 133 (step S58). Note that if the random rate is zero, the selection unit 114 skips the process of step S58. The selection unit 114 selects (1−R)×N input samples based on the reliability calculated in step S4 (step S59).

[0040] Next, the label acquisition unit 115 presents a labeling request for the N unknown samples selected by the selection unit 114 to the user, and acquires labels corresponding to each unknown sample (step S60). The data update unit 116 associates the labels acquired by the label acquisition unit 115 with the corresponding input samples and records them in the known database 132 (step S61). On the other hand, the data update unit 116 deletes the labeled input samples from the unknown database 133 (step S62). The learning device 100 returns the process to step S51 and performs the learning process for the machine learning model.

[0041] In this way, the learning device 100 according to the second embodiment can break out of a learning stagnation and advance efficient machine learning by adjusting the ratio of reliability-based input sample selection and random input sample selection based on the judgment of the user acting as an oracle.

[0042] According to at least one embodiment described above, the learning device 100 includes an analysis unit 112 and a selection unit 114. The analysis unit 112 calculates the reliability of an estimation result by a machine learning model for each of a plurality of unknown samples, which are input samples to which no corresponding output sample is assigned. The selection unit 114 selects, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model. At this time, the selection unit 114 selects the unknown sample based on the reliability, or selects the unknown sample regardless of the reliability. This allows the learning device 100 according to at least one embodiment to prevent learning from stagnating due to only similar samples being selected as labeling candidates.

[0043] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents.

[0044] In the above-described embodiment, the learning device 100 randomly selects some input samples regardless of their reliability, but this is not limiting. The learning device 100 according to other embodiments may use a selection method other than random selection that does not depend on reliability, such as selecting samples sequentially from the beginning or selecting samples so that the classification results are uniform.

[0045] In the above-described embodiment, the learning device 100 selects some or all of the input samples that form the learning dataset based on the reliability and randomly selects the remaining samples, but this is not limited to this. For example, the learning device 100 according to another embodiment may randomly select all input samples when learning is stagnating. [Explanation of symbols]

[0046] 100...Learning device 110...Processor 111...Learning unit 112...Analysis unit 113...Level determination unit 114...Selection unit 115...Label acquisition unit 116...Data update unit 117...Information output unit 118...Instruction input unit 120...Main memory 130...Storage 131...Model database 132...Known database 133...Unknown database 140...Interface D...Random rate selection screen D1...Statistical information D2...Instruction statement D3...Input form D4...Decision button

Claims

1. A learning device that trains a machine learning model based on a learning dataset that is a combination of input samples and output samples, an analysis unit that calculates the reliability of an estimation result by the machine learning model for each of a plurality of unknown samples, which are input samples that do not have a corresponding output sample assigned; a selection unit that selects, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model; Equipped with the selection unit selects the unknown sample based on the reliability, or selects the unknown sample regardless of the reliability. Learning device.

2. the selection unit determines a ratio of unknown samples selected based on the reliability and unknown samples selected without regard to the reliability. The learning device according to claim 1 .

3. a degree determination unit that determines a degree of learning of the machine learning model; the selection unit determines whether to select the unknown sample based on the reliability or regardless of the reliability, depending on the degree of learning. The learning device according to claim 1 .

4. an information output unit that outputs information about an estimation result by the machine learning model of the unknown sample selected based on the reliability; an instruction input unit that receives an input of a ratio of unknown samples to be selected based on the reliability and a ratio of unknown samples to be selected regardless of the reliability after the estimation result is output; The learning device according to claim 1 , comprising:

5. an acquisition unit for acquiring an output sample corresponding to the selected unknown sample; a learning unit that trains the machine learning model using a learning dataset generated based on the unknown samples and the output samples; The learning device according to claim 1 , comprising:

6. A learning method for training a machine learning model based on a training dataset that is a combination of input samples and output samples, comprising: The computer an analysis step of calculating the reliability of an estimation result by the machine learning model for each of a plurality of unknown samples, which are input samples that do not have a corresponding output sample assigned; a selection step of selecting, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model; Equipped with In the selecting step, the unknown sample is selected based on the reliability, or the unknown sample is selected regardless of the reliability. How to learn.

7. A computer for training a machine learning model based on a training dataset that is a combination of input samples and output samples, an analysis step of calculating the reliability of an estimation result by the machine learning model for each of a plurality of unknown samples, which are input samples that do not have a corresponding output sample assigned; a selection step of selecting, from the plurality of unknown samples, an unknown sample to which an output sample is to be assigned for training the machine learning model; Execute In the selection step, the unknown sample is selected based on the reliability, or the unknown sample is selected regardless of the reliability. Learning program.

Images