Data expansion method, learning device and program

Abstract

This data augmentation method includes alternately carrying out first optimization processing (S11) to optimize a plurality of parameters of a DNN, and second optimization processing (S12) to optimize a hyperparameter for each sample for data augmentation processing. In the first optimization processing: the DNN is caused to predict a first augmentation label from a first augmentation sample obtained by subjecting a first sample included in a training dataset to data augmentation processing; a first error function for the first augmentation label and a first correct answer label of the first sample is calculated, and a plurality of parameters are updated on the basis of the first error function. In the second optimization processing: a second sample is acquired from an evaluation dataset having a distribution similar to that of a test dataset; the DNN of which the plurality of parameters have been updated is caused to predict a second label from the second sample; a second error function of the second label and a second correct answer label of the second sample is calculated; and the hyperparameter is updated on the basis of a gradient obtained by differentiating the second error function with respect to the hyperparameter.

Description

[Technical Field]

[0001] The present disclosure relates to a data augmentation method, a learning device, and a program for a training dataset used in training a neural network. [Background technology]

[0002] When developing a system that applies image recognition, it is necessary to collect a large amount of image data for training in order to obtain a highly accurate AI model.

[0003] While it is easy to collect large-scale samples such as image data for general-purpose objects like people, it is not possible to collect large-scale samples for non-general-purpose objects like strollers.Furthermore, it is also not possible to collect large-scale samples for objects that exist only in small quantities, such as specific products or abnormal samples.

[0004] In response to this, there is a technique called data augmentation that can increase the number of samples even when the number of training samples is small. Data augmentation is a technique that pads the number of training samples, such as image data, by performing a conversion process on the samples, and various data augmentation methods have been proposed (for example, Non-Patent Document 1).

[0005] Non-Patent Document 1 discloses a technique for expanding data by filling in part of an image with a different value. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. AutoAugment: Learning augmentation policies from data. arXiv:1805.09501, 2018. [Non-patent document 2] Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim. Fast AutoAugment. In Advances in Neural Information Processing Systems, 2019. Summary of the Invention [Problem to be solved by the invention]

[0007] However, with the technology disclosed in Non-Patent Document 1, although the number of learning samples can be increased, there are cases where a highly accurate AI model cannot be obtained. In other words, if the learning samples are not appropriately converted, the generated discrimination surface (discrimination line) through learning will deviate significantly from the ideal boundary line, which will not only not contribute to improving prediction accuracy, but may also decrease prediction accuracy.

[0008] The present disclosure has been made in consideration of the above-mentioned circumstances, and aims to provide a data augmentation method, etc., that can increase the number of samples in a training dataset so as to obtain a highly accurate AI model. [Means for solving the problem]

[0009] In order to solve the above problem, a data augmentation method according to one embodiment of the present disclosure is a data augmentation method for a training dataset used for training a neural network, which alternately performs a first optimization process for optimizing a plurality of parameters including weights in the neural network and a second optimization process for optimizing hyperparameters, which are variables that define a data conversion process for each sample used when performing the data augmentation process, and in the first optimization process, the data augmentation process is performed on a first sample included in the training dataset to obtain the first augmented sample, and the neural network predicts a first augmented label from the first augmented sample, and the first augmented label and a correct answer for the first sample included in the training dataset are calculated. and a first correct label indicating the correct answer for the second sample included in the evaluation dataset, and updates the plurality of parameters based on the calculated first error function. In the second optimization process, a second sample is obtained from an evaluation dataset, which is a dataset having a distribution similar to that of a test dataset for evaluating performance of the neural network, and the neural network, in which the plurality of parameters have been updated, is caused to predict a second label from the second sample. A second error function is calculated to evaluate the error between the second label and a second correct label indicating the correct answer for the second sample included in the evaluation dataset, and the hyperparameters are updated based on a gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters.

[0010] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0011] According to the data augmentation method and the like disclosed herein, it is possible to increase the number of samples in the training dataset so as to obtain a highly accurate AI model. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example of a configuration of a learning device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer that realizes the functions of the learning device according to the embodiment by software. [Figure 3] FIG. 3 is a flowchart showing an outline of the operation of the learning device according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing the detailed operation of the first optimization process shown in FIG. [Figure 5] FIG. 5 is a flowchart showing the detailed operation of the second optimization process shown in FIG. [Figure 6] FIG. 6 is a diagram for conceptually explaining an example of a process in which the first optimization process and the second optimization process are alternately performed according to the first embodiment. [Figure 7A] FIG. 7A is a diagram illustrating an example of an evaluation dataset according to the first embodiment. [Figure 7B] FIG. 7B is a diagram illustrating an example of a training dataset according to the first embodiment. [Figure 8A] FIG. 8A is a diagram conceptually illustrating an example of a function for performing the first optimization process according to the first embodiment. [Figure 8B] FIG. 8B is a diagram for explaining the specific function of the LOSS function shown in FIG. 8A. [Figure 9] FIG. 9 is a diagram illustrating pseudocode of an algorithm 1 showing an example of an optimization process procedure for alternately performing the first optimization process and the second optimization process according to the first embodiment. [Figure 10] FIG. 10 is a diagram for explaining a problem that occurs when data expansion is performed using hyperparameters shared among all data according to a comparative example. [Figure 11] FIG. 11 is a diagram for conceptually explaining generation of a discrimination surface through learning processing. [Figure 12A]FIG. 12A is a diagram for conceptually explaining the data extension process. [Figure 12B] FIG. 12B is a diagram conceptually illustrating a problem in generating a discrimination surface through a learning process when the number of data items included in the learning dataset is small. [Figure 12C] FIG. 12C is a diagram conceptually illustrating an example of generation of a discrimination surface by a learning process using a learning data set that has been data-augmented by a data augmentation process. [Figure 13] FIG. 13 is a diagram for explaining a problem in the data extension process according to the comparative example. [Figure 14] FIG. 14 is a diagram conceptually illustrating a discrimination surface generated using a training dataset that has been data-augmented by the data augmentation process according to the present disclosure. [Figure 15] FIG. 15 is a diagram showing the verification results obtained by evaluating the error rate using a test dataset after learning using a training dataset according to an experimental example. [Figure 16] FIG. 16 is a diagram showing an accuracy learning curve when learning is performed using a learning data set according to an experimental example. DETAILED DESCRIPTION OF THE INVENTION

[0013] A data augmentation method according to one embodiment of the present disclosure is a data augmentation method for a training dataset used for training a neural network, which alternately performs a first optimization process for optimizing a plurality of parameters including weights in the neural network and a second optimization process for optimizing hyperparameters, which are variables that define a data conversion process for each sample used when performing the data augmentation process, and in the first optimization process, performs the data augmentation process on a first sample included in the training dataset to obtain a first augmented sample, causes the neural network to predict a first augmented label from the first augmented sample, and obtains the first augmented label and a first correct answer indicating a correct answer for the first sample included in the training dataset. and updating the plurality of parameters based on the calculated first error function; in the second optimization process, a second sample is acquired from an evaluation dataset, which is a dataset having a distribution similar to that of a test dataset for evaluating the performance of the neural network; the neural network, in which the plurality of parameters have been updated, is caused to predict a second label from the second sample; a second error function is calculated to evaluate the error between the second label and a second correct label indicating the correct answer for the second sample included in the evaluation dataset; and the hyperparameters are updated based on a gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters.

[0014] This allows for the optimization of hyperparameters, thereby increasing the number of samples in the training dataset to obtain a highly accurate AI model.

[0015] More specifically, in the neural network training process, which optimizes multiple parameters of the neural network, the gradient obtained by partially differentiating the hyperparameters for each sample can be used to minimize the distribution discrepancy between the data-augmented training dataset and the test dataset. Therefore, even when training a neural network using training data that has been augmented by data augmentation processing, the number of samples in the training dataset can be increased so that the generated discriminant surface by training can approach the ideal discriminant surface. In this way, the number of samples in the training dataset can be increased to obtain a highly accurate AI model.

[0016] Furthermore, by alternately optimizing multiple neural network parameters and hyperparameters for data augmentation processing for each sample, the hyperparameters for each sample can be optimized simultaneously with the learning of multiple neural network parameters. This reduces the computational costs required for optimizing multiple neural network parameters and hyperparameters for each sample.

[0017] Here, for example, when updating the hyperparameters, the hyperparameters are updated based on the gradient so as to reduce the deviation between the distribution of the training dataset after data augmentation, which is obtained by performing the data augmentation process on the training dataset, and the distribution of the evaluation dataset.

[0018] This allows the hyperparameters to be updated so as to minimize the distribution discrepancy between the data-augmented training dataset and the test dataset. Therefore, even if a neural network is trained using training data that has been augmented by data augmentation processing, the generated discriminant surface through training can be brought closer to an ideal discriminant surface.

[0019] Also, for example, the hyperparameters may be implicit in a function representing the data augmentation process, and the neural network may be a fully differentiable function.

[0020] This allows us to calculate partial derivatives with respect to the hyperparameters using implicit differentiation techniques.

[0021] Furthermore, for example, when calculating the first error function, the first error function may be calculated by using Kullback-Leibler divergence to evaluate the error between a first correct soft label obtained by soft-labeling the correct value indicated by the first correct label and the first extended label.

[0022] This allows us to suppress noise in the correct labels for samples included after data augmentation. Therefore, we can optimize the hyperparameters for each sample by using the gradient obtained by partially differentiating the hyperparameters for each sample to minimize the distribution difference between the data-augmented training dataset and the test dataset.

[0023] Furthermore, for example, when calculating the first error function, a weighting calculated for each sample may be further performed on the first error function.

[0024] This allows us to capture and suppress sample bias. Therefore, we can use the gradient obtained by partially differentiating the hyperparameters for each sample to optimize the hyperparameters for each sample so as to minimize the distributional difference between the data-augmented training dataset and the test dataset.

[0025] Furthermore, a learning device according to one embodiment of the present disclosure is a learning device for performing a data augmentation method for a training dataset used for training a neural network, the learning device including a processor and a memory, wherein the processor uses the memory to alternately perform a first optimization process for optimizing a plurality of parameters including weights in the neural network and a second optimization process for optimizing hyperparameters, which are variables that define a data conversion process for each sample used when performing the data augmentation process, and in the first optimization process, the processor performs the data augmentation process on a first sample included in the training dataset to obtain a first augmented sample, and causes the neural network to predict a first augmented label from the first augmented sample, and calculates a value for the first augmented label and a value for the training dataset. and updating the plurality of parameters based on the calculated first error function. In the second optimization process, a second sample is obtained from an evaluation dataset, which is a dataset having a distribution similar to that of a test dataset for evaluating the performance of the neural network, and the neural network, in which the plurality of parameters have been updated, is caused to predict a second label from the second sample. A second error function is calculated to evaluate the error between the second label and a second correct label indicating the correct answer for the second sample included in the evaluation dataset, and the hyperparameters are updated based on the gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters.

[0026] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0027] The embodiments described below each illustrate a specific example of the present disclosure. The numerical values, shapes, components, steps, and step orders shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components. Furthermore, in all embodiments, the respective contents can be combined.

[0028] (Embodiment) Hereinafter, an information processing method and the like of the learning device 10 according to the embodiment will be described with reference to the drawings.

[0029] [1.1 Configuration of the learning device 10] FIG. 1 is a block diagram showing an example of the configuration of a learning device 10 according to an embodiment.

[0030] The learning device 10 is a device for training a neural network and for performing a data augmentation method for a training dataset used for training the neural network. The learning device 10 alternately performs a first optimization process for optimizing multiple parameters, including weights, in the neural network, and a second optimization process for optimizing hyperparameters, which are variables that define data conversion processing for each sample used when performing data augmentation processing.

[0031] In this embodiment, the learning device 10 includes an acquisition unit 11, a data augmentation processing unit 12, a DNN 13, an error calculation unit 14, and an optimization unit 15, as shown in FIG.

[0032] [1.1.1 Acquisition part 11] In the first optimization process, the acquisition unit 11 acquires a first sample from the training dataset and inputs it to the data augmentation processing unit 12. Here, the training dataset is composed of a plurality of samples, such as image data, and their correct labels. Note that the samples are not limited to being image data.

[0033] Furthermore, in the first optimization process, the acquisition unit 11 acquires a first correct label indicating the correct answer of the first sample from the learning dataset, and inputs it to the error calculation unit 14.

[0034] Furthermore, in the second optimization process, the acquisition unit 11 acquires a second sample from the evaluation dataset and inputs it to the DNN 13. Here, the evaluation dataset is a dataset with a distribution similar to that of a test dataset for evaluating the performance of the trained DNN 13. Note that the evaluation dataset and the test dataset are different. The evaluation dataset can be generated by dividing the test dataset of an existing dataset.

[0035] Furthermore, in the second optimization process, the acquisition unit 11 acquires a second correct label indicating the correct answer of the second sample from the evaluation dataset, and inputs it to the error calculation unit .

[0036] In the second optimization process, the acquisition unit 11 may acquire a third sample from the learning dataset and input it to the data augmentation processing unit 12. In this case, the acquisition unit 11 may acquire a third correct answer label indicating the correct answer for the third sample from the learning dataset and input it to the error calculation unit 14. The third sample may be different from or the same as the first sample.

[0037] [1.1.2 Data extension processing unit 12] In the first optimization process, the data augmentation processing unit 12 performs data augmentation processing on a first sample included in the training dataset and outputs the first augmented sample. More specifically, in the first optimization process, the data augmentation processing unit 12 performs data augmentation processing on the first sample using hyperparameters, which are variables that define data conversion processing for each sample. The hyperparameters are variables that define data conversion processing, such as rotation processing, zoom processing, translation processing, and color conversion processing, and are defined for each sample.

[0038] In this embodiment, the data augmentation processing unit 12 performs data augmentation processing by applying a function indicating the data augmentation processing to samples acquired from a training dataset. Here, the hyperparameters are made implicit in the function indicating the data augmentation processing.

[0039] Here, the function indicating the data augmentation process is g, and the hyperparameter for the first sample is λ1. In this case, the data augmentation processing unit 12 can perform data augmentation processing of the first sample by applying g(λ1) to the first sample in the first optimization process, and can output the first augmented sample. Therefore, the data augmentation processing unit 12 sets the hyperparameter for the i-th sample (i is a natural number) as λ in the first optimization process. i If g(λ i ) on the i-th sample, data augmentation processing for the i-th sample can be performed.

[0040] In the second optimization process, when a third sample is input by the acquisition unit 11, the data extension processing unit 12 performs data extension processing on the third sample and outputs the third extended sample. In this case, in the second optimization process, the data extension processing unit 12 can perform data extension processing on the third sample by applying g(λ3) where λ3 is the hyperparameter for the third sample to the third sample, and therefore can output the third extended sample.

[0041] [1.1.3 DNN13] The DNN 13 is configured by a neural network having a plurality of parameters, such as a CNN. In the first optimization process, the DNN 13 predicts a first augmented label from the first augmented sample input by the acquisition unit 11. The plurality of parameters are, for example, weights, but are not limited to weights.

[0042] In this embodiment, the DNN 13 performs a prediction process in the first optimization process, in which a function representing the prediction process by the neural network is applied to the first extended sample, thereby predicting a first extended label as a correct answer for the first extended sample. Here, the function representing the prediction process by the neural network is a fully differentiable function. Furthermore, multiple parameters are implicit functions in the function representing the prediction process by the neural network.

[0043] Here, if f is a function indicating the prediction process by the neural network and θ is a number of parameters including weights, DNN13 can predict the first augmented label by applying f(θ) to the first augmented sample in the first optimization process.

[0044] Furthermore, after the plurality of parameters have been updated by the first optimization process, the DNN 13 predicts a second label from a second sample included in the evaluation dataset input by the acquisition unit 11. In other words, in the second optimization process in this embodiment, the DNN 13 predicts a second label by applying f(θ), for which the plurality of parameters have been updated by the first optimization process, to the second sample.

[0045] In addition, when the third extended sample is input by the data extension processing unit 12 in the second optimization process, the DNN 13 only needs to predict, from the third extended sample, a third extended label as the correct answer for the third extended sample.

[0046] [1.1.4 Error calculation unit 14] In the first optimization process, the error calculation unit 14 calculates a first error function for evaluating the error between the first augmented label and a first correct label indicating the correct answer for the first sample included in the training dataset. The first error function here is a loss function known as Kullback-Leibler divergence (KL divergence). The first error function may also be a cross-entropy error function.

[0047] The Kullback-Leibler divergence can be used as a measure of the difference between two probability distributions in probability theory and information theory. In other words, the Kullback-Leibler divergence is a loss function that can evaluate how similar the correct probability distribution is to the estimated probability distribution.

[0048] In addition, in the first optimization process, the error calculation unit 14 may calculate a first error function that evaluates the error between the first extended label and the first correct soft label obtained by soft-labeling the correct value indicated by the first correct label using KL divergence. Here, the soft label means that the similarity between the first extended sample and the correct label is expressed as a ratio rather than a discrete value (hard label). For example, Gumbel-softmax or softmax can be used as the soft label. This makes it possible to suppress noise in the correct label for samples included after data extension.

[0049] Furthermore, in the first optimization process, the error calculation section 14 may further perform weighting, which is calculated for each sample, on the calculated first error function.

[0050] In addition, in the second optimization process, the error calculation unit 14 calculates a second error function for evaluating the error between the second label and a second correct label indicating the correct answer of the second sample included in the evaluation dataset. Here, the second error function may be a loss function called KL divergence, similar to the first error function, or may be a cross-entropy error function.

[0051] In addition, in the second optimization process, when the DNN13 predicts the third augmented label from the third augmented sample, the first error function is used to evaluate the error between the third augmented label and the third correct label indicating the correct answer for the third sample included in the training dataset.

[0052] [1.1.5 Optimization Section 15] In the first optimization process, the optimization unit 15 updates the multiple parameters based on the first error function calculated by the error calculation unit 14. This allows the optimization unit 15 to perform the first optimization process that optimizes the multiple parameters in the neural network.

[0053] Furthermore, in the second optimization process, the optimization unit 15 updates the hyperparameters based on the gradient obtained by partially differentiating the second error function calculated by the error calculation unit 14 with respect to the hyperparameters. Note that, since the hyperparameters are implicit functions, the partial derivatives of the hyperparameters can be calculated using an implicit function differentiation method. Here, the optimization unit 15 updates the hyperparameters based on the gradient so as to reduce the deviation between the distribution of the training dataset after data augmentation, which is obtained by performing data augmentation processing on the training dataset, and the distribution of the evaluation dataset. This allows the optimization unit 15 to optimize the hyperparameters, which are variables that specify the data conversion processing for each sample used when performing data augmentation processing, in the second optimization process.

[0054] The learning device 10 configured as described above alternately performs a first optimization process on multiple parameters in a neural network and a second optimization process on hyperparameters for each sample used when performing data augmentation. This allows the distribution of the training dataset that has undergone data augmentation processing to match the distribution of the test dataset, thereby increasing the number of samples in the training dataset to obtain a highly accurate AI model.

[0055] More specifically, even when a neural network is trained using training data with an increased number of samples that have been subjected to the data augmentation process as described above, the number of samples in the training dataset can be increased so that the generated discrimination surface through training can approach an ideal discrimination surface. In this way, the number of samples in the training dataset can be increased so that a highly accurate AI model can be obtained.

[0056] [1.2 Hardware configuration of the learning device 10] FIG. 2 is a diagram showing an example of the hardware configuration of a computer 1000 that realizes the functions of the learning device 10 according to the embodiment by software.

[0057] 2, the computer 1000 is a computer that includes an input device 1001, an output device 1002, a CPU 1003, an internal storage 1004, a RAM 1005, a reading device 1007, a transmitting / receiving device 1008, and a bus 1009. The input device 1001, the output device 1002, the CPU 1003, the internal storage 1004, the RAM 1005, the reading device 1007, and the transmitting / receiving device 1008 are connected by the bus 1009.

[0058] The input device 1001 is a user interface device such as an input button, a touchpad, a touch panel display, etc., and accepts user operations. Note that the input device 1001 may be configured to accept voice operations, remote operations using a remote control, etc., in addition to accepting user touch operations.

[0059] The output device 1002 is also used as the input device 1001, and is configured by a touch pad or a touch panel display, etc., and notifies the user of information that should be made known to the user.

[0060] The internal storage 1004 is a flash memory or the like. The internal storage 1004 may also store in advance a program for realizing the functions of the learning device 10, a neural network having multiple parameters, a function f indicating a prediction process by the neural network, a function g indicating a data augmentation process, and the like.

[0061] The RAM 1005 is a random access memory, and is used to store data and the like when a program or application is executed.

[0062] The reading device 1007 reads information from a recording medium such as a USB (Universal Serial Bus) memory. The reading device 1007 reads the above-mentioned programs and applications from a recording medium on which the programs and applications are recorded, and stores the programs and applications in the built-in storage 1004.

[0063] The transmitting / receiving device 1008 is a communication circuit for performing wireless or wired communication. The transmitting / receiving device 1008 may communicate with, for example, a server device or a cloud connected to a network, download the above-mentioned programs and applications from the server device or the cloud, and store them in the built-in storage 1004.

[0064] The CPU 1003 is a central processing unit that copies programs and applications stored in the internal storage 1004 to the RAM 1005 and sequentially reads and executes instructions contained in the programs and applications from the RAM 1005. Note that the instructions may also be executed directly from the internal storage 1004.

[0065] 1.3 Operation of the learning device 10 Next, the operation of the learning device 10 configured as above will be described.

[0066] FIG. 3 is a flowchart showing an outline of the operation of the learning device 10 according to the embodiment.

[0067] First, the learning device 10 performs a first optimization process that optimizes multiple parameters in the neural network (S11). Next, the learning device 10 performs a second optimization process that optimizes hyperparameters for each sample used when performing data augmentation processing (S12). Next, the learning device 10 determines whether to terminate the process (S13). For example, if the first optimization process and the second optimization process have been performed a predetermined number of times (number of epochs), the learning device 10 determines to terminate the process (Yes in S13) and terminates this operation. On the other hand, if the first optimization process and the second optimization process have not been performed a predetermined number of times (number of epochs), the learning device 10 determines not to terminate the process (No in S13) and repeats the process from step S11.

[0068] FIG. 4 is a flowchart showing the detailed operation of the first optimization process shown in FIG.

[0069] In the first optimization process, first, the learning device 10 performs data augmentation processing on a first sample included in the learning dataset to obtain a first augmented sample (S111). Here, the learning device 10 performs data augmentation processing on the first sample using hyperparameters, which are variables that define the data conversion processing for each sample. For example, if a function indicating the data augmentation processing is g and the hyperparameter for the first sample is λ1, the learning device 10 applies g(λ1) to the first sample to obtain a first augmented sample obtained by performing data augmentation processing on the first sample.

[0070] Next, the learning device 10 causes the neural network to predict a first augmented label from the first augmented sample acquired in step S111 (S112). For example, if a function indicating the prediction process by the neural network is f and multiple parameters including weights are θ, the learning device 10 causes the neural network to predict the first augmented label by applying f(θ) to the first augmented sample.

[0071] Next, the learning device 10 calculates a first error function for evaluating the error between the first augmented label predicted in step S112 and the first correct label of the first sample included in the learning dataset (S113). Here, the first error function is, for example, a loss function called KL divergence. As described above, the learning device 10 may calculate the first error function for evaluating the error between the first augmented label and a first correct soft label obtained by soft-labeling the correct value indicated by the first correct label, using KL divergence. The learning device 10 may further perform weighting calculated for each sample on the calculated first error function.

[0072] Next, the learning device 10 updates a plurality of parameters based on the first error function calculated in step S113 (S114).

[0073] In this way, the learning device 10 can perform a first optimization process that optimizes multiple parameters such as weights in a neural network using a training data set that has been augmented by a data augmentation process after the initial or second optimization process.

[0074] FIG. 5 is a flowchart showing the detailed operation of the second optimization process shown in FIG.

[0075] In the second optimization process, first, the learning device 10 acquires a second sample from an evaluation dataset, which is a dataset having a distribution similar to that of the test dataset (S121).

[0076] Next, the learning device 10 causes the neural network, whose parameters have been updated in the first optimization process, to predict a second label from the second sample (S122). For example, if a function indicating the prediction process by the neural network is f and multiple parameters including weights are θ, the learning device 10 causes the neural network to predict the second label by applying f(θ) to the second sample.

[0077] Next, the learning device 10 calculates a second error function for evaluating the error between the second label predicted in step S122 and the second correct label of the second sample included in the evaluation dataset (S123). Here, the second error function is, for example, a loss function called KL divergence, but may also be a cross-entropy error function.

[0078] Next, the learning device 10 updates the hyperparameters based on the gradient obtained by partially differentiating the second error function calculated in step S123 with respect to the hyperparameters (S124). Here, the learning device 10 updates the hyperparameters based on the gradient so as to reduce the deviation between the distribution of the training dataset after data augmentation, which is obtained by performing a data augmentation process on the training dataset, and the distribution of the evaluation dataset.

[0079] In this way, the learning device 10 can perform the second optimization process, which uses the evaluation dataset to optimize the hyperparameters for each sample used when performing the data augmentation process.

[0080] Example 1 Example 1 of this embodiment will be described below.

[0081] FIG. 6 is a diagram for conceptually explaining an example of a process in which the first optimization process and the second optimization process are alternately performed according to the first embodiment.

[0082] Process 12a indicates the data extension processing function performed by the data extension processing unit 12, and g(λ i ) is a function that indicates the data augmentation process and is the hyperparameter λ i In the example shown in FIG. 6, in the first optimization process, Process 12a selects a sample (x i , y i ) and take the sample (x i ) with the hyperparameter λ iThen, Process 12a performs data augmentation processing using the augmented sample g(x i , y i , λ i ) to DNN13a.

[0083] DNN13a indicates a neural network having a plurality of parameters θ of DNN13, and f(θ) indicates a function indicating a prediction process by the neural network. In the example shown in FIG. 6, in the first optimization process, the extended sample g(x i , y i , λ i ) has been shown to train the DNN13a neural network.

[0084] In the example shown in FIG. 6, in the second optimization process, a sample (x i v , y i v ) and take the sample (x i v ) is input to DNN13a. DNN13a takes the sample (x i v ) and performs prediction processing. Then, the DNN 13a applies a function f(θ) to the sample (x i v ) and output the correct label for it.

[0085] AutoDO15a indicates the second optimization processing function performed by the optimization unit 15, and ∂L v / ∂λ is the error function L calculated based on the results of the DNN13a prediction process. v More specifically, in the second optimization process, the sample (x i v ) and the correct label for the sample (x i v ) correct label (y i v) and the error function L v Then, AutoDO15a calculates the calculated error function L v λ i Based on the gradient obtained by partial differentiation with respect to λ i By updating, the hyperparameter λ is optimized.

[0086] 7A is a diagram illustrating an example of an evaluation dataset according to Example 1. FIG. 7B is a diagram illustrating an example of a training dataset according to Example 1.

[0087] The evaluation dataset samples shown in Figure 7A are image data with digit numbers 1, 6, 7, and 9. Figure 7A illustrates a dataset sample with a distribution similar to that of the test dataset for evaluating the performance of the trained DNN 13a.

[0088] On the other hand, the samples of the training dataset shown in Figure 7B are image data with digit numbers 0, 0, 1, 2, 6, 9, etc. Figure 7B illustrates biased samples and the correct labels for the samples that contain noise (label noise).

[0089] Fig. 8A is a diagram conceptually illustrating an example of a function for performing the first optimization process according to the embodiment 1. Fig. 8B is a diagram illustrating a specific function of the LOSS 14b illustrated in Fig. 8A.

[0090] Data Sampler 11b indicates an acquisition processing function performed by the acquisition unit 11 in the first optimization processing, and extracts a sample x from the learning data set indicated by i to U(1, N). i Get.

[0091] Augment12b indicates the data augmentation processing function performed by the data augmentation processing unit 12 in the first optimization process, and g A (λ A ) is a function that indicates the data augmentation process and is the hyperparameter λ AIn the example shown in FIG. 8A, g in Augment 12b is A (λ A ) is the gradient ∂L in the second optimization process. v / ∂λ gives the hyperparameter λ A has been updated.

[0092] Augment12b is the sample x acquired by Data Sampler11b. i For g A (λ A ) to perform data augmentation processing. Then, the Data Sampler 11b performs data augmentation processing on the augmented sample x i A is output to DNN13b.

[0093] DNN13b represents a neural network having a plurality of parameters θ of DNN13, and f(θ) represents a function indicating a prediction process by the neural network. In the example shown in FIG. 8A, in the first optimization process, DNN13b uses the augmented sample x output from Augment12b as i A Then, the DNN 13b performs data prediction processing by applying f(θ) to the extended sample x obtained by the prediction processing. i A Correct label for

number

[0094] LOSS14b indicates a processing function for calculating an error function (first error function) in the first optimization process. s,w (λ s,w ) denotes the error function (first error function) in the first optimization process using weights (loss weights) for each sample to capture the bias of the sample and soft labels to deal with noisy ground truth labels. s、w denotes the loss weights and soft label hyperparameters, which are implicit functions.

[0095] In addition, the hyperparameter λ s、w is the gradient ∂L in the second optimization process. v It is updated by / ∂λ.

[0096] More specifically, the LOSS 14b has the functions of a Soft-label 141, a Weight KL Div 142, and a Reweight 143, as shown in FIG. 8B.

[0097] The soft-label 141 soft-labels the correct value indicated by the input correct label using, for example, Gumbel softmax or softmax, and outputs the soft-label. s (λ s ) represents a soft labeling function to deal with noisy ground truth labels, and the hyperparameter λ s is made implicit.

[0098] In the example shown in FIG. 8B, the soft-label 141 selects a sample x from a training dataset represented by i to U(1, N). i The correct label for y i is input, and the correct answer label y is i Soft-label141 is the soft-labeled soft label y i s is output to Weight KL Div 142. Here, for example, Soft-label 141 is i s = g s (y i , λ i s )=softmax(λ i S ) to find the correct label y i soft label y for each sample i s where λ i S is a hyperparameter of the soft label for each sample, which is implicit.

[0099] Weight KL Div 142 calculates an error function (first error function) in the first optimization process using a weight (loss weight) for each sample and a soft label for handling noisy correct labels. More specifically, Weight KL Div 142 calculates an error function (first error function) in the first optimization process using a soft label y output by Soft-label 141. i s and the correct labels output by DNN13b

number

number

[0100] Reweight143 uses an activation function such as softplus to generate weights for each sample to capture sample bias. i Calculate g w (λ w ) denotes a function that calculates the weight (loss weight) for each sample that is weighted to the error function L to capture the bias of the sample.

[0101] In the example shown in FIG. 8B, Reweight 143 is w (L i , λ i w )=w i L i The weight w in the formula i Calculates and outputs to Weight KL Div142. For example, Reweight143 calculates w i= 1.44×softplus(λ i W ) weights for each sample w i where λi W is a hyperparameter of the weight (loss weight) for each sample, and is implicit.

[0102] In the first optimization process, the weighted first error function calculated as above is

number

[0103] Next, an optimization process procedure for alternately performing the first optimization process and the second optimization process according to the first embodiment will be described with reference to FIG.

[0104] Fig. 9 is a diagram illustrating pseudocode of an algorithm 1 showing an example of an optimization process procedure for alternately performing a first optimization process and a second optimization process according to Example 1. Note that variables such as λ and functions such as f(θ) shown in Fig. 9 are as explained in Fig. 8A and Fig. 8B. In Fig. 9, samples of the training dataset described above are expressed as data.

[0105] Algorithm 1 shown in FIG. 9 is executed by the processor of the learning device 10, for example.

[0106] As shown in FIG. 9, the procedure in the first line specifies that the parameter θ and the hyperparameter λ are initialized.

[0107] The second line specifies that the procedure from the third line to the nineteenth line, that is, the first optimization process and the second optimization process, are repeated up to the number of epochs.

[0108] The third line specifies that the procedure from the fourth line to the ninth line, that is, the first optimization process, is repeated up to the number of batches.

[0109] In the fourth line, the training dataset D trainIt is specified that batches are sampled from a set of subsets. A batch corresponds to the data contained in a subset (the sample mentioned above) when the training dataset is divided into several subsets.

[0110] Line 5 specifies that the sampled data x is subjected to data augmentation processing.

[0111] In line 6, the data x that was augmented in line 5 is used. A It is stipulated that the correct label is predicted from

[0112] In line 7, the training dataset D in line 4 is used. train The correct label for the data x sampled from the training dataset D train It is specified that soft labels are generated from the ground truth labels obtained from

[0113] In line 8, the weighted error function is calculated using the KL divergence and differentiated with respect to the parameter θ (∇ θ ) is stipulated.

[0114] Line 9 specifies that the parameter θ of the function f(θ) that indicates the prediction process by the neural network is updated.

[0115] Line 10 specifies that if the number of epochs exceeds E, processing from line 11 onwards will be carried out.

[0116] Line 11 specifies that the procedure from lines 12 to 19, that is, the second optimization process, is repeated up to the number of batches.

[0117] On line 12, the training dataset D train It is specified that batches are sampled from

[0118] On line 13, the evaluation dataset D valIt is specified that batches are sampled from

[0119] In line 14, the evaluation dataset D sampled in line 13 is used. val Data X v It is stipulated that the correct label is predicted from

[0120] On line 15, the training dataset D sampled on line 12 is used. train The data is augmented and processed into data X. A It is stipulated that the correct label is predicted from

[0121] On line 16, the training dataset D is used on line 12. train The correct label for the data x sampled from the training dataset D train It is specified that soft labels are generated from the ground truth labels obtained from

[0122] In lines 17 and 18, the error function L shown in line 18 is v and the error function L are calculated, and the error function L v Differentiate with respect to the hyperparameter λ (∇ λ ) where the error function L v is the evaluation dataset D val The error function L evaluates the error between the data in line 16 and the correct label predicted in line 14. The error function L is calculated by dividing the soft label generated in line 16 and the data augmented data X predicted in line 15. A The error function is weighted to evaluate the error between the correct label and the original one.

[0123] Line 18 specifies that the hyperparameter λ is updated. Note that HO shown on line 19 is an abbreviation for hyperparameter optimization.

[0124] By executing the algorithm 1 defined in this way, the first optimization process and the second optimization process can be performed alternately.

[0125] [1.4 Effects, etc.] As described above, the learning device 10 and data augmentation method according to this embodiment can optimize hyperparameters, thereby increasing the number of samples in the learning dataset so as to obtain a highly accurate AI model.

[0126] More specifically, in the neural network training process, which optimizes multiple parameters of the neural network, the gradient obtained by partially differentiating the hyperparameters for each sample can be used to minimize the distribution discrepancy between the data-augmented training dataset and the test dataset. Therefore, even when training a neural network using training data that has been augmented by data augmentation processing, the number of samples in the training dataset can be increased so that the generated discriminant surface by training can approach the ideal discriminant surface. In this way, the number of samples in the training dataset can be increased to obtain a highly accurate AI model.

[0127] Conventionally, data augmentation processing required tuning large-scale hyperparameters that determine how training datasets are transformed. Furthermore, there is also the problem that even if multiple neural network parameters are trained using an incomplete training dataset that includes biased data distribution and inaccurate correct labels, a highly accurate AI model cannot be obtained. Therefore, dual optimization calculations are required: one for training the neural network, and the other for optimizing the large-scale hyperparameters of the training dataset for data augmentation processing. For this reason, optimizing hyperparameters required significant computational overhead.

[0128] In contrast, the learning device 10 and data augmentation method according to the present embodiment alternately optimize multiple neural network parameters and hyperparameters for data augmentation processing for each sample, thereby enabling optimization of hyperparameters for each sample while learning multiple neural network parameters. This also has the effect of reducing the computational costs required for optimizing multiple neural network parameters and optimizing hyperparameters for each sample.

[0129] Here, in the learning device 10 and data augmentation method according to this embodiment, the hyperparameters are updated based on the gradient so as to reduce the deviation between the distribution of the training dataset after data augmentation, which is obtained by performing a data augmentation process on the training dataset, and the distribution of the evaluation dataset.

[0130] This allows the hyperparameters to be updated so as to minimize the distribution discrepancy between the data-augmented training dataset and the test dataset. Therefore, even if a neural network is trained using training data that has been augmented by data augmentation processing, the generated discriminant surface through training can be brought closer to an ideal discriminant surface.

[0131] In the learning device 10 and data augmentation method according to the present embodiment, the hyperparameters are implicit in at least the function representing the data augmentation process, and the neural network has a fully differentiable function. This allows partial derivatives of the hyperparameters to be calculated using a differentiation method for the implicit function.

[0132] Furthermore, the learning device 10 and data augmentation method according to this embodiment calculate a first error function in the first optimization process using soft labels to accommodate noisy correct labels. This makes it possible to suppress noise in the correct labels for samples included after data augmentation. Therefore, using the gradient obtained by partially differentiating the hyperparameters for each sample, the hyperparameters for each sample can be optimized to minimize the distribution discrepancy between the data-augmented training dataset and the test dataset.

[0133] Furthermore, in the learning device 10 and data augmentation method according to this embodiment, the calculated first error function is weighted using a weight for each sample to capture the bias of the sample.

[0134] This allows us to capture and suppress sample bias. Therefore, we can use the gradient obtained by partially differentiating the hyperparameters for each sample to optimize the hyperparameters for each sample so as to minimize the distributional difference between the data-augmented training dataset and the test dataset.

[0135] (Comparative Example) A data extension method according to a comparative example and its problems will be described below.

[0136] FIG. 10 is a diagram for explaining a problem that occurs when data expansion is performed using hyperparameters shared among all data according to a comparative example.

[0137] Figure 10 shows the discrimination surface (discrimination line) when a neural network f(θ) is trained using a training dataset that has been augmented by data augmentation processing g(λ) using a hyperparameter λ shared among all data. Also, in Figure 10, the smallest solid circles in the regions above and below the discrimination surface in the central dotted square conceptually represent the distribution of the training data. The dotted circles in the upper and lower regions conceptually represent the distribution of the training dataset that has been augmented by data augmentation processing g(λ). The large solid circle in the dotted square conceptually represents the distribution of the test dataset.

[0138] When data is augmented by the data augmentation process g(λ) according to the comparative example shown in Fig. 10, all data in the training dataset, i.e., the distribution of digits, is increased evenly. The discrimination surface obtained by training using the data-augmented training dataset in this way crosses part of the test dataset, which poses a problem in that it cannot correctly discriminate the test dataset.

[0139] In other words, the data augmentation method according to the comparative example has a problem in that the generated classification surface by learning deviates significantly from the ideal boundary line. This problem is thought to arise because the data is augmented according to the distribution of the training dataset.

[0140] The problem of data extension in the comparative example will be specifically described below.

[0141] Figure 11 is a diagram conceptually explaining the generation of a discrimination surface through learning processing. As shown in Figure 11, in a classification problem, when AI performs learning processing using a training dataset consisting of various dog or cat images and their correct labels, it extracts features and generates a discrimination surface d1. As can be seen from Figure 11, the generation of discrimination surface d1 conceptually corresponds to drawing a boundary line to distinguish between dogs and cats.

[0142] FIG. 12A is a diagram for conceptually explaining the data extension process.

[0143] As shown in Figure 12A, the data augmentation process for a training dataset is a process that artificially increases or pads the data that constitutes the training dataset by performing conversion processes such as rotation, zoom, translation, and color conversion on the data in the training dataset.

[0144] Fig. 12B is a diagram conceptually illustrating a problem in generating a discrimination surface by a learning process when the number of data included in the training dataset is small. Fig. 12C is a diagram conceptually illustrating an example of generating a discrimination surface by a learning process using a training dataset that has been data-augmented by a data-augmentation process.

[0145] As shown in Figure 12B, when the number of data points in the training dataset is small, there are large gaps between the dog and cat data, so the AI ​​does not know where to draw the boundary line to distinguish between dogs and cats, as shown by the discrimination surfaces d1, d2, and d3.

[0146] On the other hand, as shown in Figure 12C, if the data is properly augmented by the data augmentation process, the gaps shown in Figure 12B can be pseudo-filled with the augmented data, making it easier for the AI ​​to draw a boundary line that distinguishes between dogs and cats, as shown by the discrimination surface d1.

[0147] Next, the issues in the data augmentation process will be described.

[0148] Fig. 13 is a diagram for explaining a problem in the data augmentation process according to the comparative example. Fig. 13 shows that if data augmentation is not performed properly, the position of the discrimination plane generated by learning will deviate from the ideal discrimination plane.

[0149] That is, in FIG. 13, by data augmenting cat data 51, the distribution of the training dataset for the augmented cat data becomes distribution Q1. Similarly, by data augmenting dog data 52, the distribution of the training dataset for the augmented dog data becomes distribution Q2. In such a case, by learning the augmented training dataset, the AI ​​generates a discrimination surface d5 that is positioned away from the ideal position on discrimination surface d4. Then, if dog image 50, which is the test dataset, is positioned away from distribution Q2 of the augmented training dataset for the augmented dog data, the AI ​​will classify dog ​​image 50, which is the test dataset, as a cat.

[0150] In this way, when data expansion is performed according to the distribution of the training dataset, the position of the discrimination surface generated by training may deviate from the ideal discrimination surface.

[0151] FIG. 14 is a diagram conceptually illustrating a discrimination surface generated using a training dataset that has been data-augmented by the data augmentation process according to the present disclosure.

[0152] In the present disclosure, the training dataset is subjected to data augmentation processing so as to match the distribution of the test dataset. More specifically, cat data 51 is augmented to the distribution Q of the test dataset for cats. t1 By augmenting the data to match the distribution Q3, the distribution of the training dataset for cat data after data augmentation can be made to match the distribution Q3. Similarly, by augmenting the data for dogs 52 to match the distribution Q3 of the test dataset for cats t2 By augmenting the data to match this, the distribution of the training dataset for dog data after data augmentation can be made to be distribution Q4.

[0153] This allows the AI ​​to generate a discrimination surface d6 at a position close to the position of the ideal discrimination surface d4.

[0154] (Experimental example) The effectiveness of the data augmentation method of the present disclosure was verified using the Street View House Numbers (SVHN) Dataset, and the verification results will be described as an experimental example.

[0155] FIG. 15 is a diagram showing the verification results obtained by evaluating the error rate using a test dataset after learning using a training dataset according to an experimental example.

[0156] The SVHN dataset incorporates a large amount of labeled data, including images with over 600,000 digits. It consists of a training dataset with 73,257 digits and a test dataset with 26,032 digits. The SVHN dataset also includes 531,131 slightly easier examples that can be used as additional training data.

[0157] In this experimental example, 32% of the training dataset of the SVHN dataset was divided and used as the evaluation dataset.

[0158] In addition, in this experimental example, verification was performed using a training dataset without data bias and label noise, and a training dataset with data bias or label noise. In Figure 15, the class imbalance ratio (IR) and label noise ratio (NR), which indicate data bias, indicate the presence or absence of data bias and label noise. Note that the training dataset for the SVHN dataset corresponds to a training dataset without data bias and label noise.

[0159] For example, if IR-NR is "100-0.1", it means that a random 10% (NR=0.1) of the total image data is randomly inverted, which means there is label noise. Also, for example, the number of image data showing {0-4} is 100 times the number of image data showing {5-9}, which means that the number of class samples is uneven, i.e., the data is biased.

[0160] Therefore, in FIG. 15, a training dataset with IR-NR of "1-0.0" corresponds to a training dataset with no data bias and no label noise. A training dataset with IR-NR of "100-0.0" corresponds to a training dataset with data bias but no label noise. Similarly, a training dataset with IR-NR of "1-0.1" corresponds to a training dataset with no data bias but with label noise. A training dataset with IR-NR of "100-0.1" corresponds to a training dataset with data bias and label noise.

[0161] Also, in FIG. A (ours), λ A、W (ours), λ A、W、S (ours) indicates the data augmentation method of the present disclosure. As the data augmentation method of the present disclosure, the learning process, i.e., the first optimization process and the second optimization process, were performed using Algorithm 1 shown in FIG. 9. The second optimization process started after the 50th epoch.

[0162] Also, in FIG. A (ours) is the hyperparameter λ, which is a variable that specifies the data conversion process for each sample used when performing data augmentation processing. A It also shows that the optimization was performed only for λ A、W、S (ours) shows the optimization of the hyperparameters for data augmentation, loss weights, and soft labels. A、W (ours) indicates that the hyperparameters of data augmentation and loss weights were optimized. These optimization processes were performed using Algorithm 1 shown in Figure 9.

[0163] In addition, in FIG. 15, λ ASHA The verification results of (ours) are shown in Fig. 15. ASHA (ours) is a hyperparameter λ that is shared among all data, not per data. ASHA This indicates that the sigma-based algorithm is optimized for

[0164] 15 also shows the verification results of Baseline and FAA as comparative examples. More specifically, Baseline shows the verification results when data augmentation processing is performed using only standard conversion processing such as random cropping, horizontal flipping, and erasure, as described in Non-Patent Document 1. FAA (Fast AutoAugment) shows the verification results when data augmentation processing is performed using the method described in Non-Patent Document 2.

[0165] As can be seen from Figure 15, when using a training dataset with data imbalance or label noise, λ ASHA ,Compared with the validation results of Baseline and FAA, λ A (ours) and λ A、W The verification results of (ours) have achieved the same or lower error rate (top 1 error rate).

[0166] Furthermore, λ ASHA ,Compared with the validation results of Baseline and FAA, λ A、W、S The validation results of (ours) show that an improved error rate (top-1 error rate) is achieved regardless of whether the training dataset contains data bias or label noise.

[0167] From the above, the hyperparameter λ A、W、S Not only are we optimizing for (ours), but we also optimize the hyperparameter λ A It can be seen that optimization is also effective for (ours).

[0168] FIG. 16 is a diagram showing an accuracy learning curve when learning is performed using a learning data set according to an experimental example. FIG. 16 shows an accuracy learning curve (c) when learning is performed using a learning data set in which IR-NR is "1-0.0". Note that in FIG. 16, the above λ A、W、S The optimization process for (ours) corresponds to AutoDO, and the above λ A、W、SIn the optimization process (data augmentation method of the present disclosure) for (ours), the start of the second optimization process is indicated as AutoDO start. Also, AutoDO start starts after the 50th epoch.

[0169] In addition, as a comparative example, FIG. 16 also shows accuracy learning curves (a) and (b) when the above-mentioned Baseline and FAA are trained using the training data set according to the experimental example.

[0170] As can be seen from Fig. 16, the data augmentation method of the present disclosure does not improve performance unless AutoDO, i.e., the second optimization process, is started. Also, the data augmentation method of the present disclosure performs AutoDO, i.e., the second optimization process, thereby reducing the hyperparameter λ A、W、S It can be seen that this can be effectively optimized, resulting in a dramatic improvement in performance.

[0171] (Possibilities for other embodiments) Although the data augmentation method and learning device of the present disclosure have been described above in the embodiments, there are no particular limitations on the entity or device that performs each process. The processes may be performed by a processor embedded in a specific device that is locally located. The processes may also be performed by a cloud server or the like that is located in a different location from the local device.

[0172] It should be noted that the present disclosure is not limited to the above-described embodiments. For example, the present disclosure may be embodied in another embodiment realized by any combination of the components described in this specification or by excluding some of the components. Furthermore, the present disclosure also includes modifications obtained by applying various modifications to the above-described embodiments that would occur to a person skilled in the art without departing from the spirit of the present disclosure, i.e., the meaning of the wording of the claims.

[0173] The present disclosure also includes the following cases:

[0174] (1) The above-mentioned device is specifically a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. Each device achieves its function when the microprocessor operates in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer to achieve a predetermined function.

[0175] (2) Some or all of the components constituting the above-described device may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured including a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0176] (3) Some or all of the components constituting the above-mentioned device may be configured as an IC card or a standalone module that can be attached to or detached from each device. The IC card or module may be a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates according to a computer program. The IC card or module may be tamper-resistant.

[0177] (4) The present disclosure may also be embodied as the methods described above, as a computer program for implementing these methods on a computer, or as a digital signal comprising the computer program.

[0178] (5) Furthermore, the present disclosure may be the computer program or the digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), semiconductor memory, etc. Alternatively, the present disclosure may be the digital signal recorded on such a recording medium.

[0179] Furthermore, the present disclosure may involve transmitting the computer program or the digital signal via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, or the like.

[0180] The present disclosure may also be a computer system including a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating in accordance with the computer program.

[0181] The program or the digital signal may also be implemented by another independent computer system by recording it on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like. [Industrial Applicability]

[0182] The present disclosure can be used for a data augmentation method, a learning device, and a program for a training dataset used in training a neural network, and in particular for a data augmentation method, a learning device, and a program that can reduce computational costs and perform neural network training and sample-specific hyperparameter optimization. [Explanation of symbols]

[0183] 10 Learning Device 11 Acquisition Department 11b Data Sampler 12 Data expansion processing section 12a Process 12b Augment 14 Error calculation section 15 Optimization Section 15a AutoDO 141 Soft-label 142 Weight KL Div 143 Reweight 1000 computers 1001 Input Device 1002 Output device 1004 Internal Storage 1007 Reading device 1008 Transmitting and receiving device 1009 Bus

Claims

1. A data augmentation method for a training dataset used in training a neural network, the data augmentation method being executed by a computer, comprising: alternately performing a first optimization process for optimizing a plurality of parameters including weights in the neural network and a second optimization process for optimizing hyperparameters, which are variables that define data conversion processing for each sample used when performing data augmentation processing; In the first optimization process, performing the data augmentation process on a first sample included in the training dataset to obtain a first augmented sample; causing the neural network to predict a first augmented label from the first augmented sample; calculating a first error function for evaluating an error between the first augmented label and a first correct label indicating a correct answer for the first sample included in the training dataset; updating the plurality of parameters based on the calculated first error function; In the second optimization process, obtaining a second sample from an evaluation dataset, the evaluation dataset having a distribution similar to that of a test dataset for evaluating performance of the neural network; causing the neural network with the updated parameters to predict a second label from the second sample; Calculating a second error function for evaluating an error between the second label and a second correct label indicating a correct answer for the second sample included in the evaluation dataset; updating the hyperparameters based on a gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters; When updating the hyperparameters, updating the hyperparameters based on the gradients so as to reduce a deviation between a distribution of the training dataset after data augmentation, which is obtained by performing the data augmentation process on the training dataset, and a distribution of the evaluation dataset; Data augmentation methods.

2. The hyperparameters are implicit in a function that indicates the data augmentation process; The neural network is a fully differentiable function. The data extension method according to claim 1 .

3. When calculating the first error function, the first error function is calculated by using Kullback-Leibler divergence to evaluate an error between a first correct soft label obtained by soft-labeling a correct value indicated by the first correct label and the first extended label. The data extension method according to any one of claims 1 to 2.

4. When calculating the first error function, a weighting calculated for each sample is further performed on the first error function. The data extension method according to any one of claims 1 to 3.

5. A learning device for performing a data augmentation method for a learning dataset used for learning a neural network, comprising: A processor and a memory, The processor uses the memory to: alternately performing a first optimization process for optimizing a plurality of parameters including weights in the neural network and a second optimization process for optimizing hyperparameters, which are variables that define data conversion processing for each sample used when performing data augmentation processing; In the first optimization process, performing the data augmentation process on a first sample included in the training dataset to obtain a first augmented sample; causing the neural network to predict a first augmented label from the first augmented sample; calculating a first error function for evaluating an error between the first augmented label and a first correct label indicating a correct answer for the first sample included in the training dataset; updating the plurality of parameters based on the calculated first error function; In the second optimization process, obtaining a second sample from an evaluation dataset, the evaluation dataset having a distribution similar to that of a test dataset for evaluating performance of the neural network; causing the neural network with the updated parameters to predict a second label from the second sample; Calculating a second error function for evaluating an error between the second label and a second correct label indicating a correct answer for the second sample included in the evaluation dataset; updating the hyperparameters based on a gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters; When updating the hyperparameters, updating the hyperparameters based on the gradients so as to reduce a deviation between a distribution of the training dataset after data augmentation, which is obtained by performing the data augmentation process on the training dataset, and a distribution of the evaluation dataset; Learning device.

6. A program for causing a computer to execute a data augmentation method for a training dataset used in training a neural network, A first optimization process for optimizing a plurality of parameters including the neural network weights and a second optimization process for optimizing hyperparameters, which are variables that define data conversion processing for each sample used when performing data augmentation processing, are alternately performed; In the first optimization process, performing the data augmentation process on a first sample included in the training dataset to obtain a first augmented sample; causing the neural network to predict a first augmented label from the first augmented sample; calculating a first error function for evaluating an error between the first augmented label and a first correct label indicating a correct answer for the first sample included in the training dataset; updating the plurality of parameters based on the calculated first error function; In the second optimization process, obtaining a second sample from an evaluation dataset, the evaluation dataset having a distribution similar to that of a test dataset for evaluating performance of the neural network; causing the neural network with the updated parameters to predict a second label from the second sample; Calculating a second error function for evaluating an error between the second label and a second correct label indicating a correct answer for the second sample included in the evaluation dataset; updating the hyperparameters based on a gradient obtained by partially differentiating the calculated second error function with respect to the hyperparameters; When updating the hyperparameters, updating the hyperparameters based on the gradients so as to reduce a deviation between a distribution of the training dataset after data augmentation obtained by performing the data augmentation process on the training dataset and a distribution of the evaluation dataset; A program that a computer runs.

Images