Data generation device, data generation method, and program

Abstract

The present invention provides a data generation device comprising: a data acquisition means that acquires, from among a plurality of pieces of data classified into classes, first data and second data classified into the same class as the first data; a perturbation acquisition means that, using a neural network which includes a partial network, a first normalization layer, and a second normalization layer and which outputs a feature amount, acquires such a perturbation that the distance between a first feature amount and a second feature amounts is as short as possible, the first feature amount being the feature amount when data obtained by adding the perturbation to the first data is input to the neural network and is normalized by the second normalization layer, the second feature amount being the feature amount when the second data is input to the neural network and is normalized by the first normalization layer; and a data processing means that generates the data obtained by adding the obtained perturbation to the first data.

Description

Technical Field

[0001] The present invention relates to a data generation device, a data generation method, and a recording medium.

Background Art

[0002] Training data used for learning of a neural network may be processed to generate new training data. For example, Patent Document 1 describes generating image data of a product by synthesizing image data of a good product and image data of a defective product of the product, such as generating image data of a defective product.

Prior Art Documents

Patent Documents

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2021-11 9 442

Summary of the Invention

Problems to be Solved by the Invention

[0004] When generating training data for a neural network that outputs feature amounts, it is preferable to be able to generate training data that can improve the accuracy of determination using the feature amounts output by the neural network.

[0005] An example of an object of the present invention is to provide a data generation device, a data generation method, and a recording medium capable of solving the above-described problems.

Means for Solving the Problems

[0006] According to a first aspect of the present invention, the data generation apparatus includes: data acquisition means for acquiring first data and second data classified in the same class as the first data from among a plurality of classified data; perturbation acquisition means for acquiring a perturbation such that the distance between a first feature, which is a feature obtained when the first data is perturbed and normalized by the second normalization layer is input to the neural network and normalized by the second normalization layer is performed, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer is performed, is as small as possible; and data processing means for generating data by adding the obtained perturbation to the first data.

[0007] According to a second aspect of the present invention, a data generation method includes a computer acquiring a first data and a second data classified in the same class as the first data from among a plurality of classified data, and using a neural network that includes a subset network, a first normalization layer and a second normalization layer and outputs feature quantities, the computer acquiring a perturbation such that the distance between a first feature quantity, which is the feature quantity obtained when the data with a perturbation added to the first data is input to the neural network and normalized by the second normalization layer, and a second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is made as small as possible, and generating data by adding the obtained perturbation to the first data.

[0008] According to a third aspect of the present invention, the recording medium is a recording medium that records a program for causing a computer to perform the following actions: acquire first data and second data classified in the same class as the first data from among a plurality of classified data; acquire a perturbation such that the distance between a first feature, which is the feature obtained when the first data is perturbed and normalized by the second normalization layer is input to the neural network, and a second feature, which is the feature obtained when the second data is input to the neural network and normalized by the first normalization layer is performed, is as small as possible; and generate data by adding the obtained perturbation to the first data. [Effects of the Invention]

[0009] According to the present invention, when generating training data for a neural network that outputs features, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the configuration of a learning device according to the first embodiment. [Figure 2] This figure shows an example of a neural network stored in the model memory unit according to the first embodiment. [Figure 3] This figure shows an example of the procedure for the learning device according to the first embodiment to perform training on a neural network. [Figure 4] This figure shows an example of the configuration of a data generation device according to the second embodiment. [Figure 5] This figure shows an example of the procedure for generating non-adversarial samples using a data generation device according to the second embodiment. [Figure 6] This figure shows an example of the configuration of a learning device according to the third embodiment. [Figure 7]This figure shows an example of the processing procedure in the data generation method according to the fourth embodiment. [Figure 8] This is a schematic block diagram showing the configuration of a computer according to at least one embodiment. [Modes for carrying out the invention]

[0011] <First Embodiment> The following describes embodiments of the present invention, but these embodiments are not intended to limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] Figure 1 is a diagram showing an example of the configuration of a learning device according to the first embodiment. In the configuration shown in Figure 1, the learning device 100 comprises a communication unit 110, a display unit 120, an operation input unit 130, a storage unit 180, and a processing unit 190. The storage unit 180 comprises a model storage unit 181. The model storage unit 181 comprises a common parameter storage unit 182, a first normalization layer parameter storage unit 183-1, and a second normalization layer parameter storage unit 183-2. The processing unit 190 comprises a data acquisition unit 191, a feature calculation unit 192, a perturbation acquisition unit 193, a data processing unit 194, and a parameter update unit 195. The first normalization layer parameter storage unit 183-1 and the second normalization layer parameter storage unit 183-2 are collectively referred to as the normalization layer parameter storage unit 183.

[0013] The learning device 100 trains a neural network. In particular, the learning device 100 processes the training data to generate new training data and uses the obtained training data to train the neural network. The learning device 100 may be configured using a computer such as a personal computer (PC) or a workstation (WS).

[0014] The following explanation will use the case where the learning device 100 uses classified image data as training data as an example. However, the training data used by the learning device 100 is not limited to image data, but can be various types of data that can be processed as described later. For example, the learning device 100 may use classified audio data as training data.

[0015] Furthermore, in the following discussion, when distinguishing between training data as a set of data and training data as individual data points within that set, the training data as a set of data will also be referred to as the training dataset. Individual training data points will be described as individual training data, one training data point, or multiple training data points.

[0016] The communication unit 110 communicates with other devices. For example, the communication unit 110 may receive training data for learning a neural network from other devices.

[0017] The display unit 120 has a display screen such as a liquid crystal panel or an LED (Light Emitting Diode) panel, and displays various images. For example, the display unit 120 may display information related to neural network learning, such as displaying an image shown by image data obtained by processing image data included in the training dataset.

[0018] The operation input unit 130 is configured to include, for example, input devices such as a keyboard and a mouse, and accepts user operations. For example, the operation input unit 130 may accept user operations related to neural network training, such as accepting user operations to set values ​​for hyperparameters in image data processing.

[0019] The memory unit 180 stores various types of data. The memory unit 180 is configured using the memory devices provided by the learning device 100. The model memory unit 181 stores a neural network as a machine learning model. The neural network stored in the model memory unit 181 receives image data as input and outputs the feature quantities of the input image data (the image features represented by the image data).

[0020] Figure 2 shows an example of a neural network stored in the model memory unit 181. The neural network 201 shown in Figure 2 is configured as a type of convolutional neural network (CNN) and includes an input layer 210, a convolutional layer 221, an activation layer 222, a pooling layer 223, a first normalization layer 230-1, a second normalization layer 230-2, a fully connected layer 240, and an output layer 250. The first normalization layer 230-1 and the second normalization layer 230-2 are collectively referred to as the normalization layer 230.

[0021] In the example shown in Figure 2, the data flow is arranged in the following order from upstream: input layer 210, convolutional layer 221, activation layer 222, and pooling layer 223, with one or more combinations of these layers being arranged in that order. Downstream from there, a fully connected layer 240 and an output layer 250 are provided. Furthermore, in each combination of the convolutional layer 221, the activation layer 222, and the pooling layer 223, a first normalization layer 230-1 and a second normalization layer 230-2 are arranged in parallel between the activation layer 222 and the pooling layer 223. The number of channels in a neural network 201 is not limited to a specific number.

[0022] Of the various parts of the neural network 201, the parts other than the first normalization layer 230-1 and the second normalization layer 230-2 are also called the common part or subset network. In the example in Figure 2, the combination of the input layer 210, the convolutional layer 221, the activation layer 222, the pooling layer 223, the fully connected layer 240, and the output layer 250 is an example of the common part.

[0023] The input layer 210 receives input data for the neural network 201. The convolutional layer 221 performs a convolution operation on the data input to itself. The convolutional layer 221 may also perform padding to adjust the data size. The activation layer 222 applies an activation function to the data input to itself. The activation function used by the activation layer 222 is not limited to a specific function. For example, a Rectified Linear Function (ReLU) may be used as the activation function, but is not limited to this. The pooling layer 223 performs pooling on the data input to itself.

[0024] The first normalization layer 230-1 normalizes the data it receives as input. This normalization is the same as that used in batch normalization, where the first normalization layer 230-1 transforms the data so that the mean and variance of the data in a given group are predetermined values. For example, to set the mean of a group of data to 0 and the variance to 1, the first normalization layer 230-1 calculates the mean and variance of the data in the group to be normalized, subtracts the mean from each data point, and then divides the resulting value by the variance.

[0025] The mean after normalization by the first normalization layer 230-1 is not limited to 0, and the variance is not limited to 1. For example, if α is a real number and β is a positive real number, the first normalization layer 230-1 may be configured to normalize so that the group mean becomes α and the variance becomes β. Furthermore, these values ​​of α and β may be the subject of learning. The values ​​of α and β may be set by learning for each first normalization layer 230-1.

[0026] The target mean and variance when the first normalization layer 230-1 normalizes data correspond to examples of the parameters of the first normalization layer 230-1. The target mean when the first normalization layer 230-1 normalizes data is also called the first mean. The target variance when the first normalization layer 230-1 normalizes data is also called the first variance.

[0027] The second normalization layer 230-2 normalizes the data input to itself. The normalization process performed by the second normalization layer 230-2 is the same as the normalization process performed by the first normalization layer 230-1 described above. The target mean and variance when the second normalization layer 230-2 normalizes data correspond to examples of the parameters of the second normalization layer 230-2. The target mean when the second normalization layer 230-2 normalizes data is also called the second mean. The target variance when the second normalization layer 230-2 normalizes data is also called the second variance.

[0028] Depending on the input data to the neural network 201, either the first normalization layer 230-1 or the second normalization layer 230-2 is selectively used. When image data included in the original training dataset (unprocessed image data) is input to the neural network 201, the first normalization layer 230-1 is used. On the other hand, when image data generated by processing by the learning device 100 is input to the neural network 201, the second normalization layer 230-2 is used. The switching of such normalization layers 230 may be performed, for example, by an instruction from the feature calculation unit 192 that inputs data to the neural network 201.

[0029] The distribution of unprocessed data and processed data differs, and if both are normalized using the same normalization layer, the full benefits of normalization may not be obtained. In contrast, in neural network 201, it is expected that the normalization effect can be obtained by selectively using either the first normalization layer 230-1 or the second normalization layer 230-2, thereby enabling efficient training of neural network 201.

[0030] The activation layer 222 may output data to only one of the first normalization layer 230-1 or the second normalization layer 230-2, thereby enabling the switching of the normalization layer 230. In this case, only the layer that receives data input from the activation layer 222 may receive parameter updates through learning and output data to the pooling layer 223.

[0031] The neural network 201 when the first normalization layer 230-1 is used is also referred to as the neural network 201 using the first normalization layer 230-1. The neural network 201 when the second normalization layer 230-2 is used is also referred to as the neural network 201 using the second normalization layer 230-2. Furthermore, when operating the trained neural network 201, the first normalization layer 230-1 may always be used.

[0032] The fully connected layer 240 converts the data it receives as input into data corresponding to the output data of the neural network 201. The output layer 250 outputs the output data from the neural network 201. For example, the output layer 250 may apply an activation function such as a softmax function to the data from the fully connected layer 240 before outputting it.

[0033] Alternatively, the fully connected layer 240 may generate the output data for the neural network 201, and the output layer 250 may output the data from the fully connected layer 240 as is. In this case, the fully connected layer 240 may also function as the output layer 250, outputting the data directly to the outside of the neural network 201.

[0034] However, the configuration of the machine learning model stored in the model storage unit 181 is not limited to a specific configuration. For example, when the model memory unit 181 stores a convolutional neural network as a machine learning model, the configuration and number of layers in the convolutional neural network can be varied. For instance, the configuration of the machine learning model stored by the model memory unit 181 may be a combination of the convolutional layer 221, activation layer 222, and pooling layer 223 included in the neural network 201 in the example of Figure 2, but without the activation layer 222.

[0035] Furthermore, the location where the combination of the first normalization layer 230-1 and the second normalization layer 230-2 is provided is not limited to a specific location. For example, the combination of the first normalization layer 230-1 and the second normalization layer 230-2 may be provided only for some of the combinations of the convolutional layer 221, the activation layer 222, and the pooling layer 223. The machine learning model stored in the model memory unit 181 may be configured such that, instead of a convolutional neural network with a batch normalization layer, the number of batch normalization layers is reduced to two and they are arranged in parallel.

[0036] However, the machine learning model stored in the model memory unit 181 is not limited to convolutional neural networks, but can be any neural network to which normalization by the first normalization layer 230-1 and the second normalization layer 230-2 can be applied.

[0037] Furthermore, the implementation method of the neural network to be trained by the learning device 100 is not limited to the method by which the model storage unit 181 stores the neural network. For example, the neural network to be trained by the learning device 100 may be implemented in hardware, such as by using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). The neural network to be trained by the learning device 100 may be configured as part of the learning device 100, or it may be configured as an external component of the learning device 100.

[0038] The common parameter storage unit 182 stores the parameter values ​​of the common parts. The common parameter storage unit 182 stores the values ​​of various parameters that are the target of learning, such as the parameters of the filters for convolution operations in the convolutional layer and the activation function in the activation layer. The parameter values ​​of the common parts are also called common parameter values.

[0039] The first normalization layer parameter storage unit 183-1 stores the parameter values ​​for each first normalization layer 230-1. The first normalization layer parameter storage unit 183-1 stores the values ​​of various parameters to be learned, such as the first mean and the first variance.

[0040] The second normalization layer parameter storage unit 183-2 stores the parameter values ​​for each second normalization layer 230-2. The second normalization layer parameter storage unit 183-2 stores the values ​​of various parameters that are the target of learning, such as the second mean and the second variance.

[0041] The processing unit 190 controls various parts of the learning device 100 to perform various processes. The functions of the processing unit 190 are performed, for example, by the CPU (Central Processing Unit) of the learning device 100 reading a program from the storage unit 180 and executing it.

[0042] The data acquisition unit 191 selects from the training data the image data to be processed, and one or more image data that are classified in the same class as the image data to be processed, but are not the image data to be processed. The data acquisition unit 191 is an example of a data acquisition means. The image data selected from the training data to be processed is also referred to as the original image data. The original image data, or the image data obtained by processing the original image data, is also referred to as the first image data. Image data that is classified in the same class as the original image data, but is other than the original image data (one of them), is also referred to as the second image data.

[0043] The first image data is an example of the first data. The first data is the original data, or data obtained by processing the original data, when the training data is not limited to image data. Here, the original data is the data selected from the training data to be processed.

[0044] The second image data is an example of the second data. The second data is one of the data sets other than the original data, and is classified in the same class as the original data, when the training data is not limited to image data.

[0045] The feature extraction unit 192 uses the neural network 201 (the neural network stored in the model memory unit 181) to calculate the features of the image data. Specifically, the feature extraction unit 192 inputs the image data into the neural network 201 and obtains the features output by the neural network 201.

[0046] The perturbation acquisition unit 193 acquires perturbations for processing the first image data using the second image data. Here, the perturbation refers to the image data difference between the image data before processing and the image data after processing (image data obtained by processing). The perturbation acquisition unit 193 is an example of a perturbation acquisition means.

[0047] The perturbation acquisition unit 193 calculates perturbations such that the feature quantities of the processed image data are closer to the feature quantities of the second image data than to the feature quantities of the original image data. A perturbation that causes the feature quantities of the processed image data to be closer to the feature quantities of image data classified in the same class as the original image data than to the feature quantities of the original image data is also called an unadversarial perturbation.

[0048] The data processing unit 194 performs processing on the first image data. Specifically, the data processing unit 194 generates processed image data by adding the perturbations acquired by the perturbation acquisition unit 193 to the first image data. More specifically, the data processing unit 194 adds the pixel value of the first image data to the pixel value of the perturbation associated with that pixel for each pixel of the first image data. The data processing unit 194 is an example of a data processing means. Image data obtained by applying a non-adversarial perturbation to image data is also called a non-adversarial example (UN).

[0049] The parameter update unit 195 updates the parameter values ​​of the neural network 201 during the training of the neural network 201. The parameter update unit 195 is an example of a parameter value update means.

[0050] When the first image data (perturbed image data) is input to the neural network 201, the parameter update unit 195 updates the parameter values ​​of the subset network and the parameter values ​​of the second normalization layer 230-2. On the other hand, when the second image data (unprocessed image data) is input to the neural network 201, the parameter update unit 195 updates the parameter values ​​of the subset network and the parameter values ​​of the first normalization layer 230-1.

[0051] Similar to parameter updates in mini-batch learning, the parameter update unit 195 may update the parameter values ​​using the average value of multiple input data for each part of the neural network 201.

[0052] Non-adversarial sample x generated by data processing unit 194 UN It can be expressed as shown in equation (1).

[0053]

number

[0054] x represents the first image data. δ f UX This represents a non-adversarial perturbation acquired by the perturbation acquisition unit 193. Non-adversarial perturbation δ acquired by the perturbation acquisition unit 193 f UX This can be expressed as shown in equation (2).

[0055]

number

[0056] x b This represents the second image data. As described above, the second image data is image data included in the training data that is classified in the same class as the original image data, but is different from the original image data. S B is a non-hostile perturbation δ fUX The second image data x used for the calculation of b represents a set. n is an integer where n ≥ 1, and the set S B contains the second image data x b represents the number of. Therefore, n is the non - adversarial perturbation δ f UX The second image data x used for the calculation of b represents the number of.

[0057] θ MainBN represents the parameter of the first normalization layer 230 - 1. θ AuxBN represents the parameter of the second normalization layer 230 - 2. θ NBN represents the parameter of the common part. That is, θ NBN represents the parameter other than the normalization layer 230.

[0058] f i {θNBN,θAuxBN} (x + δ) represents the feature obtained by inputting the image data obtained by adding the perturbation δ to the first image data x into the neural network 201 using the second normalization layer 230 - 2. f i {θNBN,θAuxBN} The feature represented by (x + δ) corresponds to an example of the first feature.

[0059] Here, the learning device 100 may repeatedly calculate the non - adversarial perturbation δ by the perturbation acquisition unit 193 and generate the non - adversarial sample x by the data processing unit 194, so as to gradually update the non - adversarial sample x. That is, the learning device 100 may further perform the process of generating the non - adversarial sample x using the non - adversarial sample x generated by the data processing unit 194 as the first image data x. f UX of, and the generation of the non - adversarial sample x UN by the data processing unit 194, and gradually update the non - adversarial sample x UN That is, the learning device 100 uses the non - adversarial sample x UN generated by the data processing unit 194 as the first image data x to further perform the process of generating the non - adversarial sample x UN

[0060] ​For example, the perturbation acquisition unit 193 uses a method that repeatedly searches for a solution, such as backpropagation, to obtain the "1 / nΣ" shown in equation (2). xb∈SB φ(f i {θNBN,θAuxBN} (x+δ),f i {θNBN,θMainBN} (x b You can also try to find a perturbation δ that minimizes the value of )).

[0061] i is a non-adversarial perturbation δ f UX Calculation of and non-adversarial sample x by data processing unit 194 UN The number of iterations for generating x, i.e., non-adversarial sample x UN This represents the number of times the update has been repeated.

[0062] Furthermore, the perturbation acquisition unit 193 generates a non-adversarial perturbation δ f UX Along with the calculation, the parameter update unit 195 may update the parameter values ​​through learning of the neural network 201. Therefore, non-adversarial sample x UN With each iteration of the update, the parameter values ​​of the neural network 201 may also be updated. In equation (2), f represents the neural network 201, and f i This is a non-adversarial sample x UN This represents neural network 201 during the i-th iteration.

[0063] In the training of the neural network 201, the parameter update unit 195 performs the "1 / nΣ" shown in equation (2). xb∈SB φ(f i {θNBN,θAuxBN} (x+δ),f i {θNBN,θMainBN} (x b You can also update the parameter values ​​of neural network 201 so that the value of ))" becomes as small as possible. The training of the neural network 201 performed by the learning device 100 can be described as training the neural network 201 using image data x+δ, which is obtained by adding a perturbation δ to the first image data x.

[0064] Alternatively, the output of neural network 201 may be input to a classification-type neural network. The parameter update unit 195 may then update the parameter values ​​of neural network 201 so that the likelihood of the correct class is as large as possible. In this case, the training of the neural network 201 performed by the learning device 100 can also be described as training the neural network 201 using image data x+δ, which is obtained by adding a perturbation δ to the first image data x.

[0065] f i {θNBN,θMainBN} (x b ) is the second image data x b This represents the feature obtained by inputting this into a neural network 201 using the first normalization layer 230-1. i {θNBN,θMainBN} (x b The feature represented by ) is an example of a second feature.

[0066] φ represents the distance between two vectors. In equation (2), φ represents the distance in the feature space. Various types of distances applicable to vectors can be used as the distance φ. For example, the distance φ may be the L1 norm, the L2 norm, or the L∞ norm.

[0067] argmin is a function that outputs the value of the variable shown below argmin that minimizes the expression shown after argmin. ||δ|| p <ε represents a constraint on the perturbation δ output by argmin, stating that the magnitude of the perturbation δ is less than a given constant ε. Any type of norm can be used as the magnitude of the perturbation δ. For example, "|| || p" may represent the L1 norm, the L2 norm, or the L∞ norm.

[0068] Equation (2) shows the features of the first image data x with a perturbation δ added, and the second image data x b The set S of distances to features. B All second image data x included in b We search for a perturbation δ that minimizes the average of the terms, and then select a non-adversarial perturbation δ. f UX This indicates that it will be done that way.

[0069] As described above, when inputting image data with a perturbation δ added to the first image data x into the neural network 201, the second normalization layer 2230-2 is used. Second image data x b When inputting into the neural network 201, the first normalization layer 230-1 is used. Furthermore, in the search for perturbation δ, we search for perturbation δ that is smaller than the constant ε.

[0070] As mentioned above, gradient methods such as backpropagation can be used to search for the perturbation δ. In neural network training, the parameter values ​​of the neural network are repeatedly updated, whereas the non-adversarial perturbation δ given by equation (2) is used. f UX In the generation process, the value of the perturbation δ is repeatedly updated.

[0071] As described above, it is also possible to repeat the process of training the neural network 201 and generating non-adversarial perturbations, generating non-adversarial samples using the obtained non-adversarial perturbations, and then using the obtained non-adversarial samples as the first image data to train the neural network 201 and generate non-adversarial perturbations.

[0072] Figure 3 shows an example of the procedure for the learning device 100 to train the neural network 201. In the process shown in Figure 3, the data acquisition unit 191 selects a first image data from the training data (step S101). The first image data here corresponds to the original image data. Furthermore, the data acquisition unit 191 selects one or more second image data from the training data (step S102).

[0073] Next, the perturbation acquisition unit 193 calculates (provisional) non-adversarial perturbations based on equation (2) (step S103). The process in step S103 may also involve the perturbation acquisition unit 193 searching for non-adversarial perturbations once using backpropagation. Next, the data processing unit 194 updates the first image data by adding the non-adversarial perturbation calculated by the perturbation acquisition unit 193 in step S103 (step S104). The updated first image data corresponds to a (provisional) non-adversarial sample. Furthermore, the parameter update unit 195 updates the parameter values ​​of the neural network 201 (step S105).

[0074] Next, the processing unit 190 determines whether or not the conditions for ending the update of the first image data have been met (step S106). The termination condition here is not limited to a specific condition. For example, the termination condition in step S106 may be that the number of iterations of the loop from step S103 to S106 has reached a predetermined number. Alternatively, the termination condition in step S106 may be set S B The second image data x included in b Distance φ(f) i {θNBN,θAuxBN} (x+δ),f i {θNBN,θMainBN} (x b The condition may also be that the value obtained by dividing the maximum distance by the minimum distance is less than or equal to a predetermined value.

[0075] If the processing unit 190 determines that the conditions for ending the update of the first image data have not been met (step S106: NO), the process returns to step S103. On the other hand, if it is determined that the conditions for ending the update of the first image data have been met (step S106: YES), the processing unit 190 determines whether or not the conditions for ending the learning of the neural network 201 have been met (step S107).

[0076] The termination conditions here are not limited to specific conditions. For example, the termination condition in step S107 may be that the number of iterations of the loop from steps S101 to S107 has reached a predetermined number. Alternatively, the termination condition in step S107 may be that the accuracy of the classification test using the features output by the neural network 201 is greater than or equal to a predetermined value.

[0077] If the processing unit 190 determines that the conditions for terminating the training of the neural network 201 have not been met (step S107: NO), the process returns to step S101. On the other hand, if the processing unit 190 determines that the conditions for terminating the learning of the neural network 201 have been met (step S107: YES), the learning device 100 terminates the process shown in Figure 3.

[0078] As described above, the data acquisition unit 191 acquires the first image data and the second image data which is classified into the same class as the first image data from among multiple class-classified image data. The perturbation acquisition unit 193 uses a neural network 201 which includes a subset network, a first normalization layer 230-1, and a second normalization layer 230-2 and outputs feature quantities to acquire a perturbation such that the distance between the first feature quantity, which is the feature quantity obtained when the data with the perturbation added to the first image data is input to the neural network 201 and normalized by the second normalization layer 230-2, and the second feature quantity, which is the feature quantity obtained when the second image data is input to the neural network 201 and normalized by the first normalization layer 230-1, is as small as possible. The data processing unit 194 generates image data by adding the obtained perturbation to the first image data.

[0079] The learning device 100 obtains non-adversarial samples, which are first image data processed so that their features approach those of the second image data. The features of this first image data are expected to be located in the feature space between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data.

[0080] When these non-adversarial samples are added to the training data and used to train a neural network, it is expected that the accuracy of the judgments made using the features output by the neural network will improve as the number of training data points increases, and furthermore, as the number of sample points located midway between multiple sample points in the feature space increases. According to the learning device 100, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0081] The neural network that is trained using training data to which non-adversarial samples generated by the learning device 100 have been added may be neural network 201, or it may be a neural network other than neural network 201.

[0082] Furthermore, the neural network being trained in this case may be a neural network having a dual normalization layer, like neural network 201; a neural network having a single normalization layer; or a neural network without a normalization layer.

[0083] Furthermore, the data acquisition unit 191 acquires multiple second image data. The perturbation acquisition unit 193 acquires perturbations such that the average distance between the first feature and the second feature is as small as possible.

[0084] According to the learning device 100, a non-adversarial sample is obtained, which is a first image data set processed to have features such that the average distance between the features of each of the multiple second image data sets is as small as possible. The features of this non-adversarial sample can be said to be located at a central position in the feature space relative to the features of the multiple second image data sets. From this, it is expected that the features of this first image data set are located near the center of the region in the feature space where the features of these multiple second image data sets should be classified into a class.

[0085] When these non-adversarial samples are added to the training data and used to train a neural network, it is expected that the accuracy of the judgments using the features output by the neural network will improve as the number of training data points increases, and furthermore, the number of sample points located near the center of the region of features that should be classified into the class to be classified as a second image increases. According to the learning device 100, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0086] Furthermore, the parameter update unit 195 updates the parameter values ​​of the neural network 201 by training the neural network 201 using image data to which perturbations have been added to the first image data. According to the learning device 100, it is expected that the accuracy of the judgment using the features output by the neural network 201 will improve.

[0087] The learning device 100 is an example of a data generation device in that it generates non-adversarial samples. If the only target of learning using non-adversarial samples generated by the learning device 100 is the neural network 201, the generation of non-adversarial samples by the learning device 100 may be limited to temporary generation. On the other hand, the non-adversarial samples generated by the learning device 100 can also be used to train neural networks other than the neural network 201. In this case, the learning device 100 may store the generated non-adversarial samples even after the training of the neural network 201 is complete.

[0088] <Second Embodiment> If a pre-trained neural network is available, it may be used to generate non-adversarial samples. This point will be explained in the second embodiment.

[0089] Figure 4 shows an example of the configuration of a data generation device according to the second embodiment. In the configuration shown in Figure 4, the data generation device 300 comprises a communication unit 110, a display unit 120, an operation input unit 130, a storage unit 180, and a processing unit 390. The storage unit 180 comprises a model storage unit 181. The model storage unit 181 comprises a common parameter storage unit 182, a first normalization layer parameter storage unit 183-1, and a second normalization layer parameter storage unit 183-2. The processing unit 190 comprises a data acquisition unit 191, a feature calculation unit 192, a perturbation acquisition unit 193, and a data processing unit 194.

[0090] In Figure 4, parts that have the same function as the parts in Figure 1 are denoted by the same reference numerals (110, 120, 130, 180, 181, 182, 183-1, 183-2, 191, 192, 193, 194), and detailed explanations are omitted here. The data generation device 300 differs from the learning device 100 in that its processing unit 390 does not include the parameter update unit 195, which is one of the components of the processing unit 190 shown in Figure 1. In all other respects, the data generation device 300 is the same as the learning device 100. The data generation device 300 generates non-adversarial samples using a pre-trained neural network 201 and does not train the neural network 201. For this reason, the processing unit 390 of the data generation device 300 does not include the parameter update unit 195 shown in Figure 1.

[0091] Figure 5 shows an example of the procedure for generating non-adversarial samples by the data generation device 300. The procedure in Figure 5 is the same as the part of the procedure in Figure 3 that generates non-adversarial samples. Steps S201 to S204 are the same as steps S101 to S104 in Figure 3. After step S204, the process proceeds to step S205.

[0092] Step S205 is the same as step S106 in Figure 3. If the processing unit 190 determines in step S205 that the conditions for ending the update of the first image data have not been met (step S205: NO), the process returns to step S203. On the other hand, if the processing unit 190 determines that the conditions for ending the update of the first image data have been met (step S205: YES), the data generation device 300 terminates the process shown in Figure 5.

[0093] According to the data generation device 300, similar to the case of the learning device 100, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0094] <Third Embodiment> Figure 6 shows an example of the configuration of a learning device according to the third embodiment. In the configuration shown in Figure 6, the data generation device 610 includes a data acquisition unit 611, a perturbation acquisition unit 612, and a data processing unit 613.

[0095] In this configuration, the data acquisition unit 611 acquires a first data and a second data that is classified in the same class as the first data, from among a plurality of data that have been classified into classes. The perturbation acquisition unit 612 uses a neural network that includes a subset network, a first normalization layer, and a second normalization layer to output feature quantities, and acquires perturbations such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The data processing unit 613 generates data by adding the obtained perturbation to the first data.

[0096] The data acquisition unit 611 is an example of a data acquisition means. The perturbation acquisition unit 612 is an example of a perturbation acquisition means. The data processing unit 613 is an example of a data processing means.

[0097] According to the data generation device 610, a non-adversarial sample is obtained, which is the first image data processed so that its features approach those of the second image data. The features of this non-adversarial sample are expected to be located in the feature space between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data.

[0098] When these non-adversarial samples are added to the training data and used to train a neural network, it is expected that the accuracy of the judgments made using the features output by the neural network will improve as the number of training data points increases, and furthermore, as the number of sample points located midway between multiple sample points in the feature space increases. According to the data generation device 610, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0099] <Fourth Embodiment> Figure 7 shows an example of the processing steps in the data generation method according to the fourth embodiment. The data generation method shown in Figure 7 includes acquiring data (step S611), acquiring perturbations (step S612), and processing the data (step S613).

[0100] In acquiring data (step S611), the computer acquires a first data item and a second data item that is classified into the same class as the first data item, from among multiple data items that have been classified into classes. In obtaining the perturbation (step S612), the computer uses a neural network that includes a subset network, a first normalization layer, and a second normalization layer to output features, and obtains a perturbation such that the distance between the first feature, which is the feature obtained when the first data with the perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature, which is the feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. In the data processing step (step S613), the computer generates data by adding the obtained perturbation to the first data.

[0101] According to the data generation method shown in Figure 7, non-adversarial samples are obtained, which are first image data processed so that their features approximate those of the second image data. The features of these non-adversarial samples are expected to be located midway between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data in the feature space.

[0102] When these non-adversarial samples are added to the training data and used to train a neural network, it is expected that the accuracy of the judgments made using the features output by the neural network will improve as the number of training data points increases, and furthermore, as the number of sample points located midway between multiple sample points in the feature space increases. According to the data generation method shown in Figure 7, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0103] Figure 8 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. In the configuration shown in Figure 8, the computer 700 comprises a CPU 710, a main memory 720, an auxiliary memory 730, an interface 740, and a non-volatile recording medium 750.

[0104] One or more of the above-described learning device 100, data generation device 300, and data generation device 610, or a part thereof, may be implemented in the computer 700. In that case, the operation of each processing unit described above is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it into the main memory 720, and executes the above processing according to the program. The CPU 710 also allocates memory areas in the main memory 720 corresponding to each of the above-described storage units according to the program. Communication between each device and other devices is performed by the interface 740 having a communication function and communicating according to the control of the CPU 710.

[0105] When the learning device 100 is implemented in the computer 700, the operation of the processing unit 190 and each of its parts is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main memory 720, and executes the above processing according to the program.

[0106] Furthermore, the CPU 710 allocates the storage area of ​​the storage unit 180 in the main memory 720 according to the program. Communication with other devices by the communication unit 110 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of various images by the display unit 120 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the operation input unit 130 is performed by the interface 740 having an input device and accepting user operations under the control of the CPU 710.

[0107] When the data generation device 300 is implemented in the computer 700, the operation of the processing unit 390 and each of its parts is stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main memory 720, and executes the above processing according to the program.

[0108] Furthermore, the CPU 710 allocates the storage area of ​​the storage unit 180 in the main memory 720 according to the program. Communication with other devices by the communication unit 110 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of various images by the display unit 120 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the operation input unit 130 is performed by the interface 740 having an input device and accepting user operations under the control of the CPU 710.

[0109] When the data generation device 610 is implemented in the computer 700, the operations of the data acquisition unit 611, the perturbation acquisition unit 612, and the data processing unit 613 are stored in auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main memory 720, and executes the above processes according to the program.

[0110] Furthermore, the CPU 710 reserves memory in the main memory 720 for processing performed by the data generation device 610, according to the program. Communication between the data generation device 610 and other devices is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Interaction between the data generation device 610 and the user is performed by the interface 740 equipped with a display device and an input device, displaying various images under the control of the CPU 710 and accepting user operations.

[0111] One or more of the above-mentioned programs may be recorded on the non-volatile recording medium 750. In this case, the interface 740 may read the program from the non-volatile recording medium 750. The CPU 710 may then either directly execute the program read by the interface 740, or temporarily save it in the main memory 720 or auxiliary memory 730 before executing it.

[0112] Alternatively, a program for executing all or part of the processing performed by the learning device 100, the data generation device 300, and the data generation device 610 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed to perform the processing of each part. The term "computer system" here includes hardware such as the operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs (Read Only Memory), CD-ROMs (Compact Disc Read Only Memory), and storage devices such as hard disks built into computer systems. The above-mentioned program may be intended to implement only a part of the functions described above, and may also be able to implement the above-mentioned functions in combination with programs already recorded in the computer system.

[0113] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0114] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0115] (Note 1) A data acquisition means that acquires a first data and a second data that is classified in the same class as the first data from among multiple data that have been classified into classes, A perturbation acquisition means that uses a neural network that includes a subset network, a first normalization layer, and a second normalization layer to output feature quantities, and acquires a perturbation such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. A data processing means that generates data by adding the obtained perturbation to the first data, A data generation device equipped with the following features.

[0116] (Note 2) The data acquisition means acquires a plurality of the second data, The perturbation acquisition means acquires the perturbation such that the average distance between the first feature and the second feature is as small as possible. The data generation device described in Appendix 1.

[0117] (Note 3) Parameter value update means for updating the parameter values ​​of the neural network by training the neural network using data to which the perturbation has been added to the first data. A data generation device as described in Appendix 1 or Appendix 2, further comprising the above.

[0118] (Note 4) A data acquisition means that acquires a first data and a second data that is classified in the same class as the first data from among multiple data that have been classified into classes, A perturbation acquisition means that uses a neural network that includes a subset network, a first normalization layer, and a second normalization layer to output feature quantities, and acquires a perturbation such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. A data processing means that generates data by adding the obtained perturbation to the first data, A parameter value update means updates the parameter values ​​of the neural network by learning the neural network using data to which the perturbation has been added to the first data, A learning device equipped with the following features.

[0119] (Note 5) Computers From among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The obtained perturbation is added to the first data to generate data. A data generation method that includes the following.

[0120] (Note 6) Computers From among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The obtained perturbation is added to the first data to generate data, The parameter values ​​of the neural network are updated by training the neural network using data to which the perturbation has been added to the first data. A learning method that includes this.

[0121] (Note 7) On the computer, Among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The process involves generating data by adding the obtained perturbation to the first data, A recording medium that stores a program to execute.

[0122] (Note 8) On the computer, Among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The process involves generating data by adding the obtained perturbation to the first data, The parameter values ​​of the neural network are updated by training the neural network using data to which the perturbation has been added to the first data. A recording medium that stores a program to execute. [Industrial applicability]

[0123] The present invention may be applied to a data generation device, a data generation method, and a recording medium. [Explanation of Symbols]

[0124] 100 Learning Devices 110 Communications Department 120 Display section 130 Operation Input Section 180 Storage section 181 Model Memory Unit 182 Common Parameter Storage Unit 183-1 First Normalization Layer Parameter Storage Unit 183-2 Second Normalization Layer Parameter Storage Unit 190, 390 Processing Unit 191 Data Acquisition Unit 192 Feature Calculation Unit 193 Perturbation Acquisition Unit 194 Data Processing Department 195 Parameter update section 300, 610 Data Generation Devices

Claims

1. A data acquisition means that acquires a first data and a second data that is classified in the same class as the first data from among multiple data that have been classified into classes, A perturbation acquisition means that uses a neural network that includes a subset network, a first normalization layer, and a second normalization layer to output feature quantities, and acquires a perturbation such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. A data processing means for generating data by adding the obtained perturbation to the first data, A data generation device equipped with the following features.

2. The data acquisition means acquires a plurality of the second data, The perturbation acquisition means acquires the perturbation such that the average distance between the first feature and the second feature is as small as possible. The data generation apparatus according to claim 1.

3. The system further includes a parameter value update means for updating the parameter values ​​of the neural network by training the neural network using data obtained by adding the perturbation to the first data. A data generation apparatus according to claim 1 or claim 2.

4. Computers From among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. The obtained perturbation is added to the first data to generate data. A data generation method that includes the following.

5. On the computer, Among multiple data sets that have been classified into classes, obtain the first data set and the second data set that is classified into the same class as the first data set. Using a neural network that includes a subset network, a first normalization layer, and a second normalization layer and outputs feature quantities, the perturbation is obtained such that the distance between the first feature quantity, which is the feature quantity obtained when the first data with a perturbation added is input to the neural network and normalized by the second normalization layer, and the second feature quantity, which is the feature quantity obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible. To generate data by adding the obtained perturbation to the first data, A program to execute.

Images