Data generation device, data generation method and program

Abstract

The present invention provides a data generation device comprising: a data acquisition means that acquires, from among a plurality of pieces of data classified into classes, first data and second data classified into the same class as the first data; a perturbation acquisition means that, using a neural network which includes a partial network, a first normalization layer, and a second normalization layer and which outputs a feature amount, acquires such a perturbation that the distance between a first feature amount and a second feature amounts is as short as possible, the first feature amount being the feature amount when data obtained by adding the perturbation to the first data is input to the neural network and is normalized by the second normalization layer, the second feature amount being the feature amount when the second data is input to the neural network and is normalized by the first normalization layer; and a data processing means that generates the data obtained by adding the obtained perturbation to the first data.

Description

Data generation device, data generation method and recording medium

[0001] The present invention relates to a data generating device, a data generating method, and a recording medium.

[0002] New training data may be generated by processing training data used for learning a neural network. For example, Patent Literature 1 describes generating image data by combining image data of products, such as generating image data of a defective product by combining image data of a non-defective product and image data of a defective product.

[0003] Japanese Patent Application Laid-Open No. 2021-118442

[0004] When generating training data for a neural network that outputs feature quantities, it is preferable to generate training data that can improve the accuracy of judgments using the feature quantities output by the neural network.

[0005] An example of an object of the present invention is to provide a data generation device, a data generation method, and a recording medium that can solve the above-mentioned problems.

[0006] According to a first aspect of the present invention, a data generation device includes: a data acquisition means for acquiring, from a plurality of data classified into classes, first data and second data classified into the same class as the first data; a perturbation acquisition means for using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquiring the perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding a perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and a data processing means for generating data in which the obtained perturbation is added to the first data.

[0007] According to a second aspect of the present invention, a data generation method includes: a computer acquiring, from a plurality of data items classified into classes, first data and second data items classified into the same class as the first data; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquiring a perturbation such that a distance between a first feature item, which is a feature item obtained when data obtained by adding a perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature item, which is a feature item obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and generating data by adding the obtained perturbation to the first data.

[0008] According to a third aspect of the present invention, a recording medium is a recording medium having recorded thereon a program for causing a computer to execute the following steps: acquire first data and second data classified into the same class as the first data from a plurality of data classified into classes; use a neural network including a partial network, a first normalization layer, and a second normalization layer to output features; acquire the perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and generate data by adding the obtained perturbation to the first data.

[0009] According to the present invention, when generating training data for a neural network that outputs features, it is possible to generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0010] FIG. 1 is a diagram showing an example of the configuration of a learning device according to a first embodiment; FIG. 2 is a diagram showing an example of a neural network stored in a model storage unit according to the first embodiment; FIG. 3 is a diagram showing an example of a processing procedure by which the learning device according to the first embodiment learns a neural network; FIG. 4 is a diagram showing an example of the configuration of a data generation device according to a second embodiment; FIG. 5 is a diagram showing an example of a processing procedure by which the data generation device according to the second embodiment generates non-adversarial samples; FIG. 6 is a diagram showing an example of the configuration of a learning device according to a third embodiment; FIG. 7 is a diagram showing an example of a processing procedure in a data generation method according to a fourth embodiment; and FIG. 8 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0011] <First Embodiment> Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0012] FIG. 1 is a diagram illustrating an example of the configuration of a learning device according to the first embodiment. In the configuration illustrated in FIG. 1, the learning device 100 includes a communication unit 110, a display unit 120, an operation input unit 130, a storage unit 180, and a processing unit 190. The storage unit 180 includes a model storage unit 181. The model storage unit 181 includes a common parameter storage unit 182, a first normalization layer parameter storage unit 183-1, and a second normalization layer parameter storage unit 183-2. The processing unit 190 includes a data acquisition unit 191, a feature calculation unit 192, a perturbation acquisition unit 193, a data processing unit 194, and a parameter update unit 195. The first normalization layer parameter storage unit 183-1 and the second normalization layer parameter storage unit 183-2 are also collectively referred to as the normalization layer parameter storage unit 183.

[0013] The learning device 100 performs neural network training. In particular, the learning device 100 processes training data to generate new training data and uses the obtained training data for training the neural network. The learning device 100 may be configured using a computer such as a personal computer (PC) or a workstation (WS).

[0014] In the following, an example will be described in which the learning device 100 uses classified image data as training data. However, the training data used by the learning device 100 is not limited to image data, and can be various data that can be processed as described below. For example, the learning device 100 may use classified audio data as training data.

[0015] In the following, when distinguishing between training data as a set of data and training data as individual data included in the set, the training data as a set of data will also be referred to as a training data set. Furthermore, training data as individual data will be expressed as individual training data, one training data, or multiple training data.

[0016] The communication unit 110 communicates with other devices. For example, the communication unit 110 may receive training data for learning the neural network from other devices.

[0017] The display unit 120 has a display screen such as a liquid crystal panel or an LED (Light Emitting Diode) panel, and displays various images. For example, the display unit 120 may display information related to neural network learning, such as an image represented by image data obtained by processing image data included in a training dataset.

[0018] The operation input unit 130 includes input devices such as a keyboard and a mouse, and accepts user operations. For example, the operation input unit 130 may accept user operations related to neural network learning, such as setting values ​​for hyperparameters in processing image data.

[0019] The storage unit 180 stores various data. The storage unit 180 is configured using a storage device included in the learning device 100. The model storage unit 181 stores a neural network as a machine learning model. The neural network stored in the model storage unit 181 receives input of image data and outputs feature amounts of the input image data (feature amounts of the image indicated by the image data).

[0020] Fig. 2 is a diagram showing an example of a neural network stored in the model storage unit 181. The neural network 201 shown in Fig. 2 is configured as a type of convolutional neural network (CNN), and includes an input layer 210, a convolutional layer 221, an activation layer 222, a pooling layer 223, a first normalization layer 230-1, a second normalization layer 230-2, a fully connected layer 240, and an output layer 250. The first normalization layer 230-1 and the second normalization layer 230-2 are also collectively referred to as normalization layers 230.

[0021] 2, in order from the upstream side of the data flow, an input layer 210 is followed by one or more combinations of a convolutional layer 221, an activation layer 222, and a pooling layer 223, in that order, and downstream thereof are provided a fully connected layer 240 and an output layer 250. In addition, for each combination of the convolutional layer 221, the activation layer 222, and the pooling layer 223, a first normalization layer 230-1 and a second normalization layer 230-2 are arranged in parallel between the activation layer 222 and the pooling layer 223. The number of channels in the neural network 201 is not limited to a specific number.

[0022] Among the parts of the neural network 201, the parts other than the first normalization layer 230-1 and the second normalization layer 230-2 are also referred to as a common part or a partial network. In the example of Figure 2, the combination of the input layer 210, the convolutional layer 221, the activation layer 222, the pooling layer 223, the fully connected layer 240, and the output layer 250 corresponds to an example of a common part.

[0023] The input layer 210 accepts input data to the neural network 201. The convolutional layer 221 performs a convolution operation on the data input thereto. The convolutional layer 221 may further perform padding to adjust the data size. The activation layer 222 applies an activation function to the data input thereto. The activation function used by the activation layer 222 is not limited to a specific function. For example, a rectified linear function (ReLU) may be used as the activation function, but is not limited thereto. The pooling layer 223 performs pooling on the data input thereto.

[0024] The first normalization layer 230-1 normalizes the data input to it. The normalization here is the same as that in batch normalization, and the first normalization layer 230-1 converts the data so that the mean and variance of the data included in one group become predetermined values. For example, to set the mean value of the data in one group to 0 and the variance value to 1, the first normalization layer 230-1 calculates the mean value and variance value of the data in the group being normalized, subtracts the mean value from each data, and divides the value after subtraction by the variance value.

[0025] The mean value after normalization by the first normalization layer 230-1 is not limited to 0, and the variance value is not limited to 1. For example, the first normalization layer 230-1 may perform normalization so that the mean value of the group becomes α and the variance value becomes β, with α being a real number and β being a positive real number. Furthermore, the values ​​of α and β may be the subject of learning. The values ​​of α and β may be set by learning for each first normalization layer 230-1.

[0026] The mean and variance that are targets when the first normalization layer 230-1 normalizes data are examples of parameters of the first normalization layer 230-1. The mean that is the target when the first normalization layer 230-1 normalizes data is also referred to as the first mean. The variance that is the target when the first normalization layer 230-1 normalizes data is also referred to as the first variance.

[0027] The second normalization layer 230-2 normalizes the data input to the second normalization layer 230-2 itself. The normalization process performed by the second normalization layer 230-2 is similar to the normalization process performed by the first normalization layer 230-1 described above. The mean and variance that are targets when the second normalization layer 230-2 normalizes the data are examples of parameters of the second normalization layer 230-2. The mean that is the target when the second normalization layer 230-2 normalizes the data is also referred to as the second mean. The variance that is the target when the second normalization layer 230-2 normalizes the data is also referred to as the second variance.

[0028] Either the first normalization layer 230-1 or the second normalization layer 230-2 is selectively used depending on the input data to the neural network 201. When image data (unprocessed image data) included in the original training data set is input to the neural network 201, the first normalization layer 230-1 is used. On the other hand, when image data generated by processing by the learning device 100 is input to the neural network 201, the second normalization layer 230-2 is used. Such switching of the normalization layer 230 may be performed, for example, by an instruction from the feature calculation unit 192 that inputs data to the neural network 201.

[0029] The distribution of unprocessed data and processed data is different, and it is thought that normalizing the unprocessed data and the processed data using the same normalization layer would not be effective enough. In contrast, in the neural network 201, the effect of normalization can be obtained by selectively using either the first normalization layer 230-1 or the second normalization layer 230-2, and it is expected that learning of the neural network 201 can be performed efficiently.

[0030] The activation layer 222 may output data to only one of the first normalization layer 230-1 and the second normalization layer 230-2, thereby switching the normalization layer 230. In this case, of the first normalization layer 230-1 and the second normalization layer 230-2, only one that receives data input from the activation layer 222 may receive parameter updates through learning and output data to the pooling layer 223.

[0031] The neural network 201 when the first normalization layer 230-1 is used is also referred to as the neural network 201 that uses the first normalization layer 230-1. The neural network 201 when the second normalization layer 230-2 is used is also referred to as the neural network 201 that uses the second normalization layer 230-2. Note that the first normalization layer 230-1 may always be used when the trained neural network 201 is in operation.

[0032] The fully connected layer 240 converts data input to the fully connected layer 240 itself into data equal to the number of pieces of output data from the neural network 201. The output layer 250 outputs the output data from the neural network 201. For example, the output layer 250 may apply an activation function such as a softmax function to the data from the fully connected layer 240 and output the data.

[0033] Alternatively, the fully connected layer 240 may generate output data for the neural network 201, and the output layer 250 may directly output the data from the fully connected layer 240. In this case, the fully connected layer 240 may also function as the output layer 250, and may directly output the data to the outside of the neural network 201.

[0034] However, the configuration of the machine learning model stored in the model storage unit 181 is not limited to a specific configuration. For example, when the model storage unit 181 stores a convolutional neural network as the machine learning model, the configuration and number of layers of the convolutional neural network can be various. For example, the configuration of the machine learning model stored in the model storage unit 181 may be a configuration in which, among the combination of the convolutional layer 221, the activation layer 222, and the pooling layer 223 included in the neural network 201 in the example of FIG. 2 , the activation layer 222 is not provided.

[0035] Furthermore, the position where the combination of the first normalization layer 230-1 and the second normalization layer 230-2 is provided is not limited to a specific position. For example, the combination of the first normalization layer 230-1 and the second normalization layer 230-2 may be provided for only some of the combinations of the convolutional layer 221, the activation layer 222, and the pooling layer 223. The configuration of the machine learning model stored in the model storage unit 181 may be a configuration in which the number of batch normalization layers is reduced to two and arranged in parallel from a convolutional neural network in which a batch normalization layer is provided.

[0036] However, the machine learning model stored in the model storage unit 181 is not limited to a convolutional neural network, and can be any of various neural networks to which normalization by the first normalization layer 230-1 and the second normalization layer 230-2 can be applied.

[0037] Furthermore, the implementation method of the neural network to be learned by the learning device 100 is not limited to the method in which the model storage unit 181 stores the neural network. For example, the neural network to be learned by the learning device 100 may be implemented in hardware, such as by using an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array). The neural network to be learned by the learning device 100 may be configured as a part of the learning device 100, or may be configured external to the learning device 100.

[0038] The common parameter storage unit 182 stores parameter values ​​of the common part. The common parameter storage unit 182 stores values ​​of various parameters to be learned, such as parameters of a filter for convolutional operation in a convolution layer and an activation function in an activation layer. The parameter values ​​of the common part are also referred to as common parameter values.

[0039] The first normalization layer parameter storage unit 183-1 stores, for each first normalization layer 230-1, the parameter values ​​of that first normalization layer 230-1. The first normalization layer parameter storage unit 183-1 stores the values ​​of various parameters to be learned, such as the first mean and the first variance.

[0040] The second normalization layer parameter storage unit 183-2 stores, for each second normalization layer 230-2, the parameter values ​​of the second normalization layer 230-2. The second normalization layer parameter storage unit 183-2 stores the values ​​of various parameters to be learned, such as the second mean and the second variance.

[0041] Processing unit 190 performs various processes by controlling each unit of learning device 100. The functions of processing unit 190 are performed, for example, by a CPU (Central Processing Unit) included in learning device 100 reading and executing a program from storage unit 180.

[0042] The data acquisition unit 191 selects, from the training data, image data to be processed and one or more image data that are classified into the same class as the image data to be processed but are other than the image data to be processed. The data acquisition unit 191 corresponds to an example of a data acquisition means. The image data to be processed selected from the training data is also referred to as original image data. The original image data or image data obtained by processing the original image data is also referred to as first image data. Image data (one of the image data) that are classified into the same class as the original image data but are other than the original image data is also referred to as second image data.

[0043] The first image data is an example of the first data. The first data is the original data or data obtained by processing the original data when the training data is not limited to image data. The original data here is data to be processed that is selected from the training data.

[0044] The second image data corresponds to an example of second data, which is data (one of) data that is classified into the same class as the original data and is other than the original data when the training data is not limited to image data.

[0045] The feature calculation unit 192 calculates the feature of the image data using a neural network 201 (a neural network stored in the model storage unit 181). Specifically, the feature calculation unit 192 inputs the image data to the neural network 201 and acquires the feature output from the neural network 201.

[0046] The perturbation acquisition unit 193 uses the second image data to acquire a perturbation for processing the first image data. The perturbation here refers to image data that is the difference between the image data before processing and the image data after processing (image data obtained by processing). The perturbation acquisition unit 193 is an example of a perturbation acquisition means.

[0047] The perturbation acquisition unit 193 calculates perturbations so that the feature quantities of the processed image data are closer to the feature quantities of the second image data than the feature quantities of the image data before processing. Perturbations that make the feature quantities of the processed image data closer to the feature quantities of image data classified into the same class as the original image data than the feature quantities of the image data before processing are also referred to as unadversarial perturbations.

[0048] The data processing unit 194 processes the first image data. Specifically, the data processing unit 194 generates processed image data by adding the perturbation acquired by the perturbation acquisition unit 193 to the first image data. More specifically, the data processing unit 194 adds, for each pixel of the first image data, the pixel value of the first image data and the pixel value of the pixel of the perturbation associated with that pixel. The data processing unit 194 corresponds to an example of a data processing means. Image data obtained by adding a non-adversarial perturbation to image data is also referred to as a non-adversarial example (UN).

[0049] The parameter update unit 195 updates the parameter values ​​of the neural network 201 through learning of the neural network 201. The parameter update unit 195 is an example of a parameter value update means.

[0050] When the first image data (image data to which perturbation has been added) is input to the neural network 201, the parameter update unit 195 updates the parameter values ​​of the partial network and the parameter values ​​of the second normalization layer 230-2. On the other hand, when the second image data (image data that has not been processed) is input to the neural network 201, the parameter update unit 195 updates the parameter values ​​of the partial network and the parameter values ​​of the first normalization layer 230-1.

[0051] Similar to the parameter updating in mini-batch learning, the parameter updating unit 195 may update the parameter values ​​using the average value of multiple input data for each part of the neural network 201 .

[0052] The non-adversarial sample x generated by the data processing unit 194 ＵＮ is expressed as in equation (1).

[0053]

[0054] x represents the first image data. ｆ ＵＸ represents the non-adversarial perturbation acquired by the perturbation acquisition unit 193. The non-adversarial perturbation δ acquired by the perturbation acquisition unit 193 ｆ ＵＸ is expressed as in equation (2).

[0055]

[0056] x ｂ represents the second image data. As described above, the second image data is image data included in the training data that is classified into the same class as the original image data and is other than the original image data. S Ｂ is the non-adversarial perturbation δ ｆ ＵＸ The second image data x used in the calculation ｂ where n is an integer greater than or equal to 1, and the set S Ｂ The second image data x included in ｂ Therefore, n represents the number of non-adversarial perturbations δ ｆ ＵＸ The second image data x used in the calculation ｂ Represents the number of.

[0057] θ ＭａｉｎＢＮ represents the parameters of the first normalization layer 230-1. ＡｕｘＢＮ represents the parameters of the second normalization layer 230-2. ＮＢＮ represents the parameter of the intersection, i.e., θ ＮＢＮ represents the parameters other than the normalization layer 230.

[0058] f ｉ {θNBN, θAuxBN}(x+δ) represents the feature amount obtained by inputting image data obtained by adding a perturbation δ to the first image data x to the neural network 201 using the second normalization layer 230-2. ｉThe feature represented by {θNBN, θAuxBN}(x+δ) is an example of a first feature.

[0059] Here, the learning device 100 receives the non-adversarial perturbation δ ｆ ＵＸ Calculation of the non-adversarial sample x by the data processing unit 194 ＵＮ Repeat the generation of non-adversarial samples x ＵＮ That is, the learning device 100 may gradually update the non-adversarial sample x generated by the data processing unit 194. ＵＮ is used as the first image data x, and the non-adversarial sample x ＵＮ A process of generating the following may be further performed.

[0060] For example, the perturbation acquisition unit 193 uses a method of repeating a solution search such as backpropagation to obtain “1 / nΣ ｘｂ∈ＳＢ φ(f ｉ {θNBN, θAuxBN} (x+δ), f ｉ {θNBN, θMainBN}(x ｂ Alternatively, the perturbation δ may be calculated so that the value of (δ)) becomes as small as possible.

[0061] i is the non-adversarial perturbation δ ｆ ＵＸ Calculation of the non-adversarial sample x by the data processing unit 194 ＵＮ The number of iterations of generating non-adversarial samples x ＵＮ This represents the number of times the update is repeated.

[0062] In addition, the non-adversarial perturbation δ ｆ ＵＸ In addition to the calculation of x, the parameter update unit 195 may update the parameter values ​​by learning the neural network 201. ＵＮ The parameter values ​​of the neural network 201 may also be updated each time the update is repeated. ｉ is the non-adversarial sample x ＵＮrepresents the neural network 201 during the i-th iteration of

[0063] In the learning of the neural network 201, the parameter update unit 195 updates the parameter "1 / nΣ ｘｂ∈ＳＢ φ(f ｉ {θNBN, θAuxBN} (x+δ), f ｉ {θNBN, θMainBN}(x ｂ The parameter values ​​of the neural network 201 may be updated so that the value of (x+δ)) becomes as small as possible. The learning of the neural network 201 performed by the learning device 100 can be said to be learning of the neural network 201 using image data x+δ obtained by adding a perturbation δ to the first image data x.

[0064] Alternatively, the output of the neural network 201 may be input to a classification-type neural network. The parameter update unit 195 may then update the parameter values ​​of the neural network 201 so as to maximize the likelihood of the correct class. In this case, the training of the neural network 201 performed by the training device 100 can also be considered training of the neural network 201 using image data x+δ obtained by adding a perturbation δ to the first image data x.

[0065] f ｉ {θNBN, θMainBN}(x ｂ ) is the second image data x ｂ represents the feature obtained by inputting f to the neural network 201 using the first normalization layer 230-1. ｉ {θNBN, θMainBN}(x ｂ ) corresponds to an example of the second feature amount.

[0066] φ represents the distance between two vectors. In equation (2), φ represents the distance in the feature space. As the distance φ, various types of distances applicable to vectors can be used. For example, the distance φ may be an L1 norm, an L2 norm, or an L∞ norm.

[0067] Argmin is a function that outputs the value of the variable shown below argmin such that the expression shown after argmin is minimized. ||δ|| ｐ <ε represents a constraint that the magnitude of the perturbation δ output by argmin is smaller than a predetermined constant ε. Any type of norm can be used as the magnitude of the perturbation δ. For example, "|| || ｐ " may represent the L1 norm, may represent the L2 norm, or may represent the L∞ norm.

[0068] Equation (2) expresses the feature amount of image data obtained by adding a perturbation δ to the first image data x and the feature amount of the second image data x ｂ The set S of distances between the features of Ｂ All the second image data x included in ｂ Search for a perturbation δ that minimizes the average of ｆ ＵＸ This means that...

[0069] As described above, when image data obtained by adding a perturbation δ to the first image data x is input to the neural network 201, the second normalization layer 2230-2 is used. ｂ The first normalization layer 230-1 is used when inputting to the neural network 201. In addition, in searching for the perturbation δ, a perturbation δ smaller than the constant ε is searched for.

[0070] As mentioned above, the perturbation δ can be searched using a gradient method such as backpropagation. While the parameter values ​​of the neural network are repeatedly updated in the learning of the neural network, the non-adversarial perturbation δ according to Equation (2) can be searched using a gradient method such as backpropagation. ｆ ＵＸ In generating the perturbation δ, the value of the perturbation δ is repeatedly updated.

[0071] As described above, the process of both training the neural network 201 and generating non-adversarial perturbations may be repeated, the non-adversarial samples may be generated using the obtained non-adversarial perturbations, and the obtained non-adversarial samples may be used as first image data to train the neural network 201 and generate non-adversarial perturbations.

[0072] 3 is a diagram showing an example of the processing procedure by which the learning device 100 learns the neural network 201. In the processing of FIG. 3, the data acquisition unit 191 selects first image data from the training data (step S101). Here, the first image data corresponds to the original image data. The data acquisition unit 191 also selects one or more second image data from the training data (step S102).

[0073] Next, the perturbation acquisition unit 193 calculates a (tentative) non-adversarial perturbation based on Equation (2) (step S103). The processing of step S103 may involve the perturbation acquisition unit 193 searching for a non-adversarial perturbation once by backpropagation. Next, the data processing unit 194 updates the first image data by adding the non-adversarial perturbation calculated by the perturbation acquisition unit 193 in step S103 to the first image data (step S104). The updated first image data corresponds to a (tentative) non-adversarial sample. Furthermore, the parameter update unit 195 updates the parameter values ​​of the neural network 201 (step S105).

[0074] Next, the processing unit 190 determines whether or not a termination condition for updating the first image data is met (step S106). The termination condition here is not limited to a specific condition. For example, the termination condition in step S106 may be a condition that the number of repetitions of the loop from steps S103 to S106 reaches a predetermined number. Alternatively, the termination condition in step S106 may be a condition that the number of repetitions of the set S Ｂ The second image data x included in ｂ The distance φ(f ｉ {θNBN, θAuxBN} (x+δ), f ｉ {θNBN, θMainBN}(x ｂ)) may be a condition that the value obtained by dividing the maximum distance by the minimum distance is equal to or less than a predetermined value.

[0075] If the processing unit 190 determines that the update termination condition for the first image data is not satisfied (step S106: NO), the process returns to step S103. On the other hand, if the processing unit 190 determines that the update termination condition for the first image data is satisfied (step S106: YES), the processing unit 190 determines whether the learning termination condition for the neural network 201 is satisfied (step S107).

[0076] The termination condition here is not limited to a specific condition. For example, the termination condition in step S107 may be that the loop from steps S101 to S107 has been repeated a predetermined number of times. Alternatively, the termination condition in step S107 may be that the accuracy rate in a classification test using the features output by the neural network 201 is greater than or equal to a predetermined value.

[0077] If the processing unit 190 determines that the learning termination condition of the neural network 201 is not satisfied (step S107: NO), the process returns to step S101. On the other hand, if the processing unit 190 determines that the learning termination condition of the neural network 201 is satisfied (step S107: YES), the learning device 100 ends the process of FIG.

[0078] As described above, the data acquisition unit 191 acquires, from among a plurality of image data classified into classes, first image data and second image data classified into the same class as the first image data. The perturbation acquisition unit 193 uses the neural network 201, which includes a partial network, a first normalization layer 230-1, and a second normalization layer 230-2 and outputs features, to acquire a perturbation that minimizes the distance between the first feature value, which is the feature value obtained when data obtained by adding the perturbation to the first image data is input to the neural network 201 and normalized by the second normalization layer 230-2, and the second feature value, which is the feature value obtained when the second image data is input to the neural network 201 and normalized by the first normalization layer 230-1. The data processing unit 194 generates image data by adding the obtained perturbation to the first image data.

[0079] The learning device 100 obtains a non-adversarial sample, which is first image data whose features have been processed to approximate those of the second image data. The features of the first image data are expected to be located midway between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data in the feature space.

[0080] When training a neural network by adding these non-adversarial samples to training data, the number of training data increases, and furthermore, the number of sample points located midway between multiple sample points in feature space increases, which is expected to improve the accuracy of judgments using the features output by the neural network. In this way, the learning device 100 can generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0081] The neural network to be trained using the training data to which the non-adversarial samples generated by the learning device 100 have been added may be the neural network 201 or a neural network other than the neural network 201.

[0082] Furthermore, the neural network to be trained in this case may be a neural network with a double normalization layer, such as neural network 201, a neural network with a single normalization layer, or a neural network with no normalization layer.

[0083] The data acquisition unit 191 acquires a plurality of second image data. The perturbation acquisition unit 193 acquires perturbations that minimize the average value of the distances between the first feature amount and each of the second feature amounts.

[0084] The learning device 100 obtains non-adversarial samples, which are first image data processed to have features that minimize the average distance between the features and each of the plurality of second image data. The features of the non-adversarial samples can be said to be located at the center of the features of the plurality of second image data in the feature space. Therefore, the features of the first image data are expected to be located near the center of the region of features in the feature space that should be classified into classes into which the plurality of second image data are classified.

[0085] When these non-adversarial samples are added to the training data to train the neural network, the number of training data increases, and furthermore, the number of sample points located near the center of the region of features in the feature space that should be classified into the class into which the second image data is classified increases, which is expected to improve the accuracy of judgment using the features output by the neural network. In this way, the learning device 100 can generate training data that is expected to improve the accuracy of judgment using the features output by the neural network.

[0086] Furthermore, the parameter update unit 195 updates the parameter values ​​of the neural network 201 by training the neural network 201 using the image data obtained by adding the perturbation to the first image data. The learning device 100 is expected to improve the accuracy of determination using the feature values ​​output by the neural network 201.

[0087] The learning device 100 corresponds to an example of a data generation device in that it generates non-adversarial samples. When the neural network 201 is the only target of learning using the non-adversarial samples generated by the learning device 100, the generation of non-adversarial samples by the learning device 100 may be limited to temporary generation. On the other hand, the non-adversarial samples generated by the learning device 100 may also be used for learning neural networks other than the neural network 201. In this case, the learning device 100 may store the generated non-adversarial samples even after learning of the neural network 201 is completed.

[0088] Second Embodiment When a trained neural network is available, the neural network may be used to generate non-adversarial samples. This point will be described in the second embodiment.

[0089] Fig. 4 is a diagram showing an example of the configuration of a data generating device according to the second embodiment. In the configuration shown in Fig. 4, a data generating device 300 includes a communication unit 110, a display unit 120, an operation input unit 130, a storage unit 180, and a processing unit 390. The storage unit 180 includes a model storage unit 181. The model storage unit 181 includes a common parameter storage unit 182, a first normalization layer parameter storage unit 183-1, and a second normalization layer parameter storage unit 183-2. The processing unit 190 includes a data acquisition unit 191, a feature calculation unit 192, a perturbation acquisition unit 193, and a data processing unit 194.

[0090] Among the components in FIG. 4 , components having similar functions to those in FIG. 1 are denoted by the same reference numerals (110, 120, 130, 180, 181, 182, 183-1, 183-2, 191, 192, 193, 194), and detailed descriptions thereof will be omitted here. The data generating device 300 differs from the learning device 100 in that the processing unit 390 does not include the parameter update unit 195, which is one of the components included in the processing unit 190 in FIG. 1 . In other respects, the data generating device 300 is similar to the learning device 100. The data generating device 300 generates non-adversarial samples using a trained neural network 201 and does not train the neural network 201. Therefore, the processing unit 390 of the data generating device 300 does not include the parameter update unit 195 of FIG. 1 .

[0091] Fig. 5 is a diagram showing an example of a processing procedure by which the data generating device 300 generates non-adversarial samples. The processing in Fig. 5 is similar to the processing for generating non-adversarial samples in the processing in Fig. 3. Steps S201 to S204 are similar to steps S101 to S104 in Fig. 3. After step S204, the processing proceeds to step S205.

[0092] Step S205 is the same as step S106 in Fig. 3. If the processing unit 190 determines in step S205 that the termination condition for updating the first image data is not met (step S205: NO), the process returns to step S203. On the other hand, if the processing unit 190 determines that the termination condition for updating the first image data is met (step S205: YES), the data generating device 300 ends the process in Fig. 5.

[0093] As with the learning device 100, the data generating device 300 can generate training data that is expected to improve the accuracy of judgments using features output by a neural network.

[0094] 6 is a diagram showing an example of the configuration of a learning device according to the third embodiment. In the configuration shown in Fig. 6, a data generating device 610 includes a data acquiring unit 611, a perturbation acquiring unit 612, and a data processing unit 613.

[0095] With this configuration, the data acquisition unit 611 acquires first data and second data classified into the same class as the first data from among multiple data classified into classes. The perturbation acquisition unit 612 uses a neural network that includes a partial network, a first normalization layer, and a second normalization layer and outputs features to acquire a perturbation that minimizes the distance between the first feature, which is the feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and the second feature, which is the feature obtained when second data is input to the neural network and normalized by the first normalization layer. The data processing unit 613 generates data by adding the obtained perturbation to the first data.

[0096] The data acquisition unit 611 is an example of a data acquisition means, the perturbation acquisition unit 612 is an example of a perturbation acquisition means, and the data processing unit 613 is an example of a data processing means.

[0097] The data generation device 610 obtains a non-adversarial sample, which is first image data whose features have been processed to approximate those of the second image data. The features of this non-adversarial sample are expected to be located midway between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data in the feature space.

[0098] When these non-adversarial samples are added to the training data to train the neural network, the number of training data increases, and furthermore, the number of sample points located in the middle of multiple sample points in the feature space increases, which is expected to improve the accuracy of judgments using the features output by the neural network. In this way, the data generation device 610 can generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0099] 7 is a diagram showing an example of a processing procedure in a data generation method according to a fourth embodiment. The data generation method shown in FIG. 7 includes acquiring data (step S611), acquiring perturbations (step S612), and processing the data (step S613).

[0100] In acquiring data (step S611), the computer acquires first data and second data classified into the same class as the first data from among a plurality of classified data. In acquiring a perturbation (step S612), the computer uses a neural network including a partial network, a first normalization layer, and a second normalization layer to output features, to acquire a perturbation that minimizes the distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized using the second normalization layer, and a second feature, which is a feature obtained when second data is input to the neural network and normalized using the first normalization layer. In processing the data (step S613), the computer generates data by adding the obtained perturbation to the first data.

[0101] 7, a non-adversarial sample is obtained, which is first image data whose features have been processed to be closer to the features of the second image data. The features of this non-adversarial sample are expected to be located midway between the features of the original image data (the first image data included in the original training dataset) and the features of the second image data in the feature space.

[0102] When training a neural network by adding these non-adversarial samples to the training data, the number of training data increases, and furthermore, the number of sample points located in the middle of multiple sample points in the feature space increases, which is expected to improve the accuracy of judgments using the features output by the neural network. In this way, the data generation method shown in Figure 7 can generate training data that is expected to improve the accuracy of judgments using the features output by the neural network.

[0103] 8 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. In the configuration shown in FIG. 8, a computer 700 includes a CPU 710, a main memory device 720, an auxiliary memory device 730, an interface 740, and a non-volatile recording medium 750.

[0104] One or more of the learning device 100, data generating device 300, and data generating device 610, or a part thereof, may be implemented in a computer 700. In this case, the operation of each of the above-described processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program. The CPU 710 also allocates storage areas in the main storage device 720 corresponding to each of the above-described storage units in accordance with the program. Communication between each device and other devices is performed by an interface 740 having a communication function and communicating under the control of the CPU 710.

[0105] When learning device 100 is implemented in computer 700, the operations of processing unit 190 and each of its units are stored in the form of a program in auxiliary storage device 730. CPU 710 reads the program from auxiliary storage device 730, loads it into main storage device 720, and executes the above-described processing in accordance with the program.

[0106] Furthermore, the CPU 710 allocates a storage area of ​​the storage unit 180 in the main storage device 720 in accordance with the program. Communication with other devices by the communication unit 110 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of various images by the display unit 120 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the operation input unit 130 is performed by the interface 740 having an input device and receiving user operations under the control of the CPU 710.

[0107] When the data generating device 300 is implemented in a computer 700, the operations of the processing unit 390 and each of its units are stored in the form of a program in an auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-described processing in accordance with the program.

[0108] Furthermore, the CPU 710 allocates a storage area of ​​the storage unit 180 in the main storage device 720 in accordance with the program. Communication with other devices by the communication unit 110 is performed by the interface 740 having a communication function and operating under the control of the CPU 710. Display of various images by the display unit 120 is performed by the interface 740 having a display device and displaying various images under the control of the CPU 710. Reception of user operations by the operation input unit 130 is performed by the interface 740 having an input device and receiving user operations under the control of the CPU 710.

[0109] When the data generating device 610 is implemented in the computer 700, the operations of the data acquiring unit 611, the perturbation acquiring unit 612, and the data processing unit 613 are stored in the form of a program in the auxiliary storage device 730. The CPU 710 reads the program from the auxiliary storage device 730, loads it into the main storage device 720, and executes the above-mentioned processing in accordance with the program.

[0110] Furthermore, the CPU 710, in accordance with the program, allocates a storage area in the main storage device 720 for processing performed by the data generating device 610. Communication between the data generating device 610 and other devices is performed by the interface 740, which has a communication function and operates under the control of the CPU 710. Interaction between the data generating device 610 and a user is performed by the interface 740, which has a display device and an input device, displaying various images under the control of the CPU 710 and accepting user operations.

[0111] One or more of the above-described programs may be recorded on nonvolatile recording medium 750. In this case, interface 740 may read the programs from nonvolatile recording medium 750. Then, CPU 710 may directly execute the programs read by interface 740, or may temporarily store the programs in main storage device 720 or auxiliary storage device 730 and then execute them.

[0112] Alternatively, a program for executing all or part of the processing performed by the learning device 100, data generating device 300, and data generating device 610 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform the processing of each unit. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into the computer system. The program may be designed to implement part of the aforementioned functions, or may be capable of implementing the aforementioned functions in combination with a program already recorded on the computer system.

[0113] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention.

[0114] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0115] (Supplementary Note 1) A data generation device comprising: a data acquisition means for acquiring first data and second data classified into the same class as the first data from a plurality of data classified into classes; a perturbation acquisition means for using a neural network including a partial network, a first normalization layer, and a second normalization layer to output features, to acquire a perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding a perturbation to the first data, is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and a data processing means for generating data in which the obtained perturbation is added to the first data.

[0116] (Supplementary Note 2) The data generating device according to Supplementary Note 1, wherein the data acquiring means acquires a plurality of the second data, and the perturbation acquiring means acquires the perturbation such that an average value of a distance between the first feature and each of the second feature is as small as possible.

[0117] (Supplementary Note 3) The data generating device according to Supplementary Note 1 or Supplementary Note 2, further comprising: a parameter value updating means for updating parameter values ​​of the neural network by training the neural network using data obtained by adding the perturbation to the first data.

[0118] (Supplementary Note 4) A learning device comprising: a data acquiring means for acquiring first data and second data classified into the same class as the first data from a plurality of data classified into classes; a perturbation acquiring means for acquiring, using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, a perturbation that minimizes a distance between a first feature, which is a feature obtained when data obtained by adding a perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer; a data processing means for generating data obtained by adding the obtained perturbation to the first data; and a parameter value updating means for updating parameter values ​​of the neural network by training the neural network using data obtained by adding the perturbation to the first data.

[0119] (Supplementary Note 5) A data generation method including: a computer acquires, from a plurality of data classified into classes, first data and second data classified into the same class as the first data; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquires the perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and generates data by adding the obtained perturbation to the first data.

[0120] (Supplementary Note 6) A learning method including: a computer acquires, from a plurality of data classified into classes, first data and second data classified into the same class as the first data; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquires a perturbation that minimizes a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer; generating data by adding the obtained perturbation to the first data; and updating parameter values ​​of the neural network by training the neural network using the data obtained by adding the perturbation to the first data.

[0121] (Supplementary Note 7) A recording medium having recorded thereon a program for causing a computer to execute the following steps: acquiring first data and second data classified into the same class as the first data from among a plurality of data classified into classes; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquiring the perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data, is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; and generating data by adding the obtained perturbation to the first data.

[0122] (Supplementary Note 8) A recording medium having recorded thereon a program for causing a computer to execute the following: acquiring first data and second data classified into the same class as the first data from among a plurality of data classified into classes; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquiring a perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data, is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is minimized; generating data by adding the obtained perturbation to the first data; and updating parameter values ​​of the neural network by training the neural network using the data obtained by adding the perturbation to the first data.

[0123] The present invention may be applied to a data generating device, a data generating method, and a recording medium.

[0124] 100 Learning device 110 Communication unit 120 Display unit 130 Operation input unit 180 Storage unit 181 Model storage unit 182 Common parameter storage unit 183-1 First normalization layer parameter storage unit 183-2 Second normalization layer parameter storage unit 190, 390 Processing unit 191 Data acquisition unit 192 Feature amount calculation unit 193 Perturbation acquisition unit 194 Data processing unit 195 Parameter update unit 300, 610 Data generation device

Claims

1. a data acquisition means for acquiring first data and second data classified into the same class as the first data from among a plurality of classified data; a perturbation acquisition means for acquiring a perturbation using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, such that a distance between a first feature, which is a feature obtained when data obtained by adding a perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; a data processing means for generating data by adding the obtained perturbation to the first data; A data generating device comprising:

2. the data acquisition means acquires a plurality of the second data; the perturbation acquisition means acquires the perturbation such that an average value of a distance between the first feature amount and each of the second feature amounts becomes as small as possible. The data generating device according to claim 1 .

3. The method further comprises a parameter value update means for updating parameter values ​​of the neural network by training the neural network using data obtained by adding the perturbation to the first data.

3. The data generating device according to claim 1.

4. The computer Acquire first data and second data classified into the same class as the first data from among a plurality of classified data; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, to obtain the perturbation such that the distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; generating data by adding the obtained perturbation to the first data; A data generation method comprising:

5. On the computer, acquiring first data and second data classified into the same class as the first data from among a plurality of classified data; using a neural network including a partial network, a first normalization layer, and a second normalization layer and outputting features, acquiring the perturbation such that a distance between a first feature, which is a feature obtained when data obtained by adding the perturbation to the first data is input to the neural network and normalized by the second normalization layer, and a second feature, which is a feature obtained when the second data is input to the neural network and normalized by the first normalization layer, is as small as possible; generating data by adding the obtained perturbation to the first data; A program to execute.