Image generation method, image generation apparatus and program

Abstract

To provide a technique which can secure robustness against adversarial attacks even for images.SOLUTION: An image generation method executes discrimination learning processing for, using a learning image, learning parameters of an image generation model so as to generate an image of quality equal to that of the learning image, and after the discrimination learning processing, classification learning for learning the parameters of an image generation model is executed so as to generate an image robust against adversarial attacks to generate an image using the learnt image generation model.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to techniques for an image generating method, an image generating device, and a program. [Background technology]

[0002] It is known that adding minute, carefully constructed perturbations to an image can cause a machine learning model to produce an incorrect output. Adversarial attacks are a general term for techniques that manipulate inputs so that a machine learning model produces an incorrect output for a given input. For example, by adding carefully designed noise to an image of a "panda," a machine learning model can mistakenly identify it as a "gibbon," even though the human eye cannot tell the difference.

[0003] Techniques for improving robustness against adversarial attacks include adversarial learning, which performs training by applying adversarial attacks to samples in the training data, generating a large number of adversarial samples, and adding these to the training data (see, for example, Non-Technical Document 1), and a method for balancing the improvement of classification accuracy for normal samples that have not been attacked (see, for example, Non-Technical Document 2). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Alexey Kurakin, Ian Goodfellow, et al., "Adversarial machine learning at scale", Proc. International Conference on Machine Learning (ICML), 2017 [Non-Patent Document 2] Hongyang Zhang, Yaodong Yu, et al., "Theoretically principled trade-off between robustness and accuracy", Proc. International Conference on Machine Learning (ICML), 2019 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the conventional techniques, these defenses are measures to make the model robust against adversarial attacks, and there is a problem in that they do not provide any defense against attacks on the images themselves.

[0006] In view of the above circumstances, an object of the present invention is to provide a technique that can ensure robustness against adversarial attacks even for images. [Means for solving the problem]

[0007] One aspect of the present invention is an image generation method that uses training images to perform a discriminative learning process to learn parameters of an image generation model so as to generate images of equivalent quality to the training images, and after the discriminative learning process, performs classification learning to learn parameters of the image generation model so as to generate images that are robust against adversarial attacks, and generates images using the trained image generation model.

[0008] One aspect of the present invention is an image generating device that includes a generator that receives generator input information and performs a process of generating an image; a classifier that performs a process of determining whether the input image is an image generated by the generator; a classification loss calculation unit that calculates a classification loss and a generation loss based on the result of classification by the classifier; a generator coefficient update unit that updates the generator coefficients based on the calculated classification loss or the generation loss; a classifier coefficient classification unit that updates the classifier coefficients based on the calculated training classification loss; a classifier that classifies the input image; a classification loss calculation unit that calculates a classification loss based on the result of classification by the classifier; and an output unit that outputs a generated image generated by the generator.

[0009] One aspect of the present invention is a program for causing a computer to function as the image generating device described above. Effect of the Invention

[0010] According to the present invention, it is possible to ensure robustness against adversarial attacks even for images. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates an example of the configuration of an image generating apparatus according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an overview of a configuration and processing of an embodiment. [Diagram 3] 1 is a flowchart showing an outline of a processing procedure performed in an embodiment. [Figure 4] 11 is a flowchart of an example of a process procedure for discriminative learning according to an embodiment. [Diagram 5] 11 is a flowchart of an example of a classification learning process according to an embodiment. [Figure 6] 11 is a flowchart of an example of a processing procedure for image generation according to an embodiment. [Figure 7] This figure shows the distribution of defensibility for original and generated images for each class (character). [Figure 8]This figure shows the distribution of defensibility for original and generated images for each class (character). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail with reference to the drawings. 1 is a diagram showing an example of the configuration of an image generating device 1 according to the present embodiment. The image generating device 1 includes, for example, an acquisition unit 11, a learning unit 12, a generator input creation unit 13, a generator 14, a classifier 16, a classification loss calculation unit 17, a generator coefficient update unit 18, a classifier coefficient classification unit 19, a classifier 20, a classification loss calculation unit 21, a storage unit 22, and an output unit 23.

[0013] The acquisition unit 11 acquires, for example, images to be used for learning.

[0014] The learning unit 12 performs a process of preparing learning data consisting of images and their labels.

[0015] The generator input creation unit 13 performs a process of creating information to be input to the generator 14 .

[0016] The generator 14 inputs generator input information to the generator 14 and performs processing to generate an image.

[0017] The classifier 16 performs a process of determining whether or not the input image is an image generated by the image generation process.

[0018] The classification loss calculation unit 17 performs a process of evaluating whether or not the result of the image classification process is desirable.

[0019] The generator coefficient update unit 18 performs processing to update the generator coefficients based on the calculated training loss (generation loss or classification loss).

[0020] The classifier coefficient classification unit 19 performs processing for updating the classifier coefficients based on the calculated training classification loss.

[0021] The classifier 20 performs a process of classifying an input image.

[0022] The classification loss calculation unit 21 performs a process of evaluating whether the result of the image classification process is desirable or not.

[0023] The storage unit 22 stores, for example, parameters and programs used by the image generating device 1 in each process and learning.

[0024] The output unit 23 outputs the generated image generated by the generator 15 to an external device. The external device is, for example, an image display device, a mobile terminal, a personal computer, or the like.

[0025] The image generating device 1 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The image generating device 1 functions as an acquisition unit 11, a learning unit 12, a generator input creation unit 13, a generator 14, a discriminator 16, a discrimination loss calculation unit 17, a generator coefficient update unit 18, a discriminator coefficient classification unit 19, a classifier 20, a classification loss calculation unit 21, and an output unit 23 by the processor executing a program. All or part of the functions of the image generating device 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (e.g., SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.

[0026] 2 is a diagram showing an overview of the configuration and processing of this embodiment. In FIG. 2, z is generator input information, G is the generator 14, D is the discriminator 16, and C is the classifier 20. The block diagram of reference symbol g10 shows a process for creating a realistic character image. The generator 14 receives generator input information and outputs a generated image g11. The classifier 16 receives a generated image and outputs a classification result (real or fake) that determines whether the input image is an image generated by the image generation process. A generation label (class y) may be input to the generator 14 and the classifier 16. During learning, the classifier 16 receives a generated image or a learning image (letter image x).

[0027] In this manner, in this embodiment, an image generation model capable of generating natural images is trained by discrimination learning processing, and then an image generation model capable of generating images robust against adversarial attacks is trained by classification learning processing. That is, in this embodiment, a cGAN having a pair of a generator 14 and a discriminator 16 is trained by a standard GAN learning processing for generating realistic character images. Then, the discriminator 16 detects fake images generated by the generator 14. As a result, according to this embodiment, it is possible to generate images that are natural and robust against adversarial attacks.

[0028] The block diagram of the reference symbol g20 shows a process of generating an image robust against an adversarial attack. That is, in this embodiment, the discriminator 16 is replaced with a classifier 20, so that the generator 14 can generate a more defensive character image. In addition, in this embodiment, the classifier 20 that predicts the class label of an image is trained in advance and used. The generator 14 receives generator input information and outputs a generated image g21. The trained classifier 20 receives the generated image g21 and outputs the class label of the image. As a result, according to this embodiment, the possibility that the generated image is classified into an incorrect label is reduced, and as a result, it is possible to generate an image robust against an adversarial attack.

[0029] As shown in FIG. 2, in this embodiment, by using the above two techniques, it is possible to generate an image that is robust against an adversarial attack, thereby strengthening the defense against the adversarial attack. In addition, in this embodiment, by adding a classifier 20 to the GAN-based model, it is possible to generate a character image with higher defense. The learning method and the like will be described later.

[0030] [Explanation of each process] Next, each process used in the learning process and image processing will be described.

[0031] (Learning data preparation process) In the training data preparation process, there is no input, and the output is a set of training samples. i and the training label y i The paired training sample (x i ,y i ), and the set {(x i ,y i} i=1 N The set of training samples is N (N is an integer equal to or greater than 2) in total, and the number of possible labels is K (K is an integer equal to or greater than 2). Also, let x∈{1,2,…,N}-th training image be i ∈R D (R D (The R character indicates a set of D-dimensional real vectors), and the learning labels are y i We express it as ∈{1,2,…,K}.

[0032] (Generator input creation process) In the generator input creation process, there is no input, and the output is the generator input information. Note that in Figure 3, this is represented by the symbol z. The generator input information is, for example, a random vector z generated from a standard normal distribution, which is input information used in the framework of a general generative adversarial network (see, for example, Reference 1). However, this is not limited to this, and other information may be used as long as it is capable of driving the generator 14.

[0033] Reference 1; Ian J. Goodfellow, Jean Pouget-Abadie, et al., “Generative adversarial networks”, Proc. Annual Conference on Neural Information Processing Systems (NeurIPS), 2014

[0034] (Image generation processing) In the image generation process, the inputs are the generator input information and the generator coefficients (+ the generation labels). The generation labels are optional and may or may not be used. The generation labels may be the same as the learning labels, but are generally different. The generator coefficients correspond to the model parameters of the generator 14. The output is the generated image (+ the generation label). The generation label is optional and may or may not be used. When the generation label is input, the generated image and the generation label are output as a pair. 3, the generator 14 is denoted by the symbol G. The generator 14 may use any technique as long as it can generate an image. For example, the generator 14 uses a framework of generative adversarial networks (GANs) in combination with image recognition processing described later. In addition, when a label for generation is given as an additional input, an image is generated using a conditional generative adversarial network framework (see, for example, Reference 2), and the generated image and the label for generation are paired as output.

[0035] Reference 2;Mehdi Mirza, Simon Osindero, “Conditional generative adversarial nets”, arxiv:1411.1784, 2014

[0036] (Image Recognition Processing) In image discrimination processing, the input is a generated image or a learning image, and in addition, a classifier coefficient (+ a learning label or a generation label). Note that the learning and generation labels or the generation labels are optional and may or may not be used. Note that the classifier coefficient corresponds to the model parameter of the classifier 16. In image discrimination processing, input is made, and it is determined whether the generated image or the learning image input to the classifier 16 configured based on the separately inputted classifier coefficient is a generated image. The output is a learning classification result. Note that the classifier 16 is denoted by the symbol D in FIG.

[0037] In this embodiment, the classifier 16 is configured according to the framework of a generative adversarial network to determine whether an image is generated or not. If a learning label or a generation label is additionally input, the classifier 16 is configured according to the framework of a conditional generative adversarial network, as in the image generation process. In other words, when classifying an image, the label can be used as a condition.

[0038] The classifier 16 performs, for example, binary classification. The learning classification result, which is the output, is r^ i We denote it as ∈{0,1}, where 0 indicates that the image was determined to be a generated image and 1 indicates that the image was determined to be a training image. Similarly, we also introduce the correct answer result that indicates whether the input image was actually a training image or a generated image, and r i ∈{0,1}. In addition, the confidence level for determining that an image is a learning image is a real number between 0 and 1, that is, 0≦r^ i It can also be expressed as ≦1. In this case, if it is 1, the confidence level that the image is a training image (real) is 100%, and if it is 0, the confidence level that the image is a generated image (fake) is 100%.

[0039] On the other hand, if no additional learning or generation labels are input, in addition to determining whether the input image is a generated image, the class label of the image may be predicted. This predicted label is y^ i∈{1,2,…,K}. Also, predictions can be expressed not by the labels themselves, but by the confidence for each label, expressed as a real number between 0 and 1. In this case, the sum of the confidence for all classes is expected to be 1. If learning labels are given, they may be used to determine whether the class is real or fake.

[0040] (Discrimination loss calculation process) In the classification loss calculation process, the inputs are the training classification result and information (+ training label) indicating whether the input to the image classification process is a generated image or a training image. Note that the training label is optional and may or may not be used. The outputs are a training classification loss and a training generation loss. That is, in the classification loss calculation process, a training classification loss for evaluating whether or not the training classification result was a desirable result and a training generation loss for evaluating whether or not the image generated by the generator 14 is judged as a training image by the classifier 16 are calculated, and the calculated training classification loss and training generation loss are output.

[0041] In addition, the classification loss calculation unit 17 calculates, for example, the correct classification result r i and predicted classification result r^ i Based on the deviation from the training loss, we calculate the training loss for the discriminative loss and the training loss for the generative loss. In this calculation, various losses used in the framework of generative adversarial networks can be used, such as Wasserstein GAN with gradient penalty (WGAN-GP) (see, for example, Reference 3).

[0042] Reference 3; Ishaan Gulrajani, Faruk Ahmed, et al., “Improving training of Wasserstein GANs”, Proc. Conference on Neural Information Processing Systems (NIPS), 2017

[0043] When the input to the image classification process is a generated image and the training classification result is fake, this indicates that the classification loss is small and the generation loss is large. When the input to the image classification process is a training image and the training classification result is real, this shows that the classification loss is small and the generation loss is large. If the above is not the case, it indicates that the discrimination loss is large and the generation loss is small.

[0044] In addition, when learning labels are given, it is possible to assume that the label of the input image is predicted in the image recognition process, and add to the recognition loss how close the prediction result is to the learning labels. Alternatively, if the training labels are given as additional inputs and the labels of the input images are also predicted during the image classification process, the training labels y i and the predicted label y^ i The deviation from the original training loss can also be added to the training loss. For this calculation, for example, cross entropy or Kullback-Leibler divergence is used.

[0045] (Generator coefficient update process) In the generator coefficient update process, the inputs are the training generation loss or training classification loss and the current generator coefficients. The output is the updated generator coefficients based on the input losses.

[0046] Since the image generation process is assumed to be represented by a neural network, the generator coefficients corresponding to the model parameters of the generator 14 used in the image generation process can be updated by backpropagating the training generation loss or training classification loss to the neural network.

[0047] (Classifier coefficient update process) In the classifier coefficient update process, the inputs are the training classification loss and the current classifier coefficients. The output is the updated classifier coefficients based on the input losses.

[0048] Since the image classification process is assumed to be represented by a neural network, the classifier coefficients corresponding to the model parameters of the classifier 16 used in updating the classifier coefficients can be updated by backpropagating the training classification loss to the neural network.

[0049] (Image classification processing) In the image classification process, the inputs are the generated image and the classifier coefficients. The classifier coefficients correspond to the model parameters of the classifier 20. Note that the classifier coefficients are fixed model parameters that have been separately trained. Note that a new process for training the classifier 20 may be provided depending on the application and necessity. In the image classification process, the classifier 20 predicts the class label of the generated image and outputs a classification result for learning that corresponds to this predicted label.

[0050] The classifier 20 may be, for example, a general neural network. However, the classifier 20 must be trained in advance to perform accurate classification. The classifier 20 is trained, for example, by training images or images to be generated and training labels to learn parameters. The classifier 20 is trained, for example, by a Convolutional Neural Network (CNN) having two convolutional layers and two fully connected layers for classifying, for example, 26 letters of the alphabet. Each convolutional layer involves ReLU (Rectified Linear Unit) activation and max pooling. In the training process, for example, a negative log-likelihood loss is used and is minimized by the optimization method AdaDelta. The training process is terminated, for example, by a standard early stopping criterion, and the training is stopped if the validation loss does not decrease over the next 10 epochs.

[0051] Also, the classifier 20 may output a confidence level for each label expressed as a real number between 0 and 1, rather than predicting the label itself. That is, in the image classification process, a predicted class label may be output, or a confidence level for each label may be output.

[0052] (Classification loss calculation process) In the classification loss calculation process, the inputs are the training classification results and the training labels. In the classification loss calculation process, a training classification loss for evaluating whether the training classification result is a desirable result or not is calculated, and the calculated training classification loss is output. When the predicted label in the classifier 20 matches the training label, it indicates a small loss value. When the predicted label in the classifier 20 does not match the training label, it indicates a large loss value.

[0053] Since the image classification process is assumed to be represented by a neural network, the training classification loss corresponding to the model parameters of the classifier 20 used in the image classification process can be updated by backpropagating the training classification loss to the neural network.

[0054] The training classification loss is calculated, for example, by i and the predicted label y^ obtained as the training classification result. i The deviation from the original is calculated. For this calculation, for example, cross entropy or Kullback-Leibler divergence is used.

[0055] The loss function of the generator 14 is, for example, the following formula (1): The loss function of the discriminator 16 is, for example, the following formula (2):

[0056]

number

[0057]

number

[0058] In equations (1) and (2), p(x,y) is the joint distribution of (x,y), p(z,y) is the joint distribution of (z,y), and z is a random noise vector.

[0059] Moreover, the loss function of the classifier 20 is the negative log-likelihood of the following equation (3).

[0060]

number

[0061] In formula (3), E (double underlined letter) (z、y)~p(z、y) [·], E (double-line character) (x、y)~p(x、y) [·] means the expected value of [·]. C y (x) is the y-th element of the logit vector obtained from the classifier 20. The idea of ​​this loss function, equation (3), is that we want to train the generator 14 to output character images that minimize the classification loss, i.e., the generated character images are more likely to be recognized by the classifier 20 (C).

[0062] Here, as a comparative example, a conventional Fast Gradient Sign Method (FGSM) (see, for example, Reference 4) will be described. In FGSM, in order to generate an adversarial example x', the gradient ∇ θ We use J(θ,x,y), where θ is the parameter of the target model, and J(θ,x,y) is the loss function for model training. Specifically, the adversarial example x' generated from x by FGSM is expressed as follows:

[0063]

number

[0064] The second term on the right-hand side of equation (4) is the adversarial perturbation, and ε controls the amplitude of the perturbation. The perturbed example x' is expected to give a larger loss value than x. Hence, x' can act as an adversarial example.

[0065] Reference 4; Ian J. Goodfellow, Jonathon Shlens, et al., “EXPLAINING AND HARNESSING ADVERSARIAL EXAMPLES”, In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) 2015, 2015

[0066] In the comparative example, adversarial examples can be generated by repeating the FGSM in equation (4). In contrast, in this embodiment, the meaning of the formula (3) is the opposite of the attacking movement represented by the formula (4), so the generated image has a stronger defensive capability.

[0067] [Processing Procedure] First, an outline of the processing procedure performed in this embodiment will be described with reference to Fig. 3, which is a flowchart showing the outline of the processing procedure performed in this embodiment. (Step S1) The image generating device 1 performs discriminative learning to learn a model that generates an image of quality comparable to the learning image. (Step S2) The image generating device 1 performs classification learning to learn a model that generates an image that is robust against adversarial attacks. (Step S3) The image generating device 1 generates an image using the trained image generation model.

[0068] (Discrimination learning process) FIG. 4 is a flowchart of an example of the discriminative learning processing procedure of this embodiment. (Step S11) The learning unit 12 performs a learning data preparation process. (Step S12) The generator input creation 13 performs a generator input creation process. (Step S13) The generator 14 performs image generation processing. (Step S14) The classifier 16 performs image classification processing on the generated image. (Step S15) The discrimination loss calculation unit 17 calculates a discrimination loss and a generation loss based on the result of the image discrimination process. (Step S16) The generator coefficient update unit 18 performs a generator coefficient update process based on the generation loss. (Step S17) The classifier coefficient update unit 19 performs a classifier coefficient update process based on the classification loss. After the process, the image generating device 1 returns to the process of step S12.

[0069] (Classification learning process) FIG. 5 is a flowchart of an example of the classification learning processing procedure of this embodiment. (Step S21) The generator input creation 13 performs a generator input creation process. (Step S22) The generator 14 performs image generation processing. (Step S23) The classifier 20 performs image classification processing. (Step S24) The classification loss calculation unit 21 performs a classification loss calculation process. (Step S25) The generator coefficient update unit 18 performs a generator coefficient update process. After the process, the image generating device 1 returns to the process of step S21.

[0070] (Image generation processing) FIG. 6 is a flowchart of an example of a processing procedure for generating an image according to this embodiment. (Step S31) The generator input creation 13 performs a generator input creation process. (Step S32) The generator 14 performs image generation processing. (Step S33) Output unit 23 outputs the generated image to an external device.

[0071] [evaluation] Next, the results of evaluation of the image generating device 1 described above will be explained. In the evaluation, we collected binary images (64 × 64 pixels) of English uppercase letters from over 3,000 fonts included in Google (registered trademark) Fonts (https: / / fonts.google.com) and divided them into 30,000 images for training, 4,000 images for validation, and 30,000 images for testing. In the evaluation, we used character types (A, B, ..., Z) as class labels and used them as conditions for image generation based on a conditional generative adversarial network.

[0072] In order to evaluate the robustness against adversarial attacks, a measure called defensibility was defined and used in the evaluation. For this defensibility, an adversarial attack method called the Fast Gradient Sign Method (see, for example, Reference 4) is used. More specifically, attacks are repeatedly performed on the target image until the predicted class of the classifier when the attacked image is input is other than the correct class, and the number of repetitions is defined as defensibility. In the evaluation, the larger the defensibility value, the more robust the target image is against adversarial attacks. Note that, by definition, images whose predicted class of the classifier 20 does not match the correct class must be excluded from the evaluation targets in advance.

[0073] 7 and 8 are diagrams showing the distribution of defensibility of real images (original) and generated images (generated) for each class (character). In FIG. 7 and FIG. 8, the histograms of codes g101 to g126 are histograms of the defensive strength of 1,000 original images (code g131) and 1,000 generated images (code g132) for each character, and the images in the lower part of the histograms are the top 8 generated images with the highest defensive strength. The histogram of code g140 is a histogram of the defensibility value (k) and the total number of samples (items) for 26 characters. In the histograms of codes g101 to g126 and code g140, the horizontal axis is the defensibility value (k) and the vertical axis is the number of samples (items), as in code g130. The defensibility value (k) was counted by attacking the original or generated image with a simplified version of PGD (Projected Gradient Descent), a common iterative attack algorithm (see, for example, Reference 5).

[0074] 7 and 8, it can be seen that this embodiment can generate images with higher defensibility than the actual images in almost all classes (characters). Furthermore, it can be seen that this embodiment has a smaller variance even in classes where the defensibility mode of the generated images is close to that of the actual images, and the number of images with low defensibility is greatly reduced.

[0075] Reference 5; Aleksamder Madry, Aleksandar, et al., “Towards deep learning models resistant to adversarial attacks”, In: Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018

[0076] As described above, in this embodiment, two-stage learning is performed: discriminative learning processing and categorical learning processing. That is, in this embodiment, an image generation model capable of generating natural images is trained in discriminative learning processing, and then an image generation model capable of generating images that are robust against adversarial attacks is trained in categorical learning processing.

[0077] As a result, according to this embodiment, it is possible to generate images that are natural yet robust against adversarial attacks.

[0078] As described above, in this embodiment, in addition to the above processes, image classification process and classification loss calculation process are performed. That is, in this embodiment, in the classification learning process at the latter stage, the classifier 20 that predicts the class label of the image is trained and used, instead of the discriminator 16 that distinguishes whether the image is a generated image or not.

[0079] According to this embodiment, a two-stage learning process, that is, a discriminative learning process and a classification learning process, can generate images that are natural yet robust against adversarial attacks. Furthermore, according to this embodiment, the image classification process and classification loss calculation process reduce the possibility that the generated image will be classified into an incorrect label, thereby making it possible to generate images that are robust against adversarial attacks. Furthermore, in this embodiment, since the meaning of the expression (3) is the opposite of the attacking movement represented by the expression (4), the generated image has a higher defensive power.

[0080] Moreover, according to this embodiment, unlike the conventional technology, it is possible to strengthen the defense against adversarial attacks by generating an image that is robust against the adversarial attacks. For example, when considering character images, an image that is robust against adversarial attacks leads to the proposal of a "character design" that is unlikely to be misclassified as another character type even when subjected to an adversarial attack, and is important as a new character design in the age of artificial intelligence. Therefore, the above can be achieved by generating character images using the method of this embodiment.

[0081] (Modification) In the present embodiment, the character string is described using capital letters of the alphabet, but is not limited to this. The method of the present embodiment can also be applied to other characters, such as lowercase letters of the alphabet and Greek letters.

[0082] In the above example, a character image has been described as an example of a generated image, but the present invention is not limited to this. The method of the present embodiment can also be applied to other images, such as pattern and design images.

[0083] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Industrial Applicability]

[0084] The present invention is applicable to an image generating device, an image identifying device, an image classifying device, an image generating method, an image identifying method, an image classifying method, and the like. [Explanation of symbols]

[0085] 1...image generating device, 11...acquisition unit, 12...learning unit, 13...generator input creation unit, 14...generator, 16...discriminator, 17...discrimination loss calculation unit, 18...generator coefficient update unit, 19...discriminator coefficient classification unit, 20...classifier, 21...classification loss calculation unit, 22...storage unit, 23...output unit

Claims

1. performing a discrimination learning process for learning parameters of an image generation model using a learning image so as to generate an image of a quality equivalent to that of the learning image; After the discriminative learning process, classification learning is performed to learn parameters of the image generation model so as to generate an image that is robust against adversarial attacks; Generate an image using the trained image generation model. Image generation method.

2. A generator, which is an image generation model, generates an image using the input learning image, A classifier performs an image classification process on the generated image to determine whether or not an input image is an image generated by the generator; A classification loss calculation unit calculates a classification loss and a generation loss based on the image classification processing result, a generator coefficient update unit performs a generator coefficient update process, which is a parameter of the generator, based on the generation loss; a classifier coefficient update unit performs a classifier coefficient update process based on the classification loss; The image generating method according to claim 1 .

3. A generator generates an image using the input learning data, A classifier classifies the generated image; A classification loss calculation unit calculates a classification loss based on the classification result. A generator coefficient update unit performs a generator coefficient update process, which is a parameter of the generator, based on the classification loss. The image generating method according to claim 1 .

4. A generator, which is an image generation model, generates an image using the input learning data, and assigns the input generation label to the generated image. A classifier performs an image classification process for determining whether an input image is an image generated by the generator, using the generated generated image and the generation label; A classification loss calculation unit calculates a classification loss and a generation loss based on the image classification processing result, a generator coefficient update unit performs a generator coefficient update process, which is a parameter of the generator, based on the generation loss; a classifier coefficient update unit performs a classifier coefficient update process based on the classification loss; The image generating method according to claim 1 .

5. The image classification processing result is If it is a training image, it is true. If it is a generated image, it is false. Alternatively, the confidence that the condition is true or false may be a numerical value. The image generating method according to claim 2.

6. The result calculated by the discrimination loss calculation unit is When an input to the classifier is the generated image and the classification result is false, the classification loss is small and the generation loss is large; When an input to the classifier is a learning image and the classification result is true, the classification loss is small and the generation loss is large; When an input to the classifier is not the generated image and the classification result is not false, or when an input to the classifier is not the learning image and the classification result is not true, the classification loss is large and the generation loss is small. The image generating method according to claim 2.

7. A generator that receives generator input information and performs processing to generate an image; a classifier that performs a process of determining whether an input image is an image generated by the generator; a discrimination loss calculation unit that calculates a discrimination loss and a generation loss based on a result of discrimination by the discriminator; a generator coefficient update unit that updates a generator coefficient based on the calculated discrimination loss or the generation loss; a classifier coefficient classification unit that updates the classifier coefficients based on the calculated training classification loss; A classifier that classifies an input image; A classification loss calculation unit that calculates a classification loss based on a result of classification by the classifier; an output unit that outputs a generated image generated by the generator; An image generating device comprising:

8. Computer, A program for causing the image generating device according to claim 7 to function.

Images