Image generation method and device, storage medium and computer program product

Abstract

The invention provides an image generation method and device, a storage medium and a computer program product, and the method comprises the steps: inputting random data and label information into a generator, and obtaining generated data; inputting the generated data and real data corresponding to the label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by judging the generated data and the true data; training a generator and a discriminator by using the first category probability information and the true and false probability information; and after the training is completed, the trained generator is utilized to generate vivid image data. The diversity of generated images can be improved, and it is ensured that the generative adversarial network can be applied to a large-scale data set.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image generation method and device, a storage medium, and a computer program product. Background Art

[0002] The idea of ​​Generative Adversarial Networks (GAN) comes from game theory. The role of the generator is to generate realistic images to deceive the discriminator by learning the distribution of real sample data. The task of the discriminator is to distinguish whether the image is real or generated by the generator. It is essentially a binary classification network. Through adversarial training between the generator and the discriminator, the capabilities of both parties are continuously improved, so that the generated distribution continues to approach the real distribution, and finally reaches the ideal state of Nash equilibrium, that is, the two distributions completely overlap.

[0003] The original GAN ​​is an unsupervised model, which makes the generated samples too random and unable to control the generation of specified samples. Therefore, GAN was improved by inputting label information and noise into the generator at the same time. Due to the introduction of conditional constraints, the model can generate images corresponding to specific labels. However, as the number of labels increases, the diversity of images generated by the model decreases, resulting in the inability of generative adversarial networks to be applied on large-scale datasets. Summary of the invention

[0004] The present application provides an image generation method and device, a storage medium, and a computer program product, which can improve the diversity of generated images and ensure that the generative adversarial network can be applied on large-scale data sets.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, the present application proposes an image generation method, the method comprising:

[0007] Input random data and label information into the generator to obtain generated data;

[0008] Input the generated data and the real data corresponding to the label information into the discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data;

[0009] The generator and the discriminator are trained using the first category probability information and the true and false probability information; and after the training is completed, realistic image data is generated using the trained generator.

[0010] In a second aspect, the present application provides an image generating device, the image generating device comprising:

[0011] A generating unit, used for inputting random data and label information into a generator to obtain generated data;

[0012] A discriminator, configured to input the generated data and the real data corresponding to the label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data;

[0013] A training unit is used to train the generator and the discriminator using the first category probability information and the true and false probability information; and after the training is completed, using the trained generator to generate realistic image data.

[0014] In a third aspect, the present application proposes an image generating device, which includes: a processor, a memory and a communication bus; the communication bus is used to realize the connection and communication between the processor and the memory; the processor implements the above-mentioned image generating method when executing the running program stored in the memory.

[0015] In a fourth aspect, the present application proposes a storage medium having a computer program stored thereon, which implements the above-mentioned image generation method when executed by a processor.

[0016] In a fifth aspect, the present application proposes a computer program product, comprising a computer program, which implements the above-mentioned image generation method when executed by a processor.

[0017] The present application provides an image generation method and device, storage medium, and computer program product. The method includes: inputting random data and label information into a generator to obtain generated data; inputting real data corresponding to the generated data and label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data; using the first category probability information and the true and false probability information to train the generator and the discriminator; and after the training is completed, using the trained generator to generate realistic image data. Using the above implementation scheme, a classification process is introduced in the discriminator, and the classification process only classifies the generated data and competes with the generator. This method can force the binding of label information and data content, thereby forcing the generator to generate images whose content is highly matched with the label information, improving the diversity of the generated images, and thus ensuring that the generative adversarial network can be applied on large-scale data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of an image generation method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of an exemplary dense residual network structure provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of an exemplary coordinate attention structure provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of an exemplary generator provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of an exemplary discriminator structure provided in an embodiment of the present application Figure 1 ;

[0023] Figure 6 A schematic diagram of an exemplary discriminator structure provided in an embodiment of the present application Figure 2 ;

[0024] Figure 7 A network diagram of an exemplary generative adversarial network provided in an embodiment of the present application;

[0025] Figure 8 A schematic diagram of the structure of an image generating device provided in an embodiment of the present application Figure 1 ;

[0026] Fig. 9 A schematic diagram of the structure of an image generating device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0027] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0029] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second\third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0030] The present application embodiment provides an image generation method, such as Figure 1 As shown, the method may include:

[0031] S101. Input random data and label information into a generator to obtain generated data.

[0032] It should be noted that the generative adversarial network consists of a generator and a discriminator, wherein the function of the generator is to generate realistic false images to deceive the discriminator. In an embodiment of the present application, the generator includes at least one of the following network structures: a residual network, a coordinate attention layer.

[0033] In an embodiment of the present application, the residual network may be a dense residual network.

[0034] In the embodiment of the present application, the number of residual networks can be one or more, which is selected according to actual conditions. The embodiment of the present application does not limit the number of residual networks.

[0035] It should be noted that as the number of network layers increases, the accuracy of the model will decrease. This problem can be alleviated by using a dense residual network. The dense residual network densely connects all the previous network layers with the subsequent network layers, so it can retain richer image features. This operation helps the reverse propagation of gradients, thereby deepening the depth of the convolutional neural network and improving accuracy. In addition, the dense residual network can achieve feature reuse by connecting features on the channel, which makes the dense residual network have fewer parameters and more streamlined calculations.

[0036] For example, the dense residual network structure is as follows Figure 2As shown, the input feature data X0 passes through the H1 network structure, the output feature data X1, X0 and X1 pass through the H2 network structure, the output feature data X2, X0, X1, X2 pass through the H3 network structure, the output feature data X3, X0, X1, X2, X3 pass through the H4 network structure, the output feature X4, X0, X1, X2, X3, X4 pass through the transition layer (Transitionlayer), and the final result is output. Among them, the four network structures H1, H2, H3, and H4 are all composed of batch normalization layer (BatchNormalize, BN) + rectified linear unit (Rectified Linear Unit, Relu) + convolution (Convolution, Conv). I will not go into details here.

[0037] It can be understood that the residual network is built into the generator to aggregate feature information, thereby retaining important feature information.

[0038] It should be noted that in order to retain important feature information and significantly improve model performance, coordinate attention is used in both the generator and the discriminator. The coordinate attention can capture position information and channel information from two dimensions, space and channel, and embed the position information into the channel attention, thereby improving the quality and effect of image generation. This attention method decomposes the channel attention into two parallel one-dimensional feature encoding processes, which can capture long-distance dependencies along the spatial direction, and verify the precise position information obtained in another direction, and finally merge the two to enhance the expressiveness of the features.

[0039] For example, the structure of coordinate attention is as follows Figure 3 As shown in the figure, in order to obtain the attention on the width and height of the image and encode the precise position information, the input feature map C×H×W is firstly subjected to global average pooling (X Avg Pool and Y Avg Pool) in the width and height directions respectively, and the feature maps C×H×1 and C×1×W in the width and height directions are obtained respectively. Then, the feature maps in the two directions are concatenated and convolved (Concat+ConV2d) to obtain the feature map of C / r×1×(W+H), and then normalized and nonlinearized (BatchNorm+Non-linear) are performed, and the feature map size is still C / r×1×(W+H), and then Conv2d and Sigmod are performed in the width and height directions respectively, and finally the weight adjustment (reweight) operation is performed together with the input features to output the image with important features marked.

[0040] For example, the structural diagram of the generator proposed in the embodiment of the present application is shown in Figure 4, including: convolution layer (Conv), RDB, upsampling layer (UpSampling), RDB, upsampling layer, RDB, upsampling layer, coordinate attention layer (Coordinate Attention), convolution layer + Tanh function. Among them, RDB includes four skip connection network structures, the first three of which are composed of Conv+BN+ReLU; the fourth network structure is composed of Concat+Conv; finally, the output data of the fourth network structure is added to the input data of RDB to output the final data processed by RDB.

[0041] In an embodiment of the present application, random data and label information are combined as input to the generator, and then input into the RDB after passing through a convolution layer. The RDB jumps the features of each layer with all the previous layers. This method helps the back propagation of gradients in the training project. Then it passes through the upsampling layer. After training the above three RDB+upsampling operations, it passes through a coordinate attention layer and a convolution layer, and finally outputs the generated data.

[0042] It should be noted that the generated data is generated image data.

[0043] S102. Input the generated data and the real data corresponding to the label information into the discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data.

[0044] In the embodiment of the present application, the task of the discriminator is to distinguish whether the image comes from the generator, calculate the loss and return the gradient.

[0045] In one embodiment, the discriminator includes an N-layer network structure; wherein the N-th network structure includes a parallel convolution layer and a first softmax layer, and the generated data and the real data corresponding to the label information are input into the discriminator to obtain first category probability information and true and false probability information, including: inputting the generated data and the real data into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, respectively inputting the generated data into the first softmax layer to obtain the first category probability information, and inputting the generated data and the real data into a convolution layer to obtain the true and false probability information.

[0046] In an embodiment of the present application, the first N layers of the network structure of the discriminator include at least one of the following: a spectral normalization layer and a coordinate attention layer.

[0047] It is understandable that the spectral normalization layer can effectively limit the severity of function changes and greatly improve the stability of model training.

[0048] It should be noted that the parameters W of each layer in the neural network are decomposed by SVD, and then its maximum singular value is limited to 1 to satisfy the 1-Lipschiz condition. Then, each time W is updated, it is divided by the maximum singular value. In this way, the stretching coefficient of each layer for the input x will be less than 1, thereby achieving Lipschiz continuity.

[0049] It can be understood that adding coordinate attention in the discriminator part imposes complex conditional constraints on the image, thereby improving the accuracy of discrimination.

[0050] For example, the structure of the discriminator is as follows Figure 5 As shown, it includes: Conv+LeakyReLU, two Conv+Spectral Normalization (SN)+LeakyReLU, Coordinate Attention, Conv+SN+LeakyReLU, Coordinate Attention, the last layer is a parallel Conv layer and the first softmax layer.

[0051] In an embodiment of the present application, the generated data and the real data are input into the Conv layer of the last layer after passing through the first N layers of the discriminator structure, and the true and false probability information is output to characterize the true and false of the image; at the same time, the generated data is processed by the first N layers of the discriminator structure, and the processing result is input into the first softmax layer of the last layer for classification to obtain the first category probability information, which is used to characterize the classification result of the false sample.

[0052] In an embodiment of the present application, the true or false probability information can be a number between 0 and 1. If the input data is real data Xreal, then the output number becomes closer and closer to 1. If the input data is forged generated data Xfake, then the output number becomes closer and closer to 0.

[0053] In the embodiment of the present application, the first category probability information may be a k-dimensional class probability vector p=(p1, p2, ..., pj, ..., pk), where k is the type of the sample.

[0054] In another embodiment, the discriminator includes an N-layer network structure; wherein the N-th network structure includes a parallel convolution layer, a first softmax layer and a second softmax layer, and after the random data and label information are input into the generator to obtain the generated data, the following steps are further performed: the generated data and the real data are input into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, the generated data are respectively input into the first softmax layer to obtain the first category probability information, the generated data and the real data are input into a convolution layer to obtain the true and false probability information, and the generated data and the real data are input into the second softmax layer to obtain the second category probability information; the generator and the discriminator are trained using the first category probability information, the second category probability information and the true and false probability information.

[0055] It should be noted that the discriminator plus the second softmax layer constitutes an existing Auxiliary Classifier Generative Adversarial Network (ACGAN) network with an auxiliary classifier. The embodiment of the present application can add a first softmax layer on the basis of the existing ACGAN network to classify only the generated data to obtain the corresponding first category probability information.

[0056] It should be noted that see Figure 5 ,like Figure 6 As shown in the figure, the structure of the discriminator includes: Conv+LeakyReLU, two Conv+SN+LeakyReLU, Coordinate Attention, Conv+SN+LeakyReLU, CoordinateAttention, the last layer is a parallel Conv layer, the first softmax layer and the second softmax layer.

[0057] It should be noted that the network structures of the first softmax layer and the second softmax layer are the same, but the configuration parameters may be the same, different, or partially the same and partially different. Therefore, the first category probability information of the generated data output by the first softmax layer and the second category probability information output by the second softmax layer may be the same or different. The specific selection can be made according to the actual situation, and the embodiment of the present application does not make specific limitations.

[0058] In an embodiment of the present application, the generated data and the real data are input into the Conv layer of the last layer after passing through the first N layers of the discriminator structure, and the true and false probability information is output to characterize the true and false of the image; the generated data and the real data are input into the second softmax layer, and the second category probability information is output to characterize the probability that the sample belongs to a certain category; at the same time, the generated data is processed by the first N layers of the discriminator structure, and the processing result is input into the first softmax layer of the last layer for classification, and the first category probability information is obtained to characterize the classification result of the false sample.

[0059] It can be understood that the use of convolutional layers in both the generator and the discriminator of the present application to replace the fully connected layers with a large number of parameters can greatly improve the training stability and training speed of the generative adversarial network model.

[0060] S103, using the first category probability information and the true and false probability information to train the generator and the discriminator; and after the training is completed, using the trained generator to generate realistic image data.

[0061] In an embodiment of the present application, the discrimination loss is determined using true and false probability information; the first category probability information is used to determine the first classification loss; and the discrimination loss and the first classification loss are used to train the generator and the discriminator.

[0062] In an embodiment of the present application, the process of training a generator and a discriminator using a discriminant loss and a first classification loss includes: inputting the discriminant loss and the first classification loss into an objective function to obtain a function value; the objective function is composed of at least a discriminant loss term and a first classification loss term; and training the generator and the discriminator through the function value.

[0063] In the embodiment of the present application, the first classifier C T Only the generated data is classified. The last layer of the classifier is a softmax network, which can predict the probability that the generated label belongs to each class, and form a game between it and the generator, as shown in formula (1).

[0064]

[0065] Among them, G is the generator, D is the discriminator, E is the mathematical expectation, Y represents the label information, Z is the random Gaussian noise, P Z To generate the sample distribution, P Y is the predicted label distribution.

[0066] In another embodiment, the second category probability information is also used to determine the second classification loss. After that, the process of training the generator and the discriminator using the discriminant loss, the first classification loss and the second classification loss includes: inputting the discriminant loss, the first classification loss and the second classification loss into the objective function to obtain the function value; the objective function is composed of at least the discriminant loss term, the first classification loss term and the second classification loss term; and training the generator and the discriminator through the function value.

[0067] Based on formula (1), the objective function can refer to formula (2).

[0068]

[0069] Among them, X is the real sample, C is the second classifier, and λ c and λ h A hyperparameter introduced to balance the loss of true and false discrimination and the loss of auxiliary classification.

[0070] It should be noted that the optimization process for the generative adversarial network can be as follows: first, fix the generator G and train the discriminator D; then fix the discriminator D and train the generator G. The two are optimized alternately. Generally, it is necessary to train the discriminator k times and then train the generator once. The label information can be effectively utilized. Not only can the generator G be specified to generate images with a certain type of label, but the label information can also be used in the discriminator D to expand the entire loss function, thus improving the performance of the entire generative adversarial network. In the above-mentioned optimization iterative learning process, the purpose of G is to generate a fake generated sample Xfake image that is closer to the real sample Xreal image through random noise, and the purpose of D is to distinguish and discriminate the real sample Xreal from the fake generated sample Xfake image generated by G. The result of the confrontation is that the probability that D is true for the real sample Xreal and the generated fake generated sample Xfake image is equal.

[0071] It can be understood that a classification process is introduced in the discriminator, which only classifies the generated data and competes with the generator. This method can enforce the bundling of label information and data content, thereby forcing the generator to generate images whose content is highly matched with the label information, improving the diversity of the generated images, thereby ensuring that the generative adversarial network can be applied on large-scale datasets.

[0072] Based on the above embodiments, the present application embodiment proposes a generative adversarial network, such as Figure 7 As shown, it includes the generator Generator G, the discriminator Discriminator D, and the first classifier Classifier C 1 , the second classifier Classifier C 2Random noise Noise z and label information Label y are input to Generator G, which outputs Fake (i.e. generated data). Fake and Real (real data) are input to Discriminator D, which outputs Real / Fake that represents the authenticity of the image. Fake and Real are input to Classifier C. 1 , output Label1 (second category probability information), and input Fake into ClassifierC 2 , output Label2 (first category probability information).

[0073] It should be noted that Discriminator D, Classifier C 1 、Classifier C 2 The structure is shared and separated only at the last level. Figure 7 It is divided only according to function and does not represent complete structural independence.

[0074] The present application embodiment provides an image generating device. Figure 8 As shown, the image generating device 1 comprises:

[0075] A generating unit 10, used for inputting random data and label information into a generator to obtain generated data;

[0076] A discriminator 11 is used to input the generated data and the real data corresponding to the label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data;

[0077] The training unit 12 is used to train the generator and the discriminator using the first category probability information and the true and false probability information; and after the training is completed, use the trained generator to generate realistic image data.

[0078] Optionally, the discriminant unit 11 is also used to input the generated data and the real data into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, respectively input the generated data into the first softmax layer to obtain the first category probability information, and input the generated data and the real data into the first convolution layer to obtain the true and false probability information.

[0079] Optionally, the discriminator includes an N-layer network structure; wherein the N-th layer network structure includes a convolution layer, a first softmax layer and a second softmax layer in parallel;

[0080] The discriminator 11 is further configured to input the generated data and the real data into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, respectively input the generated data into the first softmax layer to obtain the first category probability information, input the generated data and the real data into the first convolution layer to obtain the true and false probability information, and input the generated data and the real data into the second softmax layer to obtain the second category probability information;

[0081] The training unit 12 is further used to train the generator and the discriminator using the first category probability information, the second category probability information and the true and false probability information.

[0082] Optionally, the training unit 12 is further used to determine the discrimination loss using the true and false probability information; determine the first classification loss using the first category probability information; and train the generator and the discriminator using the discrimination loss and the first classification loss.

[0083] Optionally, the training unit 12 is further used to input the discriminant loss and the first classification loss into an objective function to obtain a function value; the objective function is composed of at least a discriminant loss term and a first classification loss term; and the generator and the discriminator are trained through the function value.

[0084] Optionally, the generator includes at least one of the following network structures: a residual network and a coordinate attention layer.

[0085] Optionally, the first N layers of the network structure of the discriminator include at least one of the following: a spectral normalization layer and a coordinate attention layer.

[0086] An image generation device provided in an embodiment of the present application inputs random data and label information into a generator to obtain generated data; inputs real data corresponding to the generated data and label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data; the generator and the discriminator are trained using the first category probability information and the true and false probability information; and after the training is completed, realistic image data is generated using the trained generator. It can be seen that the image generation device proposed in this embodiment introduces a classification process in the discriminator, and the classification process only classifies the generated data and competes with the generator. This method can enforce the binding of label information and data content, thereby forcing the generator to generate images with highly matched content and label information, improving the diversity of generated images, thereby ensuring that the generative adversarial network can be applied on large-scale data sets.

[0087] Fig. 9A schematic diagram of the structure of an image generating device 1 provided in an embodiment of the present application Figure 2 In practical applications, based on the same disclosure concept of the above embodiments, Fig. 9 As shown, the image generating device 1 of this embodiment includes: a processor 13 , a memory 14 and a communication bus 15 .

[0088] The processor 13 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the processor function may also be other, which is not specifically limited in this embodiment.

[0089] In the embodiment of the present application, the communication bus 15 is used to realize the connection and communication between the processor 13 and the memory 14; when the processor 13 executes the running program stored in the memory 14, the following image generation method is realized:

[0090] Input random data and label information into the generator to obtain generated data; input the generated data and the real data corresponding to the label information into the discriminator to obtain first category probability information and true and false probability information; wherein, the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data; use the first category probability information and the true and false probability information to train the generator and the discriminator; and after the training is completed, use the trained generator to generate realistic image data.

[0091] Furthermore, the discriminator includes an N-layer network structure; wherein the N-th layer network structure includes a convolution layer and a first softmax layer in parallel,

[0092] The above-mentioned processor 13 is also used to input the generated data and the real data into the discriminator. After passing through the first N-1 layers of the network structure of the discriminator, the generated data is input into the first softmax layer to obtain the first category probability information, and the generated data and the real data are input into the first convolution layer to obtain the true and false probability information.

[0093] Furthermore, the discriminator includes an N-layer network structure; wherein the N-th layer network structure includes a convolution layer, a first softmax layer and a second softmax layer in parallel,

[0094] The processor 13 is further configured to input the generated data and the real data into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, respectively input the generated data into the first softmax layer to obtain the first category probability information, input the generated data and the real data into the first convolution layer to obtain the true and false probability information, and input the generated data and the real data into the second softmax layer to obtain the second category probability information; and train the generator and the discriminator using the first category probability information, the second category probability information and the true and false probability information.

[0095] Furthermore, the processor 13 is further configured to determine the discrimination loss using the true and false probability information;

[0096] Using the first category probability information, a first classification loss is determined; and using the discriminant loss and the first classification loss to train the generator and the discriminator.

[0097] Furthermore, the processor 13 is also used to input the discrimination loss and the first classification loss into an objective function to obtain a function value; the objective function is at least composed of a discrimination loss term and a first classification loss term; and the generator and the discriminator are trained by the function value.

[0098] Furthermore, the generator includes at least one of the following network structures: a residual network and a coordinate attention layer.

[0099] Furthermore, the first N layers of the network structure of the discriminator include at least one of the following: a spectral normalization layer and a coordinate attention layer.

[0100] An embodiment of the present application provides a storage medium having a computer program stored thereon. The computer-readable storage medium stores one or more programs. The one or more programs can be executed by one or more processors and applied to an image generating device. The computer program implements the image generating method as described above.

[0101] Based on the above embodiments, an embodiment of the present application provides a computer program product, including a computer program, which can be executed by one or more processors, and the computer program implements the image generation method as described above.

[0102] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0103] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for enabling an image display device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0104] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An image generation method, characterized in that: The method comprises: Input random data and label information into the generator to obtain generated data; Input the generated data and the real data corresponding to the label information into the discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data; The generator and the discriminator are trained using the first category probability information and the true and false probability information; and after the training is completed, realistic image data is generated using the trained generator.

2. The method according to claim 1, characterized in that The discriminator includes an N-layer network structure; wherein the N-th layer network structure includes a parallel convolution layer and a first normalized exponential function softmax layer, and the real data corresponding to the generated data and the label information are input into the discriminator to obtain the first category probability information and the true and false probability information, including: The generated data and the real data are input into the discriminator. After passing through the first N-1 layers of the network structure of the discriminator, the generated data are respectively input into the first softmax layer to obtain the first category probability information, and the generated data and the real data are respectively input into the first convolution layer to obtain the true and false probability information.

3. The method according to claim 1, characterized in that The discriminator includes an N-layer network structure; wherein the N-th layer network structure includes a parallel convolution layer, a first softmax layer, and a second softmax layer. After the random data and label information are input into the generator to obtain the generated data, the method further includes: Input the generated data and the real data into the discriminator, and after passing through the first N-1 layers of the network structure of the discriminator, input the generated data into the first softmax layer to obtain the first category probability information, input the generated data and the real data into the first convolution layer to obtain the true and false probability information, and input the generated data and the real data into the second softmax layer to obtain the second category probability information; The generator and the discriminator are trained using the first category probability information, the second category probability information, and the true and false probability information.

4. The method according to claim 1 or 2, characterized in that: The step of training the generator and the discriminator using the first category probability information and the true and false probability information includes: Determine the discrimination loss using the true and false probability information; Determine a first classification loss using the first classification probability information; The generator and the discriminator are trained using the discriminative loss and the first classification loss.

5. The method according to claim 4, characterized in that The step of training the generator and the discriminator using the discriminant loss and the first classification loss comprises: Inputting the discrimination loss and the first classification loss into an objective function to obtain a function value; the objective function is composed of at least a discrimination loss term and a first classification loss term; The generator and the discriminator are trained by the function value.

6. The method according to claim 1, characterized in that The generator includes at least one of the following network structures: a residual network and a coordinate attention layer.

7. The method according to claim 1, characterized in that The first N layers of the discriminator network structure include at least one of the following: a spectral normalization layer and a coordinate attention layer.

8. An image generating device, characterized in that: The image generating device comprises: A generating unit, used for inputting random data and label information into a generator to obtain generated data; A discriminator, configured to input the generated data and the real data corresponding to the label information into a discriminator to obtain first category probability information and true and false probability information; wherein the first category probability information is obtained by classifying the generated data, and the true and false probability information is obtained by discriminating the generated data and the real data; A training unit is used to train the generator and the discriminator using the first category probability information and the true and false probability information; and after the training is completed, using the trained generator to generate realistic image data.

9. An image generating device, characterized in that: The image generating device comprises: a processor, a memory and a communication bus; the communication bus is used to realize the connection and communication between the processor and the memory; when the processor executes the running program stored in the memory, the method according to any one of claims 1 to 7 is realized.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.

Images