Image generation method and system based on multi-discriminator cooperation

Abstract

This specification provides an image generation method and system based on multi-discriminator collaboration. The method includes: acquiring an initial set of real images; preprocessing the images to obtain a real image dataset; constructing a generator network and a discriminator network; inputting noise into the generator network to obtain a generated image set; feeding the generated image set and real images into each discriminator to obtain the discrimination results and averaging them to obtain a joint loss function; applying gradient normalization to the joint loss function to obtain a normalized loss function; calculating the loss functions of the discriminator and generator networks and updating their parameters to obtain the optimal generator network; and generating images using the optimal generator network. This invention, through the collaboration of multiple discriminators and gradient normalization, alleviates problems such as convergence instability and mode collapse during GAN training, thereby improving the quality and diversity of images generated by GANs.

Description

Technical Field

[0001] This document relates to the field of computer vision technology, and in particular to an image generation method and system based on multi-discriminator collaboration. Background Technology

[0002] With the rapid growth of computer computing power and the development of neural network theories, image generation technology has made great progress. Typical models include autoregressive models such as Pixel Recurrent Neural Networks, streaming models such as Real-valued Non-volume Preserving Transformations, variational autoencoders, and Generative Adversarial Networks (GANs). Compared to traditional unsupervised learning models such as autoencoders and autoregressors, GANs offer advantages such as faster computation speed, higher quality generated samples (especially for high-frequency components of images), and greater flexibility in expansion.

[0003] As an implicit model, GANs do not solve for the expression of the true distribution of data. However, they can generate simulated samples by mimicking the true distribution of data through adversarial learning between the generator network and the discriminator network. This is determined by the special structure of the GAN network. GANs consist of a generator network (G) and a discriminator network (D). G generates realistic simulated images from the input noise vector, while D attempts to distinguish between simulated images and real images. The two networks compete against each other and learn from each other to improve their own network performance. Ultimately, the distribution of the images generated by G increasingly approximates the distribution of real images.

[0004] The performance of GANs is often constrained by problems such as convergence instability and mode collapse. Training instability is often caused by vanishing and exploding gradients. Vanishing gradients prevent Kullback-Leibler divergence or Jensen-Shannon divergence from providing effective gradient information for neural network optimization as a measure of the difference between the generated image distribution and the real image distribution. Mode collapse results in poor diversity of generated images, meaning the model has not fully learned the feature patterns of real images, while gradient explosion causes oscillations in network parameters, making it difficult for the model to converge.

[0005] In view of this, there is an urgent need to provide a solution to the problems of low quality and lack of diversity of generated images in existing GANs methods. Summary of the Invention

[0006] This specification provides one or more embodiments of an image generation method based on multi-discriminator cooperation, including the following steps:

[0007] An initial set of real images is collected, and the initial set of real images is preprocessed to obtain a real image dataset.

[0008] Construct a generator network and a discriminator network. The discriminator network consists of multiple discriminators with different initialization methods and learning rates.

[0009] Sampled Gaussian white noise is input into the generator network to obtain the generated image set;

[0010] Based on the generated image set, an equal number of real images are obtained from the real image dataset, the real images are stitched together with the generated images, and then fed into each discriminator to obtain each discrimination result;

[0011] The joint loss function is obtained by averaging the results of each discrimination. Gradient constraint is applied to the joint loss function using gradient normalization to obtain the normalized loss function. The discriminator loss function and generator loss function are calculated using the normalized loss function, and the parameters of the discriminator network and generator network are updated until the optimal generator network is obtained. Finally, the image is generated using the optimal generator network.

[0012] This specification provides one or more embodiments of an image generation system based on multi-discriminator cooperation, including

[0013] Image acquisition and processing unit: used to acquire an initial set of real images, and to preprocess the initial set of real images to obtain a real image dataset;

[0014] Image generation unit: Used to generate images through a trained generator network;

[0015] The image generation unit includes

[0016] Generator Network Construction Module: Used to construct generator networks;

[0017] Discriminator Network Construction Module: Used to construct the discriminator network, which consists of multiple discriminators with different initialization methods and learning rates;

[0018] Training module: Sample Gaussian white noise is input into the generator network to obtain a generated image set; based on the generated image set, an equal number of real images are obtained from the real image dataset of the image acquisition and processing unit, the real images are stitched together with the generated images and then fed into each discriminator to obtain each discrimination result;

[0019] The joint loss function is obtained by averaging the results of each discrimination. The gradient normalization module is used to apply gradient constraints to the joint loss function to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated using the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained.

[0020] The method provided by this invention inputs real and generated images into multiple discriminators with different initialization methods and learning rates, allowing the generator to compete against these discriminators. This makes the generator's learning of image feature patterns more comprehensive, avoiding the mode collapse problem during GAN training. Multiple discriminators make decisions on the input images and provide a joint loss function. By performing gradient normalization on the joint loss function, the integrity of gradient information during backpropagation is ensured while avoiding gradient explosion during training. Furthermore, this method can be used in conjunction with Wasserstein loss to avoid the gradient vanishing problem. Through the collaboration of multiple discriminators and gradient normalization, problems such as convergence instability and mode collapse during GAN training are mitigated, improving the quality and diversity of GAN-generated images. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating an image generation method based on multi-discriminator collaboration provided for one or more embodiments of this specification;

[0023] Figure 2 A schematic diagram of the flow framework of the image generation method based on multi-discriminator cooperation provided in one or more embodiments of this specification;

[0024] Figure 3 Figure (a) shows the generator network structure diagram in the image generation method based on multi-discriminator cooperation provided in one or more embodiments of this specification, and Figure (b) shows the structure diagram of the upsampling residual network module.

[0025] Figure 4Figure (a) shows the discriminator network structure diagram in the image generation method based on multi-discriminator cooperation provided in one or more embodiments of this specification; Figure (b) shows the structure diagram of the optimized residual network module; and Figure (c) shows the structure diagram of the residual network module.

[0026] Figure 5 A network block diagram of the gradient normalization module in an image generation method based on multi-discriminator cooperation provided in one or more embodiments of this specification;

[0027] Figure 6 Example images of real and generated images from experimental cases of the image generation method based on multi-discriminator cooperation provided in one or more embodiments of this specification;

[0028] Figure 7 A comparison of the IS scores of images generated using the gradient normalization module and without it in experimental cases of the multi-discriminator collaborative image generation method provided in one or more embodiments of this specification.

[0029] Figure 8 A comparison of the training results of FID scores of images generated using the gradient normalization module and without it in experimental cases of the multi-discriminator collaborative image generation method provided in one or more embodiments of this specification.

[0030] Figure 9 A schematic diagram of an image generation system framework based on multi-discriminator collaboration provided for one or more embodiments of this specification;

[0031] Figure 10 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0033] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0034] Method Implementation Examples

[0035] According to an embodiment of the present invention, an image generation method based on multi-discriminator cooperation is provided. Figures 1-2 show a flowchart and a schematic diagram of the process framework of the image generation method based on multi-discriminator cooperation provided in this embodiment, respectively. The image generation method based on multi-discriminator cooperation according to an embodiment of the present invention includes the following steps:

[0036] S101. Acquire an initial set of real images, and preprocess the initial set of real images to obtain a real image dataset.

[0037] S102. Construct a generator network and a discriminator network. The discriminator network consists of multiple discriminators with different initialization methods and learning rates.

[0038] S103. Sample Gaussian white noise and input it into the generator network to obtain the generated image set;

[0039] S104. Based on the generated image set, obtain an equal number of real images from the real image dataset in step S101, stitch the real images together with the generated images and send them into each discriminator to obtain each discrimination result.

[0040] S105. The joint loss function is obtained by averaging the results of each discrimination. The gradient constraint of the joint loss function is applied by the gradient normalization method to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated by the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained. Finally, the image is generated by the optimal generator network.

[0041] This embodiment's method inputs real and generated images into multiple discriminators with different initialization methods (such as all-zero initialization, random initialization, or Kaiming initialization) and learning rates. This allows the generator to compete against multiple discriminators, making the generator's learning of image feature patterns more comprehensive and avoiding the mode collapse problem during GAN training. Multiple discriminators make decisions on the input images and provide a joint loss function. By performing gradient normalization on the joint loss function, the integrity of gradient information during backpropagation is ensured while avoiding gradient explosion during training. Furthermore, this embodiment's method can be used in conjunction with Wasserstein loss to avoid the gradient vanishing problem. Through the collaboration of multiple discriminators and gradient normalization methods, problems such as convergence instability and mode collapse during GAN training are mitigated, improving the quality and diversity of GAN-generated images.

[0042] In this embodiment, the preprocessing of the initial set of real images specifically involves performing random horizontal flipping and regularization on the images in the initial set, and setting the following parameters required for model training: the probability p of horizontal flipping during preprocessing. H and the maximum value of horizontal and vertical translation. trans Batch size M, total number of iterations (total_epoch), learning rate α of the generator network. G The learning rates α1, α2, α3 and their decay rates decay of the three discriminator networks;

[0043] In this embodiment, as Figure 3 As shown in (a), the generator network constructed in step S102 consists of a linear layer, three identical upsampled residual network modules, a batch normalization layer, a ReLU activation layer, a convolutional layer, and a Tanh activation layer; the structure of the upsampled residual network module is as follows: Figure 3 As shown in (b); the input of the generator network is a random vector obtained by sampling Gaussian white noise, and the output is a simulated image. The generated simulated image and the real image are concatenated into a tensor type variable and then fed into each discriminator. The generator will calculate the loss function and update the parameters according to the discrimination results given by the discriminator.

[0044] In this embodiment, as Figure 4 As shown, the discriminator network in step S102 consists of the following structures: a single-layer optimized residual network module that better preserves background information; three identical conventional residual network modules that establish an identity mapping between input and output; a ReLU function activation layer; a single-layer binary adaptive mean convergence layer; and a single linear layer. The structure of the optimized residual network module and the residual network module is as follows: Figure 4 (b) and Figure 4As shown in (c), the discriminator network takes a tensor-type image as input and outputs a judgment result indicating whether the input image is a real image. The judgment result, after being normalized by the gradient normalization module, is used to calculate the loss function of each discriminator and generator (multiple discriminators share the same loss function), thereby performing gradient backpropagation and sequentially updating the network parameters of multiple discriminators and the generator. Different initialization methods and learning rates of the discriminator network will affect the convergence speed and even the final result. During the training of GANs, the generator network parameters are updated through the backpropagation gradient of the discriminator. However, a single discriminator inevitably has some unlearned feature patterns during training, which will lead to a decrease in the diversity and quality of the generated images. To mitigate the impact of this problem, multiple different discriminators can be used to cover the sample space to a greater extent. This allows the network to learn richer feature patterns, and the joint loss function obtained by multiple discriminators working together has a smaller variance than that of a single discriminator, which helps to stabilize the training process.

[0045] Suppose the network's decision result f(x) and the actual result f ac The error of f(x) is Δd = f(x) - f ac If (x), then the experimental error follows a Gaussian distribution. Therefore, the joint loss function obtained by averaging the results of multiple discriminators is:

[0046]

[0047] That is, the loss function of a multi-discriminator has a smaller variance, as shown in the following equation:

[0048]

[0049] When constructing different discriminator networks, different initialization methods and learning rates can be used to ensure the diversity between the discriminator networks and to ensure that the generator can learn the feature patterns of real images more comprehensively. Although the multi-discriminator architecture allows any number of discriminators to work simultaneously, in actual training, too many discriminators can lead to homogenization among them, while too few discriminators may cause coupling between the generator and a single discriminator, increasing the network load without achieving the expected goal. Through training networks with different numbers of discriminators, the results show that three discriminators can provide a stable distribution with a certain variance. Therefore, this network uses three discriminators to form the discriminator network.

[0050] In this embodiment, in step S105, the average value of each discrimination result is calculated to obtain the joint loss function. The gradient normalization method is used to apply gradient constraints to the joint loss function to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated using the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained. The specific implementation is as follows.

[0051] In this embodiment, the gradient space of the discriminator in the original GANs is too steep, which not only leads to training instability but may also cause mode collapse in the generator. Existing techniques express the discriminator as a continuous Lipschitz function under a Lipschitz constant L, such as Spectral Normalization. Generally, it is undesirable for network models to be sensitive to input perturbations; robustness is required. Robustness is reflected in two aspects: first, the model is insensitive to perturbations of its parameters; second, the model is insensitive to input perturbations. If a model gives two different judgments for images differing by only one pixel, it is too sensitive to input perturbations. If the model's input is constrained by a constant L, as shown in the following equation:

[0052] ||f(x1)-f(x2)||≤L·||x1-x2||

[0053] Where ||x1-x2|| represents the norm of the difference between input x1 and input x2, and ||f(x1)-f(x2)|| represents the norm of the difference between output f(x1) and output f(x2). In the above formula, the output of the model is constrained by the input, and the model is said to satisfy the Lipschitz constraint, where the constant L is called the Lipschitz constant.

[0054] Compared to traditional neural networks, spectral normalization replaces the normalization layer with a spectral normalization layer, which divides the discriminator's weight matrix by its maximum singular value, ensuring that the Lipschitz constraint is satisfied for all input weight matrices. During backpropagation, the changes in network weights during each update are within a certain range, making them less susceptible to extreme data and smoothing out the entire training process.

[0055] However, spectral normalization applies Lipschitz constraints between layers, making it a module-level method. As the number of network layers increases, the Lipschitz constant decreases rapidly, as proven below:

[0056] Let the output of the k-th layer of the network be f. k (x), the nonlinear activation function is the ReLU function, and the parameter matrix is ​​W. k The bias is b kIf the Lipschitz constant between layers is L, then we have

[0057] ||f k (x1)-f k (x2)||=||relu(W k ·f k-1 (x1)+b k )-relu(W k ·f k-1 (x2)+b k )|| ≤L relu ||(W k ·f k-1 (x1)+b k )-(W k ·f k-1 (x2)+b k )|| ≤L relu L k ||f k-1 (x1)-f k-1 (x2)||=||f k-1 (x1)-f k-1 (x2)||

[0058] Right now:

[0059]

[0060] As shown in the above equation, the Lipschitz constraint applied between layers by the spectral normalization method decays multiplicatively with the increase of the number of layers, i.e.:

[0061] L k <L k-1 <L<L1;

[0062] Therefore, when using spectral normalization, network capacity is often sacrificed in order to achieve Lipschitz constraints. For discriminator networks, smaller network capacity means that they are more easily fooled by the generator, resulting in poor training results.

[0063] Therefore, in order to circumvent and solve the above problems, this embodiment uses a gradient normalization method to constrain the gradient of the joint loss function obtained by calculating the average value, as follows: Figure 5 The diagram shown is a network block diagram of the gradient normalization module in this embodiment. Let the original joint loss function be f(x), and the normalized loss function be... As shown in the following formula:

[0064]

[0065] Normalized loss function The gradient norm is bounded, as proven below:

[0066]

[0067] Under the constraints of gradient normalization methods, The gradient norm ranges from [-1, 1], avoiding the gradient explosion problem and stabilizing the training process. It should be noted that although averaging the results of the three discriminators can also provide optimization direction for the generator, this result has not undergone gradient normalization, resulting in large gradient fluctuations and potential training instability. The module's output is the normalized discriminator result, with its gradient controlled within [-1, 1], thus stabilizing the training process. This module is also easy to integrate into the network; simply normalizing the discriminator's result ensures that the backpropagated gradient range is [-1, 1].

[0068] Since the output range after gradient normalization is [-1,1], the Wasserstein loss of the discriminator network and the generator network degenerates into hinge loss. Unlike the spectral normalization method, the gradient normalization module can be well combined with the Wasserstein distance (also known as the Earth-Mover distance, EM distance), thereby avoiding the gradient vanishing problem during training.

[0069] The joint loss function f(x) provided by multiple discriminators is normalized to obtain the normalized loss function. Based on this result, the Wasserstein loss functions of the generator network and the discriminator network can be calculated, and the loss functions of the discriminator network D and the generator network G can be determined as follows:

[0070]

[0071]

[0072] Where M is the batch size, x is the real image sample, z is the noise vector that follows a multivariate Gaussian distribution, and G(z) i () represents the noise vector z received as input to the generator. i The simulated image generated later.

[0073] Loss function L D and L G The network parameters of the discriminator and generator will be updated separately through backpropagation.

[0074] In this embodiment, the generator network is trained using a cross-training method. When updating the parameters of the generator network G, the parameters of all discriminators D are frozen, and when updating the parameters of D, the parameters of G are also frozen. The generator parameters are updated only once after the network parameters of each discriminator have been updated five times, until the learning rate of the generator decays to 0 under the control of the Adam optimizer, the training process stops, and the trained and optimal generator network is obtained.

[0075] In a preferred embodiment, the method further includes testing the trained generator network, including the following steps:

[0076] When testing the trained generator network, the network is first loaded. Then, Gaussian white noise is sampled to obtain a noise vector, which is fed into the generator to produce the generated image. The distribution distance between the generated image and the real image is measured using the `get_inception_score_and_fid` function from the `pytorch_gan_metrics` library, resulting in the Inception Score (IS) and Fréchet InceptionDistance (FID) scores. Among the evaluation metrics used for the generated image, IS is an indicator of the quality of the generated image; a higher IS indicates higher quality. FID is an indicator of the diversity of the generated image; a lower FID indicates that the distribution of the generated image is closer to the real distribution, indicating better quality and diversity.

[0077] Compared to existing technologies, the method provided in this embodiment uses gradient normalization to constrain the output of the discriminator function. This method can also be combined with Wasserstein loss to avoid training instability caused by gradient explosion and vanishing gradients during GAN training. By having multiple discriminators with different initializations and learning rates collaborate, a joint loss function is obtained, allowing the generator to learn a wider range of real image features, thus avoiding mode collapse caused by a single discriminator being deceived by some generated images. Furthermore, since this method normalizes and constrains the joint loss function of multiple discriminators' outputs, unlike spectral normalization which imposes constraints on each layer of the network, it is a model-level method, and the network performance does not rapidly decline with increasing network depth. This method has achieved excellent experimental results, as detailed below.

[0078] The advantages of this embodiment are illustrated below through specific experimental examples.

[0079] Training phase:

[0080] This experiment obtained the CIFAR-10 dataset from the official website (http: / / www.cs.toronto.edu / ~kriz / cifar-10-python.tar.gz). The CIFAR-10 dataset contains 60,000 images of size 32×32×3, which are divided into 50,000 training instances and 10,000 test instances, for a total of 10 classes. In the image preprocessing stage, the images are first randomly horizontally flipped with a probability of 0.5 and horizontally and vertically translated with a maximum value of 0.2 to increase the diversity of the sample data. Finally, the images are standardized to accelerate the convergence speed of the model.

[0081] In this experiment, the batch size was set to 64, and the learning rate α of the generator G was set to... G Set to 2×10 -4 The learning rate α of discriminators D1 and D3 D1 and α D3 Set to 4×10 -4 The learning rate α of discriminator D2 D2 Set to 8×10 -4 The total number of iterations was set to 200k, with the Adam optimizer used as α. G α D1 α D2 and α D3 The optimizer ensures that all learning rates decay to 0 after 200k iterations;

[0082] Based on the constructed generator network, discriminator network, and gradient normalization module, a 64×128×128 tensor is obtained by sampling Gaussian white noise 64 times at a size of 128×128. The resulting noise vector is input into the generator network to obtain a 64×3×32×32 tensor, which is 64 simulated images with 3 channels and a size of 32×32. 64 real images are sampled from the dataset and combined into a 64×3×32×32 tensor. This tensor is then merged with the simulated images to form a 128×3×32×32 tensor, which is then input into each discriminator. The entire network structure is trained using the above method. The network parameters of each discriminator are updated five times before the generator parameters are updated once. After the generator parameters are updated 200k times, the learning rate decays to 0 under the control of the Adam optimizer, and the training process stops, resulting in a trained generator network.

[0083] Testing phase:

[0084] When testing the trained generator network, the first step is to load the trained generator network. Then, a noise vector is obtained by sampling Gaussian white noise at 128×128 resolution 50k times. This noise vector is then fed into the generator to produce 50k generated images. The IS and FID scores of the generated images are calculated using the `get_inception_score_and_fid` function from the `pytorch_gan_metrics` library. Example comparisons of some generated images with real images are provided below. Figure 6 As shown, the images generated by the network are quite realistic, proving that the network has learned the feature patterns of real images well. In order to quantitatively study the quality and diversity of generated images, two evaluation metrics, Inception Score and Fréchet Inception Distance, are introduced. Using GANs without the normalization module as the baseline, networks with the normalization module and the baseline network are trained respectively. After every 5k iterations of training, the generator network generates 50k images, and the IS and FID scores of the network are calculated based on these 50k images.

[0085] Here, IS is an indicator of the generated image quality; a higher IS score indicates higher image quality. The convergence process of the IS score with the number of epochs in the experiment is as follows: Figure 7 As shown, the IS score of the network using the normalization module is improved by 30.3% compared to the baseline, and the baseline network experienced convergence failure in the later stages of training. FID is an indicator of generated image diversity; the smaller the FID, the closer the distribution of generated images is to the true distribution. The convergence process of the FID score with the number of epochs in the experiment is shown below. Figure 8 As shown, the FID score of the network using the normalization module is improved by 69.4% compared to the baseline. The baseline network also experienced the problem of FID score convergence failure in the later stages of training, while the training process of the network using the normalization module remained stable. It can be seen that the gradient normalization module has a good effect on the training of stable GANs. Finally, the network using the gradient normalization method module has an IS score of 8.549 and an FID score of 11.136.

[0086] System Implementation Examples

[0087] According to an embodiment of the present invention, an image generation system based on multi-discriminator cooperation is provided. As shown in FIG9, which is a schematic block diagram of the image generation system based on multi-discriminator cooperation provided by the present invention, the image generation system based on multi-discriminator cooperation according to an embodiment of the present invention includes:

[0088] Image acquisition and processing unit: used to acquire an initial set of real images, and to preprocess the initial set of real images to obtain a real image dataset;

[0089] Image generation unit: Used to generate images through a trained generator network;

[0090] The image generation unit includes

[0091] Generator Network Construction Module: Used to construct generator networks;

[0092] Discriminator Network Construction Module: Used to construct the discriminator network, which consists of multiple discriminators with different initialization methods and learning rates;

[0093] Training module: Sample Gaussian white noise is input into the generator network to obtain a generated image set; based on the generated image set, an equal number of real images are obtained from the real image dataset of the image acquisition and processing unit, the real images are stitched together with the generated images and then fed into each discriminator to obtain each discrimination result;

[0094] The joint loss function is obtained by averaging the results of each discrimination. The gradient normalization module is used to apply gradient constraints to the joint loss function to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated using the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained.

[0095] In this embodiment, the generator network consists of a linear layer, three identical upsampling residual network modules, a batch normalization layer, a ReLU activation layer, a convolutional layer, and a Tanh activation layer.

[0096] The discriminator network consists of the following layers: an optimized residual network module that better preserves background information, three identical residual network modules that establish identity mapping between input and output, a ReLU function activation layer, a binary adaptive mean convergence layer, and a linear layer.

[0097] In this embodiment, the normalized loss function of the discriminator network As shown in the following formula:

[0098]

[0099] Loss function of discriminator network The gradient norm is bounded, as proven below:

[0100]

[0101] Under the constraints of gradient normalization methods, The gradient norm ranges from [-1, 1].

[0102] The loss functions for the generator network and the discriminator network are as follows:

[0103]

[0104]

[0105] Among them, L G Let L be the loss function of the generator network. D The loss function of the discriminator network; M is the batch size, x is the real image sample, z is the noise vector that follows a multivariate Gaussian distribution, G(z) i ) represents the noise vector z received as input to the generator. i The generated image is generated later.

[0106] The embodiments of the present invention are method embodiments corresponding to the above system embodiments. The specific operations of each processing step can be understood by referring to the description of the method embodiments, and will not be repeated here.

[0107] like Figure 10 As shown, the present invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the image generation method based on multi-discriminator cooperation in the above embodiments, or when the computer program is executed by a processor, it implements the image generation method based on multi-discriminator cooperation in the above embodiments. When the computer program is executed by the processor, it implements the following method steps:

[0108] S101. Acquire an initial set of real images, and preprocess the initial set of real images to obtain a real image dataset.

[0109] S102. Construct a generator network and a discriminator network. The discriminator network consists of multiple discriminators with different initialization methods and learning rates.

[0110] S103. Sample Gaussian white noise and input it into the generator network to obtain the generated image set;

[0111] S104. Based on the generated image set, obtain an equal number of real images from the real image dataset in step S101, stitch the real images together with the generated images and send them into each discriminator to obtain each discrimination result.

[0112] S105. The joint loss function is obtained by averaging the results of each discrimination. The gradient constraint of the joint loss function is applied by the gradient normalization method to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated by the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained. Finally, the image is generated by the optimal generator network.

[0113] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0114] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image generation method based on multi-discriminator cooperation, characterized in that, Including the following steps: An initial set of real images is collected, and the initial set of real images is preprocessed to obtain a real image dataset. Construct a generator network and a discriminator network. The discriminator network consists of multiple discriminators with different initialization methods and learning rates. Sampled Gaussian white noise is input into the generator network to obtain the generated image set; Based on the generated image set, an equal number of real images are obtained from the real image dataset, the real images are stitched together with the generated images, and then fed into each discriminator to obtain each discrimination result; The joint loss function is obtained by averaging the results of each discrimination. The gradient constraint of the joint loss function is then applied using the gradient normalization method to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated using the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained. Finally, the image is generated using the optimal generator network. The normalized loss function of the discriminator network As shown in the following formula: Loss function of discriminator network The gradient norm is bounded, as proven below: Under the constraints of gradient normalization methods, The gradient norm ranges from [-1, 1].

2. The image generation method based on multi-discriminator cooperation as described in claim 1, characterized in that, The joint loss function is obtained by averaging the discrimination results of multiple discriminators. The gradient of the joint loss function is constrained by the gradient normalization method to obtain the normalized loss function. Based on this loss function, the Wasserstein loss functions of the generator network and the discriminator network are calculated and determined respectively. The two loss functions are used to update the network parameters of the discriminator and the generator respectively through backpropagation.

3. The image generation method based on multi-discriminator cooperation as described in claim 1, characterized in that, The generator network and discriminator network are trained using a cross-training method. When updating the generator parameters, the parameters of all discriminators are frozen, or when updating the discriminator parameters, the parameters of the generator are frozen. The generator parameters are updated once after each discriminator's network parameters have been updated five times.

4. The image generation method based on multi-discriminator cooperation as described in claim 1, characterized in that, The loss functions of the generator network and the discriminator network are as follows: Among them, L G Let L be the loss function of the generator network. D The loss function of the discriminator network; M is the batch size, x is the real image sample, z is the noise vector that follows a multivariate Gaussian distribution, G(z) i ) represents the noise vector z received as input to the generator. i The generated image is generated later.

5. An image generation system based on multi-discriminator cooperation, characterized in that, include Image acquisition and processing unit: used to acquire an initial set of real images, and to preprocess the initial set of real images to obtain a real image dataset; Image generation unit: Used to generate images through a trained generator network; The image generation unit includes Generator Network Construction Module: Used to construct generator networks; Discriminator Network Construction Module: Used to construct the discriminator network, which consists of multiple discriminators with different initialization methods and learning rates; Training module: Samples Gaussian white noise as input to the generator network to obtain the generated image set; Based on the generated image set, an equal number of real images are obtained from the real image dataset of the image acquisition and processing unit. The real images are then stitched together with the generated images and sent to each discriminator to obtain the discrimination results. The joint loss function is obtained by averaging the results of each discrimination. The gradient constraint of the joint loss function is applied through the gradient normalization module to obtain the normalized loss function. The discriminator loss function and the generator loss function are calculated through the normalized loss function, and the parameters of the discriminator network and the generator network are updated until the optimal generator network is obtained. The normalized loss function of the discriminator network As shown in the following formula: Loss function of discriminator network The gradient norm is bounded, as proven below: Under the constraints of gradient normalization methods, The gradient norm ranges from [-1, 1].

6. The image generation system based on multi-discriminator cooperation as described in claim 5, characterized in that, The generator network consists of a linear layer, three identical upsampling residual network modules, a batch normalization layer, a ReLU activation layer, a convolutional layer, and a Tanh activation layer.

7. The image generation system based on multi-discriminator cooperation as described in claim 5, characterized in that, The discriminator network consists of a discriminator layer consisting of an optimized residual network module that better preserves background information, three identical residual network modules that establish identity mappings between input and output, a ReLU function activation layer, a binary adaptive mean convergence layer, and a linear layer.

8. The image generation system based on multi-discriminator cooperation as described in claim 5, characterized in that, The loss functions of the generator network and the discriminator network are as follows: Among them, L G Let L be the loss function of the generator network. D The loss function of the discriminator network; M is the batch size, x is the real image sample, z is the noise vector that follows a multivariate Gaussian distribution, G(z) i () represents the noise vector z received as input to the generator. i The generated image is generated later.

Images