Image generation method based on regression loss and gradient constraint
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 魏珅
- Filing Date
- 2026-03-09
- Publication Date
- 2026-06-09
Smart Images
Figure CN122174915A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image generation, and more particularly to an image generation method based on regression loss and gradient constraints. Background Technology
[0002] Generative Adversarial Networks (GANs) are a classic generative model consisting of a generator and a discriminator. These two components alternately optimize through an adversarial minimax game process, and GANs have been widely used in image generation. However, traditional GAN frameworks suffer from significant training limitations, resulting in restricted generative performance. The generator's parameter updates depend solely on the discriminator's judgment scores of the generated samples. This feedback is an indirect and unstable weak supervision signal, lacking clear target guidance. It is prone to continuous oscillations during training and is difficult to converge efficiently to the target data distribution. The gradient space of the discriminator is often too sharp, which makes the generator prone to mode collapse during training and limits the overall performance of the model. Existing methods for improving discriminators have significant shortcomings. Spectral normalization applies Lipschitz conditions by imposing constraints layer by layer, which can easily weaken the discriminator's fitting ability due to excessively strong constraints. Gradient penalty is a soft constraint, and its upper bound is not fixed, making it difficult to strictly guarantee the Lipschitz conditions during training and failing to ensure gradient stability on unseen data.
[0003] To address the aforementioned issues, a new image generation method is urgently needed that optimizes the training process of both the generator and the discriminator, thereby improving the overall training stability and image generation performance of GANs. Summary of the Invention
[0004] Purpose of the invention: To overcome the shortcomings of traditional generative adversarial networks in image generation, such as poor training stability, weak generator supervision signals, and sharp gradient space of discriminators that easily lead to mode collapse, and to provide an image generation method based on regression loss and gradient constraints.
[0005] Technical Solution: To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, it is an image generation method based on regression loss and gradient constraints, applied to a generator... and discriminator Generative adversarial networks composed of, The parameters to be optimized for the generator The core of this method, which uses parameters to be optimized for the discriminator, is to construct an image generation algorithm (RLGC) with regression loss and gradient constraints. This includes regression reconstruction of the generator loss and hard gradient constraints on the discriminator. The specific steps are as follows: Step S1: Initialize training parameters and model parameters Set the total number of training rounds. The number of discriminator iterations corresponding to each generator iteration The number of samples required to calculate the regression loss of the generator. and batch size Initialize generator parameters Discriminator parameters This prepares for subsequent iterative training.
[0006] Step S2: Iterative training of the discriminator and hard gradient constraints Perform discriminant analysis The training is iterated through rounds, and the specific operations for each round are as follows: S2.1 From the prior distribution The sampling batch is noise samples From the distribution of real data The sampling batch is Real image samples ; S2.2 Input the noise sample into the generator to obtain the generated image. ; S2.3 For each real image and generating images The original output of the discriminator is normalized using a hard gradient constraint module to obtain the normalized discriminator output. The calculation formula is as follows: ; ; in, and The gradient norm of the discriminator's original output with respect to the input. and This is a general term used to avoid situations where the original output of the discriminator approaches infinity or the gradient norm approaches 0. S2.4 Calculate the batch discriminator loss based on the normalized discriminator output. The formula is: S2.5 The Adam optimization algorithm is used to optimize the discriminator loss. Optimize and update the discriminator parameters. .
[0007] Step S3: Iterative training of the generator and optimization of the regression loss Discriminator complete After one round of iterations, the generator is trained once, as follows: S3.1 From the distribution of real data The sampling batch is Real image samples From the prior distribution Medium sampling noise sample ; S3.2 will A noise sample is input into the generator to obtain the generated image. The expected value of the discriminator's output under the condition of the generated image is calculated, using the following formula: S3.3 Based on the discriminator output of the real image and the aforementioned expected value, construct and calculate the batch generator regression loss. The formula is: Its overall optimization objective is: S3.4 The Adam optimization algorithm is used to optimize the generator regression loss. To optimize, update the generator parameters according to the following rules. : in, For learning rate, The difference in the discriminator's expected output when real data and generated data are used as inputs. This is the current training round.
[0008] Step S4: Complete the overall training and achieve image generation. Repeat steps S2 and S3 until completion. After a total of rounds of training, the optimal generator parameters are obtained. and optimal discriminator parameters Input random noise into the optimal generator This allows for the generation of high-quality images.
[0009] Furthermore, in this invention, the discriminator employs a piecewise linear activation function, making the Hessian matrix of the discriminator... Given a zero matrix, the simplified derivation of the norm of the output gradient after gradient normalization yields: Strict Lipschitz constraints are implemented on the discriminator, limiting the Lipschitz constant to less than 1.
[0010] Furthermore, the overall loss optimization objective of the discriminator in this invention is: .
[0011] Beneficial effects: This invention reconstructs the traditional adversarial loss of the generator into a regression loss, transforms the generator training objective into a least-squares optimization problem, provides the generator with stronger and clearer supervision signals, avoids training oscillations caused by traditional weak supervision signals, and enables the generator to converge reliably even when the discriminator performance is weak, and efficiently learns the target data distribution. This invention introduces a hard gradient constraint module to the discriminator, imposing strict Lipschitz constraints at the model level to limit the gradient norm to within 1. Compared to the layer-by-layer constraints of spectral normalization, it does not limit the expressive power of the neural network. Compared to the soft constraints of gradient penalty, it can ensure gradient stability on unseen data, effectively smooth the gradient space of the discriminator, and avoid mode collapse during generator training. The RLGC algorithm of this invention only optimizes the generator loss and discriminator output, without introducing additional hyperparameters or sampling operations. The training process is simple and easy to implement. While improving training stability, it significantly improves the image generation performance of the model on the target data distribution. The generator parameter update gradient not only includes the discriminator gradient but also incorporates the difference between the discriminator's expected output and the real data's output. This provides dual supervision for generator training. As the generator parameters approach their optimal values, this difference tends to zero, further improving the stability of generator training. Attached Figure Description
[0012] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0013] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0014] Reference Figure 1 This paper presents an image generation method based on regression loss and gradient constraints, applied to a generative adversarial network (GAN). The generator of this network employs a deep convolutional neural network (CNN) structure, responsible for mapping random noise to generated images. The discriminator also uses a deep CNN structure, with piecewise linear activation functions (such as ReLU) selected to distinguish between real and generated images. The discriminator's Hessian matrix... It is a zero matrix, which facilitates the simplification of the gradient norm.
[0015] In this embodiment, the total number of training rounds is set. The number of discriminator iterations corresponding to each generator iteration The number of samples required to calculate the generator regression loss Batch size Learning rate The Adam optimization algorithm is used, with parameters... , .
[0016] The specific training process is as follows: S1. Initialize the parameters of the convolutional and fully connected layers of the generator and discriminator using a normal distribution initialization method; S2, Beginning wheel( The overall training first involves five rounds of iterative training on the discriminator: S2.1 In each round of discriminator training, the prior distribution is obtained from the standard normal distribution. Sample 64 noise samples From the real CIFAR-10 dataset Sample 64 real images ; S2.2 Input noise samples into the generator to obtain the generated image. ; S2.3 For each and Calculate the original output of the discriminator , The gradient norm of the input is used to obtain the normalized output through a hard gradient constraint module. and ; S2.4 Calculate the batch discriminator loss The discriminator parameters are updated using the Adam optimization algorithm. ; S3. After the discriminator completes 5 rounds of iteration, the generator is trained in 1 round of iteration: S3.1 Sample 64 real images from the CIFAR-10 dataset 128 noise samples were sampled from a standard normal prior distribution. ; S3.2 Calculate the expected output value of the discriminator corresponding to the 128 generated images. ; S3.3 Calculate the batch generator regression loss Update the generator parameters according to the parameter update rules. ; S4. Repeat the above training process for the discriminator and generator. After completing 100 rounds of training, obtain the optimal generator parameters. and optimal discriminator parameters ; S5. Take random standard normal noise and input it into the optimal generator. Generate high-quality images that are consistent with the distribution of the CIFAR-10 dataset.
[0017] In this embodiment, a strong supervision signal is provided to the generator through regression loss, and the generator training process is free from obvious oscillations, thus improving the convergence speed. Through the hard gradient constraint module, the gradient norm of the discriminator is strictly limited to within 1, the gradient space is smooth, the generator training does not exhibit mode collapse, and the clarity and diversity of the generated images are significantly improved compared to traditional GANs.
[0018] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. An image generation method based on regression loss and gradient constraints, characterized in that, Applied to generative adversarial networks, the generative adversarial network including a generator and discriminator , For generator parameters, The discriminator parameters include the following steps: S1. Initialize generator parameters Discriminator parameters Set the total number of training rounds The number of discriminator iterations corresponding to each generator iteration The number of samples required to calculate the generator regression loss and batch size ; S2. Iteratively train the discriminator: In each round of discriminator training, the discriminator is trained from the prior distribution. Sampling batches are noise From the distribution of real data Sampling batches are Real images For each real image and the image generated by the generator The original output of the discriminator is normalized by using a hard gradient constraint module to obtain the normalized discriminator output. The discriminator loss is calculated based on the normalized output. The discriminator parameters are updated using the Adam optimization algorithm. ; S3. Iteratively train the generator: the discriminator completes... After rounds of iteration, from the distribution of real data Sampling batches are Real images From the prior distribution sampling noise ; Calculate the expected value of the discriminator output under the generated image conditions. A regression loss is constructed based on this expected value and the discriminator output of the real image. The calculation formula is: In the hard gradient constraint module, the new output of the discriminator after gradient normalization The calculation formula is: And the normalized output satisfies Lipschitz constraint.
2. The image generation method based on regression loss and gradient constraints according to claim 1, characterized in that, The generator parameters The update rules are as follows: in, For learning rate, For real data and generating data The difference between the discriminator's expected output when each input is used as an input. This is the current training round.
3. The image generation method based on regression loss and gradient constraints according to claim 1, characterized in that, The discriminator loss The calculation formula is: In actual iterative calculations, the formula for calculating the batch discriminator loss is:
4. The image generation method based on regression loss and gradient constraints according to claim 1, characterized in that, The expected value output by the discriminator under the generated image condition The batch calculation formula is as follows:
5. The image generation method based on regression loss and gradient constraints according to claim 1, characterized in that, The regression loss The batch calculation formula is as follows: .
6. The image generation method based on regression loss and gradient constraints according to claim 1, characterized in that, The discriminator employs a piecewise linear activation function, such that the discriminator's Hessian matrix... For a zero matrix, After simplification, the derivation yields: 。