Optimization method of diffusion model, image generation method, electronic device, and medium

Abstract

The application discloses an optimization method of a diffusion model, which firstly randomly samples steps from Markov chain paths of a diffusion process in a mini batch to obtain a step t in a current batch, then randomly adds noise to each pixel point in an image of the step t to obtain a noise image of the step t, and predicts an original image through a model based on the noise image to obtain noise predicted by the model, then calculates mean square loss between the noise predicted by the model and real noise, finally samples an additional random label from a label set following a preset distribution, and re-predicts the noise of the noise image based on the random label, and calculates distribution adjustment loss. The method adjusts conditional transition probability in the sampling process, implicitly forces the generated image to approximate the target prior distribution in each sampling step, and fills the research gap in the direction of training a more robust generation model based on long-tail distribution data.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to an optimization method for a diffusion model, an image generation method based on the diffusion model, an electronic device, and a medium. Background Technology

[0002] In the field of computer vision, common deep generative models can be divided into four categories: Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Flow-Based Models (FBMs), and Diffusion Models (DMs). These four generative models have significantly different training and sampling mechanisms. VAEs consist of an encoder and a decoder. The encoder first projects high-dimensional data into a low-dimensional space using hidden layers, while the decoder reconstructs the high-dimensional data from the low-dimensional features. GANs are currently the most mainstream model in the generative field. They consist of a generator and a discriminator. The generator produces real samples, and the discriminator distinguishes between real and generated samples. The training mechanism of GANs requires alternating updates to the generator and discriminator. Flow-based models directly learn the probabilities of the data. Their primary mathematical tool is a powerful density estimation statistical tool: Normalized Flow (NF). NF transforms simple distributions into complex distributions by applying a series of invertible transformation functions. Based on this mathematical tool, the model can directly update its parameters using the negative log-likelihood function. Diffusion models are generative models based on maximum likelihood. The Denoising Diffusion Probability Model (DDPM) is its most typical and widely used representative. Diffusion models have become the best result for many tasks in the field of generation and have been applied in many fields, including text-image generation, image editing, speech synthesis, medical imaging, video generation, and adversarial learning.

[0003] Existing generative models are trained based on the assumption of a uniformly distributed data label. However, in the real world, data distribution is often highly skewed. This is especially true for many domain-specific generative tasks, such as medical images, fine-grained datasets in taxonomy, and data scraped from the web. It's difficult to collect a large amount of data equally for each category; sometimes the training sets for head and tail categories can differ by a factor of one hundred or more. Head categories are those with more training samples, while tail categories have fewer. Since a severely uneven distribution is called a long-tailed distribution, the class imbalance problem is also known as the long-tail problem. Class imbalance is very common in practical applications, such as anomaly detection, the uneven distribution of species in nature, and the analysis of rare pathological data. In these cases, the performance of deep learning models is often dominated by head classes, while learning from tail classes is severely lacking. The goal of imbalance learning is to learn an unbiased model from imbalanced data. Depending on the application scenario, research on imbalance learning can cover many fields, including classification models and generative models. In the field of classification tasks, techniques specifically studying imbalance learning phenomena are defined as long-tail recognition tasks. Research in this field can be categorized into three paradigms: class rebalancing, information augmentation, and module improvement. In class rebalancing, resampling methods adjust the sampling probabilities of different classes to achieve class rebalancing; loss-sensitive learning adjusts the loss weights corresponding to samples based on different classes; and logarithmic adjustment methods, based on theoretical derivation, directly change the model's output value. Information augmentation includes transfer learning and data augmentation. Transfer learning seeks to transfer information from the source domain to the target domain, including head-and-tail knowledge transfer and model pre-training methods. Data augmentation is a general technique to prevent overfitting. Module improvement methods adjust the classifier, including but not limited to representation learning and classifier module design.

[0004] However, the class imbalance problem remains largely unresolved in the field of generative models. Existing solutions to class imbalance in generative models primarily include class-rebalancing generative adversarial networks (CBGANs) and group spectrum regularizers, both based on CBGAN design. CBGANs, based on the exponential forgetting phenomenon observed in neural networks, employ resampling to introduce a new, theoretically motivated class-balancing regularizer, encouraging the model to focus on underrepresented tail classes; however, its performance is limited. Group spectrum regularizers, on the other hand, are tailored to the specific architecture of CBGANs, but this also prevents them from being transferred to other models besides CBGANs. Summary of the Invention

[0005] To address some or all of the problems in existing technologies and improve the performance of diffusion models under class imbalance conditions, the first aspect of this invention provides an optimization method for diffusion models, comprising:

[0006] In a mini-batch, steps are randomly sampled from the Markov chain path of the diffusion process to obtain step t in the current batch;

[0007] Randomly add noise to each pixel in the image of step t to obtain the noisy image of step t;

[0008] Based on the noisy image, the original image is predicted using a model to obtain the noise predicted by the model;

[0009] Calculate the mean square loss between the noise predicted by the model and the actual noise;

[0010] Additional random labels are obtained by sampling from a set of labels, wherein the set of labels follows a preset distribution; and

[0011] Based on the random labels, the noise in the noisy image is re-predicted, and the distribution adjustment loss is calculated.

[0012] Furthermore, step t is obtained according to the following steps:

[0013] In each mini-batch, random sampling is performed on a set containing all integers from 1 to T to obtain a random step t, where T is the total number of steps in the diffusion process.

[0014] Furthermore, the random noise addition includes:

[0015] Calculate the mean and covariance matrix of the Gaussian distribution based on step t;

[0016] A noise matrix of the same size as the image is obtained based on Gaussian distribution sampling; and

[0017] Random noise is added to each pixel in the image based on the noise matrix.

[0018] Furthermore, the random noise addition includes:

[0019] During the forward pass, noise is progressively added to the original image to obtain a noisy image, where the noisy image at step t is obtained through... Obtained directly by sampling, among which Let be the accumulated noise level from step 1 to step t, and ∈ be the noise sampled from a multivariate standard Gaussian distribution.

[0020] Furthermore, the noise predicted by the model is obtained according to the following steps:

[0021] Based on the noisy image, the noise predicted by the model is obtained by predicting the original image using a model with ∈ as a parameter, where ∈ is the noise sampled from a multivariate standard Gaussian distribution.

[0022] Furthermore, the mean square loss is determined by the KL divergence D. KL Simplified, the result is obtained using the following formula:

[0023] E q [∑ t≥1 D KL [q(x t-1 |x t ,x0)||p θ (x t-1 |x t )]]=E q [∑ t≥1 L t-1 ],

[0024] in, Let represent the loss at time t-1, where μ is the expectation of the true Gaussian distribution. θ (x t Let ,t) be the expected distribution predicted by the model, C be a constant term, and L0 = E q [logp θ (x0|x t )).

[0025] Furthermore, the mean square loss is determined by the KL divergence D. KL Simplified, the result is obtained using the following formula:

[0026] E q [∑ t≥1 D KL [q(x t-1 |x t ,x0)||p θ (x t-1 |x t )]]=E q [∑ t≥1 L t-1 ],

[0027] in, Let denot be the loss at time t-1, and let ∈ be the noise sampled from the multivariate standard Gaussian distribution. θ For model prediction noise, and L0 = E q [logp θ (x0|x t )).

[0028] Furthermore, the random labels are obtained by sampling according to the following steps:

[0029] Monte Carlo sampling is performed on the target label distribution to obtain the label set; and

[0030] Additional random labels are obtained by sampling from the set of labels.

[0031] Furthermore, the distribution adjustment loss is calculated based on the mean squared loss of the predicted content obtained from the correct label and the random label.

[0032] Furthermore, the distribution adjustment loss includes a regularization loss L. reg and consistency loss L commit ,in:

[0033]

[0034]

[0035] Here, sg() is the gradient stopping function.

[0036] A second aspect of the present invention provides an image generation method, comprising:

[0037] The diffusion model is optimized according to the method described above; and

[0038] The optimized diffusion model employs a classifier-free sampling method to generate images.

[0039] Based on the diffusion model acquisition method described above, a third aspect of the present invention provides an electronic device for optimizing a diffusion model, comprising a memory and a processor, wherein the memory is configured to store a computer program that executes the diffusion model optimization method described above when the processor is running.

[0040] A fourth aspect of the present invention also provides a computer-readable storage medium for optimizing a diffusion model, which stores a computer program that, when run on a processor, executes the aforementioned method for optimizing the diffusion model.

[0041] This invention provides an optimization method for a diffusion model. When training data is imbalanced, it trains a conditional diffusion model based on the imbalanced training data, enabling the generation of images with higher diversity in the tail category while preserving the fidelity of the tail category sampled images. The acquisition method adjusts the conditional transition probability during sampling, implicitly forcing the generated images to approximate the target prior distribution in each sampling step. Furthermore, it designs an additional mean squared error loss, thereby enhancing the similarity of conditionally generated images based on different categories, transferring common information from the head category to the tail category without compromising the model's performance in the head category. This acquisition method fills a research gap in training more robust generative models based on long-tailed distributed data.

[0042] Experimental results demonstrate that the optimization method significantly improves the diversity of generated images across four common metrics (FID, IS, Precision, and Recall), visualization, and downstream tasks. For example, the optimization method significantly enhances the model's training results on the CIFAR100LT and CIFAR100 datasets, particularly its generation diversity and coverage. The model also provides a new benchmark for generation tasks on the CIFAR100 dataset (FID = 2.721). Attached Figure Description

[0043] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0044] Figure 1 A flowchart illustrating an optimization method for a diffusion model according to an embodiment of the present invention is shown.

[0045] Figure 2 This diagram illustrates a process for optimizing a diffusion model according to an embodiment of the present invention.

[0046] Figure 3 This diagram illustrates a flowchart of an image generation method according to an embodiment of the present invention; and

[0047] Figure 4 The diagram illustrates a process of an image generation method according to an embodiment of the present invention. Detailed Implementation

[0048] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or with other alternatives and / or additional methods, materials, or components. In other instances, well-known structures, materials, or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific quantities, materials, and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details. Furthermore, it should be understood that the embodiments shown in the drawings are illustrative representations and are not necessarily drawn to scale.

[0049] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0050] It should be noted that the embodiments of the present invention describe the process steps in a specific order; however, this is only for illustrating the specific embodiment and not for limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to the process.

[0051] In embodiments of the present invention, the term "correct label" is defined as follows: a label given to the generator during image generation, the purpose of which is to better guide the generation process. In embodiments of the present invention, the correct label is generally the category of the noisy image to be restored, such as: sky, fish, bird.

[0052] For datasets with significant differences between the head and tail categories, bi-module iteratively updated generative adversarial networks (GANs) often face training instability, failing to adequately capture tail category patterns. Conditional diffusion models also produce a considerable number of low-quality images when class distribution is uneven. While GANs can generate high-quality images for the head category (where training data is plentiful), images for the tail category are likely to display semantics unrelated to that category. Numerically, the performance degradation of the tail category is reflected in the increase of the common FID (Frame Indicator) metric in generation tasks. Currently, for generative models with limited training data, existing research often focuses on data augmentation and adversarial GANs. GAN-based methods can train generative models with limited data. However, regarding the impact of imbalanced class distribution, there is very little research on GANs, primarily in the field of recognition problems. Currently, no research addresses the issue of imbalanced class distribution in diffusion models.

[0053] Existing diffusion models typically assume a uniform data distribution, thus treating each training sample equally during training. However, this training strategy leads to a decline in the quality of tail category generation, and the FID score increases significantly as the size of the training samples for each category decreases. The inventors discovered a fidelity-diversity tradeoff caused by the guidance term. Specifically, conditional diffusion models tend to generate images with ambiguous category semantics for tail categories. While the pattern collapse problem in tail categories is alleviated by the guidance term, the additional guidance term leads to severe overfitting, resulting in a degradation in the model's diversity for tail categories. For example, as the guidance rate increases, FID (a combined metric of diversity and fidelity) typically worsens, while IS (biased towards fidelity) improves. This tradeoff is even more pronounced for tail categories with fewer training images. Further research by the inventors revealed that most of the recognizable high-quality generated images in the tail category are highly similar to the samples in the training set. This indicates that the traditional guided diffusion model lacks the ability to generate images in unknown regions. In addition, the tail category generation feature, which almost replicates the training samples, makes the guided diffusion model very vulnerable to member inference attacks.

[0054] To address the above issues and improve the performance of diffusion models under class imbalance, this invention proposes a Class-Balancing Diffusion Model (CBDM) and its training / optimization method, introducing the long-tail generation task into the diffusion model. The diffusion model uses a regularization term to calibrate the imbalanced prior distribution, thereby enhancing the diversity of the tail images. Specifically, in each mini-batch, steps are randomly sampled from the Markov chain path of the diffusion process to obtain step t in that batch. Then, based on step t obtained by random sampling of the path, a noise matrix of the same size as the image is obtained by sampling according to a Gaussian distribution. Noise is randomly added to each pixel in the image to obtain the noisy image of step t. Based on the noisy image obtained in the above steps, the original image is predicted by the model to obtain the noise predicted by the model. The noise can restore the noisy image to a clear image. According to the loss definition given by the denoising diffusion probability model, the mean square loss between the model's predicted noise and the real noise is calculated. Finally, additional random labels y' are sampled from the label set following a certain distribution. The noise of the noisy image is re-predicted based on the random labels y', and the mean square loss of the predicted content obtained based on the correct label and the random label is used as the distribution adjustment loss. It can be seen that the optimization method of the diffusion model in the embodiment of the present invention adjusts each propagation step in the diffusion model so that the model approximates the label distribution under the balanced dataset. Furthermore, this invention adds a regularization loss to the DDPM loss, specifically the quadratic distance between the noise predicted by the conditional diffusion model and the predicted noise based on the sampled random labels. This loss acts like a regularization term, forcing the model to pay more balanced attention to the tail and head classes. The method can be very concisely embedded into the training of any diffusion model, thus it can be widely applied to various conditional diffusion models, offering advantages such as high interpretability, simple implementation, and low time and space complexity.

[0055] The present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0056] Figure 1 , Figure 2 The figures illustrate a flowchart and process diagram of an optimization method for a diffusion model according to an embodiment of the present invention. As shown, an optimization method for a diffusion model includes:

[0057] First, in step 101, path sampling occurs. The diffusion model, inspired by non-equilibrium thermodynamics, defines a progressively diffusing Markov chain, slowly adding random noise to the data until the image approximates white noise, and then learning to reverse the diffusion process to construct the desired data samples from the noise. Based on this, in one embodiment of the invention, steps are first randomly sampled from the Markov chain path of the diffusion process to obtain step t in the current batch. In one embodiment of the invention, in each mini-batch, step t is sampled from a uniform distribution of the Markov chain path. Specifically, if the total number of steps in the diffusion process is T, random sampling is performed in a set containing all integers from 1 to T to obtain random step t. In one embodiment of the invention, the value of T is preferably 1000.

[0058] Next, in step 102, image noise is added. Random noise is added to each pixel in the image of step t to obtain the noisy image of step t. In one embodiment of the invention, the mean and covariance matrix of the Gaussian distribution are calculated based on step t, and then a noise matrix of the same size as the image is obtained by sampling based on the Gaussian distribution. Random noise is then added to each pixel in the image to obtain the noisy image of step t. Specifically, let the original image be x0. During the forward process, noise is gradually added to the original image x0, and let β... t To determine the variance of the noise added at each step of forward propagation, the probability of the noisy image propagating step by step is:

[0059]

[0060] Among them, due to step x t The state depends only on x in the previous step t-1 The value of β is such that the forward diffusion process actually constitutes a Markov chain propagating in time. t =1-β t , ∈ represents noise sampled from a multivariate standard Gaussian distribution. By superimposing the noise from each step, the noise image of step t can be equivalently obtained through... Direct sampling, among which a represents the cumulative noise level from step 1 to step t. s The noise level at step s is the single-step noise level. The higher the step t, the stronger the noise variance.

[0061] Next, in step 103, noise is predicted. Based on the noisy image obtained in step 102, the original image is predicted using a model to obtain the noise predicted by the model, wherein the noise can restore the noisy image to a clear image. In one embodiment of the present invention, based on the noisy image obtained in step 102, the original image is predicted using a model with ∈ as a parameter to obtain the noise ∈ predicted by the model. θ ;

[0062] Next, in step 104, the mean squared loss is calculated. Based on the loss definition given by the denoising diffusion probability model, the noise ∈ predicted by the model is calculated. θ The mean squared loss between the original image x0 and the true noise ∈. Specifically, similar to other generative models, the main objective of the denoising diffusion probability model is to maximize the log-likelihood of the original image x0. As shown below, by applying the usual variational constraints to the negative likelihood and decomposing the forward process into a cumulative multiplication of Markov chains, the log-likelihood E q [-logp θ [x0] can be simplified to:

[0063]

[0064] Since the first term does not contain trainable parameters, it can be ignored during training. After further simplification, the loss function can be expressed as the KL divergence between the prior transition probabilities and the model's predicted transition probabilities at each step. In one embodiment of this invention, the mean squared loss is calculated according to the following formula:

[0065] E q [∑ t≥1 D KL [q(x t-1 |x t ,x0)||p θ (x t-1 |x t )]]=E q [∑ t≥1 L t-1 ];

[0066] Calculations show that the loss at time t-1 is... in, μ represents the expectation of the true Gaussian distribution. θ (x t ,t) represents the expected distribution predicted by the model, and C is a constant term. In another embodiment of the invention, the weighting parameter is ignored, L t-1 It can be further simplified to Where ∈ represents the Gaussian noise added to the image during the image noise addition step, i.e., the real noise. θThis is noise predicted by the model. Furthermore, when t=1, L t-1 =L0=E q [logp θ (x0|x t )];

[0067] Next, in step 105, random labels are sampled. Additional random labels y′ are obtained by sampling from a set of labels following a preset distribution. In one embodiment of the invention, the random labels are sampled according to the following principle:

[0068] When training a diffusion model with parameter θ on an imbalanced dataset, during backsampling, its conditional distribution p θ (x t-1 |x t (y) can be calibrated in the following ways:

[0069]

[0070] Through derivation, the target training objective calibrated in step t is represented as: remember The target label distribution to be adjusted is, for example, a category-balanced label distribution. The upper bound of this loss can be derived as follows:

[0071]

[0072] Based on this, in one embodiment of the present invention, the target label distribution is... Monte Carlo sampling is performed to obtain a label set Y, where y' is one of the elements in set Y; that is, additional random labels y' are obtained by sampling from the label set Y.

[0073] Finally, in step 106, the distribution adjustment loss is calculated. Based on the random label y', the noise in the noisy image is re-predicted, and the distribution adjustment loss is calculated. In one embodiment of the invention, the mean squared loss of the predicted content obtained based on the correct label and the random label is used as the distribution adjustment loss. In another embodiment of the invention, the distribution adjustment loss is a regularization loss L. reg and consistency loss L commit The sum of, among which

[0074]

[0075]

[0076] Here, sg() represents gradient stopping. The gradient stopping operation is to prevent model crashes, because directly applying... Optimizing with a loss function of this form may cause the model to collapse, thus converging to a trivial solution, i.e., given x... t Regardless of the input y, the generator outputs only the same result, and applying a stopping gradient to the adjustment term effectively prevents model collapse. Furthermore, the consistency loss L... commit This strengthens the role of the adjustment term and further avoids trivial solutions. The consistency loss is symmetrical in form to the regularization loss, and both need to be adjusted within the... The operation of applying gradient stopping.

[0077] This completes the optimized training of the diffusion model, resulting in a conditional diffusion model capable of obtaining more diverse images from tail-category sampling while preserving the fidelity of tail-category sampled images. A more robust generative model was also obtained by training on long-tailed distribution data. The acquisition method described above can be implemented in just a few lines of code, and its lightweight version also allows for fine-tuning of existing conditional models. Verification shows that the method effectively optimizes common generative metrics based on the CIFAR100 and CIFAR100LT datasets and significantly improves tail-category diversity.

[0078] Based on the optimization method of the diffusion model described above, Figure 3 and Figure 4 The figures illustrate a flowchart and process diagram of an image generation method according to an embodiment of the present invention. As shown, an image generation method includes:

[0079] First, in step 201, the model is trained. The diffusion model is trained according to the optimization method described above; and

[0080] Finally, in step 202, the image is generated. After obtaining the trained diffusion model, image generation is performed using a classifier-free guided (CFG) sampling method. The classifier-free guided method aims to conditionally guide the generation process by the difference between the model's predictions under given class conditions and unconditional conditions. Its advantage lies in its extreme simplicity; only one line of code needs to be changed during training, allowing the model to be trained with unconditional generation loss with a 10% probability and with conventional conditional generation loss with a 90% probability. Furthermore, a lower guidance rate (typically less than 0.2) can slightly improve the FID metric of the generated image. Specifically, the sampling process and the diffusion process consist of the same number of steps in opposite directions. In each sampling step, the diffusion model performs back-diffusion, i.e., the sampling formula is as follows:

[0081]

[0082] Classifier-free guided (CFG) sampling methods do not require training an additional model; instead, they implicitly use the idea of ​​classification guidance during image generation. The model's predictions during sampling can be divided into two parts: one corresponding to the gradient of the unconditionally generated log-likelihood, and the other corresponding to the gradient of the classifier-guided log-likelihood. Because... Therefore, noise guided by category conditions The formula is as follows:

[0083]

[0084] In the above formula, the first term is the generated result of the conditional model, and the second term is the guiding term. However, just as with the performance of logarithmic guiding terms in classification problems, a guidance of unit 1 is often not the optimal choice. When the guiding magnitude is not unit 1, the guidance rate is defined as ω. This guiding term usually improves sampling fidelity (reflected in the decrease of FID) at the cost of sacrificing sampling diversity (reflected in the increase of IS). Based on this:

[0085]

[0086] The final classifier-free guided sampling method is based on the conditional prediction results ∈ θ (x t ,y) and unconditional prediction results ∈ θ (x t Linear interpolation representation of ):

[0087]

[0088] Where θ are model parameters, ∈ is the model for predicting noise, and x t ω is the noise image at step t, and ω is the guiding strength of the classifier-free guided sampling method.

[0089] Based on the aforementioned optimization method for the diffusion model, the present invention also provides an electronic device for optimizing the diffusion model, comprising a memory and a processor, wherein the memory is configured to store a computer program that executes the aforementioned optimization method for the diffusion model when the processor is running.

[0090] The present invention also provides a computer-readable storage medium for optimizing a diffusion model, which stores a computer program that, when run on a processor, executes the optimization model acquisition method as described above.

[0091] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.

Claims

1. An optimization method for a diffusion model, characterized in that, Including the following steps: In a mini-batch, steps are randomly sampled from the Markov chain path of the diffusion process to obtain the steps in the current batch. ; For the steps Randomly add noise to each pixel in the image to obtain the step. Noisy images; Based on the noisy image, the original image is predicted using a model to obtain the noise predicted by the model; Calculate the mean square loss between the noise predicted by the model and the actual noise; Additional random labels are obtained by sampling from a set of labels, wherein the set of labels follows a preset distribution; and Based on the random labels, the noise in the noisy image is re-predicted, and a distribution adjustment loss is calculated based on the mean squared loss of the predicted content obtained from the correct labels and the random labels, wherein the distribution adjustment loss includes a regularization loss. and consistency loss ,in: ， ， in, Gradient stopping.

2. The optimization method as described in claim 1, characterized in that, Random sampling of the steps includes: In each mini-batch, random sampling is performed on a set containing all integers from 1 to T to obtain random steps. , where T is the total number of steps in the diffusion process.

3. The optimization method as described in claim 1, characterized in that, The random noise addition includes: Based on the steps Calculate the mean and covariance matrix of the Gaussian distribution; A noise matrix of the same size as the image is obtained based on Gaussian distribution sampling; and Random noise is added to each pixel in the image based on the noise matrix.

4. The optimization method as described in claim 1, characterized in that, The random noise addition includes: During the forward pass, noise is gradually added to the original image to obtain a noisy image, where the steps include... Noisy images via Obtained directly by sampling, among which , where is the accumulated noise level from step 1 to step t. This is noise sampled from a multivariate standard Gaussian distribution.

5. The optimization method as described in claim 1, characterized in that, The noise predicted by the model is obtained through the following steps: Based on the noisy image, by using The model with parameters is used to predict the noise of the original image, where... This is noise sampled from a multivariate standard Gaussian distribution.

6. The optimization method as described in claim 1, characterized in that, The mean square loss is determined by the KL divergence. Simplified calculation yields: ， in, , representing the loss at time t-1, where The expectation of the true Gaussian distribution, The expected distribution predicted by the model. For constant terms, and .

7. The optimization method as described in claim 1, characterized in that, The mean square loss is determined by the KL divergence. Simplified calculation yields: ， in, , representing the loss at time t-1. For noise sampled from a multivariate standard Gaussian distribution, To predict noise for the model, and .

8. The optimization method as described in claim 1, characterized in that, The random labels are obtained by sampling according to the following steps: Monte Carlo sampling is performed on the target label distribution to obtain the label set; and Additional random labels are obtained by sampling from the set of labels.

9. An image generation method, characterized in that, Including the following steps: The diffusion model is optimized according to the method described in any one of claims 1 to 8; and The optimized diffusion model employs a classifier-free sampling method to generate images.

10. An electronic device for optimizing a diffusion model, characterized in that, It includes a memory and a processor, wherein the memory is configured to store a computer program that, when the processor is running, executes an optimization method for the diffusion model as described in any one of claims 1 to 8.

11. A computer-readable storage medium for optimizing a diffusion model, characterized in that, The device contains a computer program that, when run on a processor, executes the optimization method for the diffusion model as described in any one of claims 1 to 8.

Images