Method and device for applying optimal step length to diffusion model, equipment and storage medium
By constructing a symmetric distribution and iterative training, the optimal time step of the diffusion model is determined, which solves the problem that the maximum number of noise additions depends on experience in the existing technology, optimizes the model performance and resource allocation, and generates high-quality images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2024-10-09
- Publication Date
- 2026-04-17
AI Technical Summary
Existing diffusion models typically rely on experience when setting the maximum number of noise additions, which increases the complexity of model training and inference, reduces the reliability of the model, and lacks quantitative evaluation, making it difficult to optimize model performance.
By constructing a symmetric distribution, iteratively selecting time steps to train the diffusion model for noise reduction, calculating the distribution difference, determining the value of the optimal time step, and using it as the maximum number of noise additions, the model parameters are optimized, and the model performance is optimized using the loss function and gradient descent.
It improves the reliability of the images generated by the model, optimizes the allocation of computing resources, reduces training and application time, and generates high-quality images.
Smart Images
Figure CN121883291A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision, and in particular to a method, apparatus, device, and storage medium for applying optimal step size to a diffusion model. Background Technology
[0002] In recent years, diffusion models have made significant progress in the field of generative models. As a powerful generative model, it has been widely used in multiple fields such as image generation, image inpainting, and text generation. Diffusion models generate high-quality samples by gradually adding Gaussian noise to the data and learning the inverse process, i.e., the noise reduction process.
[0003] With the development of deep learning technology, the neural network architecture within deep learning provides a powerful foundation for image feature extraction and processing. Simultaneously, continuous advancements in hardware technology have made it possible to train large-scale diffusion models. The widespread adoption of the internet and the accelerated pace of digitization have generated massive amounts of image data; these abundant data resources provide ample material for training diffusion models, enabling them to learn a wider range of image patterns and features, thereby improving the quality of generated images.
[0004] In the diffusion model, a maximum number of noise additions needs to be set. In practice, this is usually set by developers based on experience, which may increase the complexity of model training and inference, leading to a decrease in model reliability. Summary of the Invention
[0005] This application provides a method, apparatus, device, and storage medium for applying optimal step size to a diffusion model to improve the reliability of the model.
[0006] On the one hand, this application provides an image generation method, including:
[0007] Noise is added based on the current number of noise additions T for the samples and the training image to generate sample data x. T ; Using sample data x T Construct a symmetrical distribution centered on the target; repeat iterative training until the preset number of iterations is reached. Iterative training includes: randomly selecting time step i within time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy and denoised sample data obtained in each training iteration is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result. Based on the evaluation result, the optimal time step value is determined. The optimal time step value is used as the current number of times the sample is noisy, and the process returns to generate sample data x based on the current number of times the sample is noisy T. TThe steps are repeated until the value of the optimal time step remains unchanged; the maximum number of noise additions to the diffusion model is configured to the value of the optimal time step to obtain the trained diffusion model; and the trained diffusion model is used to generate an image.
[0008] In some examples, based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The model calculates the loss value of the predicted noise and the actual noise using a loss function, and updates the model parameters by performing gradient descent optimization based on the loss value.
[0009] In some examples, the probability density function or cumulative distribution function of the denoised sample data in a symmetric distribution is obtained; and the probability density function or cumulative distribution function of the noisy sample data in a symmetric distribution is obtained; the distribution difference between the probability density function or cumulative distribution function of the denoised sample data and the noisy sample data is calculated; and the sample data whose distribution difference exceeds a preset threshold is evaluated as the evaluation result of this iteration training.
[0010] In some examples, the value of the time step corresponding to the sample data that appears most frequently in the evaluation results is taken as the value of the optimal time step.
[0011] In some examples, standard Gaussian noise is added to the training image based on the current number of times the sample has been noised, T.
[0012] In some examples, a noisy image is input; the noisy image is fed into a trained diffusion model for prediction and denoising until the maximum number of denoising operations is reached, and the image currently output by the diffusion model is used as the generated image.
[0013] On the other hand, this application provides an image generation model apparatus, comprising:
[0014] The module is used to add noise to the current sample based on the number of noise additions T and the training image, generating sample data x. T ; Using sample data x T A symmetrical distribution is constructed around the center; the training module is used to repeatedly iterate until a preset number of iterations is reached. The iterative training includes: randomly selecting time step i in time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-iThe distribution difference between the noisy and denoised sample data obtained in each training iteration is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result. The processing module determines the optimal time step value based on the evaluation result. The optimal time step value is used as the current number of times the sample is noisy, and the process returns to generate sample data x based on the current number of times the sample is noisy (T). T The steps are repeated until the value of the optimal time step remains unchanged; the generation module is used to configure the maximum number of noise additions of the diffusion model to the value of the optimal time step, so as to obtain the trained diffusion model; and the trained diffusion model is used to generate an image.
[0015] In some examples, the training module is used to: train the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The model calculates the loss value of the predicted noise and the actual noise using a loss function, and updates the model parameters by performing gradient descent optimization based on the loss value.
[0016] In some examples, the training module is used to: obtain the probability density function or cumulative distribution function of the denoised sample data in a symmetric distribution; and obtain the probability density function or cumulative distribution function of the noisy sample data in a symmetric distribution; calculate the distribution difference between the probability density function or cumulative distribution function of the denoised sample data and the noisy sample data; and evaluate the sample data whose distribution difference exceeds a preset threshold as the evaluation result.
[0017] In some examples, the training module is used to: take the value of the time step corresponding to the sample data that appears most frequently in the evaluation results as the value of the optimal time step.
[0018] In some examples, the building block is used to add standard Gaussian noise to the training image based on the current number of times the sample has been noised, T.
[0019] In another aspect, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the aforementioned method.
[0020] In another aspect, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the aforementioned method.
[0021] This application provides a method, apparatus, device, and storage medium for applying optimal step size to a diffusion model. First, sample data is generated by adding noise, and a symmetrical distribution is constructed. The diffusion model is then iteratively trained using selected time steps and noisy sample data to obtain denoised sample data. The distribution difference between the noisy and denoised sample data obtained in each iteration is calculated, and the difference is evaluated based on the symmetrical distribution to obtain an evaluation result. Based on the evaluation result, the optimal time step value is determined. The optimal time step value is used as the current number of noise additions to the samples, and the symmetrical distribution is constructed again until the optimal time step value remains unchanged. The maximum number of noise additions to the diffusion model is then configured to the optimal time step value to obtain a trained diffusion model. Subsequently, the trained diffusion model is used to generate images. This scheme utilizes the symmetry of the distribution to quantitatively evaluate the effect of noise addition and denoising at each step, thereby determining the maximum number of noise additions. This facilitates efficient allocation of computing resources and optimization of model performance, reduces model training and application time, and thus improves the reliability of the image generated by the model. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0023] Figure 1 The diagram above illustrates a flowchart of an image generation method.
[0024] Figure 2 A schematic diagram showing a symmetrical distribution;
[0025] Figure 3 The diagram above exemplarily illustrates the structure of an image generation apparatus;
[0026] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0027] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0030] Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device. As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.
[0031] The diffusion model generates images primarily through two processes: noise addition and noise reduction. The noise reduction process involves progressively reducing noise before generating the final image. In general, throughout the process, the diffusion model learns the distribution of the training data, gradually adding noise during the noise addition process and progressively restoring the image during the noise reduction process, ultimately generating a new image.
[0032] In related technologies, image generation based on diffusion models requires setting a fixed maximum number of noise additions. In practice, this is usually set by developers based on experience, which may increase the complexity of model training and inference, leading to reduced model reliability. For example, a fixed maximum number of noise additions cannot be adjusted based on factors such as data complexity, noise level, and feature distribution. Furthermore, excessively long noise addition sequences can lead to wasted computational resources and increased complexity of model training and inference. In addition, existing diffusion models lack quantitative evaluation, making it difficult to find optimal model parameters when optimizing model performance.
[0033] The technical solution provided in this application aims to solve the aforementioned technical problems of the prior art. In the embodiments of this application, sample data is first generated by adding noise; a symmetrical distribution is constructed; a diffusion model is iteratively trained using time steps and noisy sample data to obtain denoised sample data; the distribution difference between the noisy and denoised sample data obtained in each iteration is calculated, and the distribution difference is evaluated based on the symmetrical distribution to obtain an evaluation result; based on the evaluation result, the value of the optimal time step is determined; the value of the optimal time step is used as the current number of noise additions to the sample, and the symmetrical distribution is constructed again until the value of the optimal time step remains unchanged. The maximum number of noise additions to the diffusion model is configured to the value of the optimal time step to obtain a trained diffusion model, which is then used to generate images. This solution utilizes the symmetry of the distribution to quantitatively evaluate the effect of noise addition and denoising at each step, thereby determining the value of the maximum number of noise additions. This facilitates the effective allocation of computing resources and the optimization of model performance, reduces the time for model training and application, and thus improves the reliability of the image generated by the model.
[0034] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0035] Example 1
[0036] Figure 1 The diagram illustrates a flowchart of an image generation method. This embodiment involves generating images using a diffusion model, such as... Figure 1 As shown, the method includes:
[0037] S101, add noise based on the current number of noise additions T for the samples and the training image to generate sample data x. T ; Using sample data x T Construct a symmetrical distribution centered on the target.
[0038] S102, repeat the iterative training until the preset number of iterations is reached. The iterative training includes: randomly selecting time step i in time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy sample data and the denoised sample data obtained in each iteration of training is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result.
[0039] S103, Based on the evaluation results, determine the value of the optimal time step; use the value of the optimal time step as the current number of times noise is added to the sample, and return to execute the process of generating sample data x based on the current number of times noise is added to the sample T. T The steps are repeated until the value of the optimal time step remains unchanged.
[0040] S104, Configure the maximum number of noise additions to the diffusion model to the value of the optimal time step to obtain the trained diffusion model; use the trained diffusion model to generate an image.
[0041] The method of adding noise is unrestricted; for example, it can be based on a noise-adding formula. As an example, a Markov chain with multiple time steps can be defined for noise addition. Noise is added to the training image at each time step. A Markov chain is a stochastic process where the state at time t+1 depends only on the state at time t and is independent of states between times t. In the diffusion model, by defining a Markov chain, the noise-adding process can be decomposed into a series of discrete steps, each depending only on the state of the previous step, where each state corresponds to data with varying degrees of added noise.
[0042] In practical scenarios, when generating images using a diffusion model, noise is first added based on the current number of noise additions T for the samples and the training images to generate sample data x. T ; Using sample data x T Construct a symmetrical distribution centered on the target; repeat iterative training until the preset number of iterations is reached. Iterative training includes: randomly selecting time step i within time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy and denoised sample data obtained in each training iteration is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result. Based on the evaluation result, the optimal time step value is determined. The optimal time step value is used as the current number of times the sample is noisy, and the process returns to generate sample data x based on the current number of times the sample is noisy T. T The steps are repeated until the value of the optimal time step remains unchanged; the maximum number of noise additions to the diffusion model is configured to the value of the optimal time step to obtain the trained diffusion model; and the trained diffusion model is used to generate images.
[0043] The initial number of noise-adding iterations and the number of training iterations can be set according to the actual situation. For example, the number of training iterations can be 500. Alternatively, the initial number of noise-adding iterations can be set to 200. Taking the above process as an example, generating sample data x by adding noise... 200 , with x 200Construct a symmetrical distribution centered on the target; repeat the training iteration 500 times, where the iteration training includes: randomly selecting time step 100 from time step 0-200 to obtain noisy sample data x. 100 Based on the noisy sample data x 100 The diffusion model was trained with 100 time steps to achieve denoising, resulting in denoised sample data p. 100 This iterative training process is repeated until 500 iterations are completed, resulting in 500 noisy and denoised sample data points. The distribution difference between these 500 noisy and denoised sample data points is calculated, and based on a symmetric distribution, the distribution difference is evaluated to obtain an evaluation result. Based on the evaluation result, the optimal time step value is determined and used as the current number of times noise is added to the sample. The process is repeated until the optimal time step value remains unchanged. The maximum number of times noise is added to the diffusion model is configured to the optimal time step value, resulting in a trained diffusion model. The trained diffusion model is then used to generate images. For example, if the optimal time step value is 80, and this is used as the number of times noise is added to the sample, the process is repeated, and after the iterative training ends, the determined optimal time step value remains 80; therefore, the maximum number of times noise is added to the diffusion model is 80.
[0044] In practical applications, incorporating a symmetrical distribution into the diffusion model can create a more balanced noise addition and reduction process, reducing instability and oscillations during training. Figure 2 This is a schematic diagram of a symmetrical distribution, specifically, as shown below. Figure 2 As shown, with x T The system is constructed around a center of symmetry to ensure that any noisy sample data and noisy sample data satisfy symmetry; at the same time, the addition and removal of standard Gaussian noise have similar properties.
[0045] This scheme incorporates symmetric distribution considerations during model training. For different numbers of noise additions to samples, the optimal maximum number of noise additions is determined by evaluating the characteristics of the noise-added and noise-reduced samples within a symmetric distribution. Specifically, the model training process is illustrated with some examples. In one example, in S102, based on the noise-added sample data x... i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i ,include:
[0046] The diffusion model is trained to reduce noise based on the noise-added sample data xi and time step i, resulting in the noise-reduced sample data pT-i and the predicted noise.
[0047] Based on the predicted noise and the actual noise, the loss value of the predicted noise is calculated using a loss function; based on the loss value, the model parameters are updated by performing gradient descent optimization.
[0048] Among them, the loss value and loss function are used to measure the difference between the samples generated by the model and the real data, and guide the training process of the model. By comparing the loss value, the parameters of the model are continuously adjusted so that the loss value gradually decreases, thereby making the generated results of the model closer and closer to the real data.
[0049] In some examples, in S102, the evaluation results obtained by evaluating the noisy sample data and the denoised sample data according to the symmetric distribution include: obtaining the probability density function or cumulative distribution function of the denoised sample data in the symmetric distribution; and obtaining the probability density function or cumulative distribution function of the noisy sample data in the symmetric distribution; calculating the distribution difference between the probability density function or cumulative distribution function of the denoised sample data and the noisy sample data; and evaluating the sample data whose distribution difference exceeds a preset threshold as the evaluation result.
[0050] Specifically, the approximate equality of the probability density functions or cumulative distribution functions of the two data sets indicates that the denoised sample data is far from the distribution center. The probability density function (PDF) and cumulative distribution function (CDF) are important tools for describing probability distributions, helping to understand the distribution characteristics of sample data and guiding the training and generation process of diffusion models. To ensure the symmetry and sign of the denoised and noisy sample data, PDF and CDF can be used; Min||f(x) can be applied. i )-f(p T-i ||2 is used as an additional optimization term to ensure symmetry, while ||1-cdf(f,x) is used. i )-cdf(f,p T-i )||2 to ensure its positive or negative sign.
[0051] In some examples, in S103, determining the value of the optimal time step based on the obtained evaluation results includes taking the value of the time step corresponding to the sample data that appears most frequently in the evaluation results as the value of the optimal time step.
[0052] In this example, the preset threshold is determined through actual testing and analysis of the image generation effect. Specifically, firstly, evaluation metrics are determined; test data is collected by selecting a representative real image dataset for comparison with the generated image, and a large number of image samples, i.e., test data, are generated using a diffusion model; secondly, actual testing is conducted, namely, the subjective evaluation of the generated image by observers and the objective evaluation of the generated image by the evaluation metrics, and the value of each evaluation metric is calculated; finally, the objective evaluation values are statistically analyzed and combined with the subjective evaluation to determine the threshold.
[0053] This embodiment provides an image generation method. For example, firstly, noise is added based on the current number of noise additions T and the training image to generate sample data x. T ; Using sample data x T A symmetric distribution is constructed around the center; in each iteration of training, the diffusion model and the symmetric distribution are trained simultaneously; after the iteration training is completed, the distribution difference between the noisy sample data and the denoised sample data is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result; based on the obtained evaluation result, the value of the optimal time step is determined; such that |pdf(p T-k )-f(x T )|>d or |cdf(f,p T-k If )-1|>d, it indicates that the denoising operation after k iterations has little impact on the samples, so the number of times the samples are denoised can be reduced to k. Repeat training until the value at the optimal time step remains unchanged, i.e., k=0. The value at the optimal time step is then the maximum number of denoising iterations. Specifically, d represents the error threshold; where the symmetric distribution of training includes, but is not limited to, the dimensionality C of the feature map corresponding to the denoised feature map in the diffusion model, i.e., learning C standard deviations, and expanding through the dimensions H and W, thus forming a distribution with x... T We have C*H*W normal distributions centered at symmetry, and we standardize each distribution to a standard normal distribution. In this example, the optimal time step is determined based on the evaluation results obtained from each iteration of training. For example, assuming the maximum number of noise additions is T = 1000, evaluation using symmetric distributions reveals that when the number of noise reduction iterations is 500, the difference between the pdf or cdf of the denoised sample data and the denoised sample data in the symmetric distribution is minimal. At this point, we can consider that noise reduction after 500 iterations is meaningless, thus determining the optimal time step value to be 500.
[0054] In some examples, standard Gaussian noise is added to the training image based on the current number of noise additions, T. Specifically, there can be various noise addition strategies. As an example, standard Gaussian noise can be added to the training image. Standard Gaussian noise, also known as standard normally distributed noise, is a type of random noise with a specific probability distribution. In diffusion models, adding standard Gaussian noise can effectively learn the distribution of the data and generate high-quality samples.
[0055] In practical applications, using a trained diffusion model to generate images can produce high-quality images, such as those with realistic details, diverse styles, and high resolution. In some examples, a noisy image is input; this image is fed into a trained diffusion model for predictive denoising until the maximum number of denoising iterations is reached. The output image of the diffusion model is then used as the generated image. The input noisy image can be a randomly selected image from a standard Gaussian noise distribution. In some conditional generation tasks, the randomly sampled noisy image can be combined with specific conditional information. By inputting these conditions along with the noisy image into the model, the model can perform targeted denoising and generation based on the given conditions, thus achieving more accurate and satisfactory image generation.
[0056] This embodiment provides an image generation method, which first adds noise based on the current number of noise additions T and the training image to generate sample data x. T ; Using sample data x T Construct a symmetrical distribution centered on the target; repeat iterative training until the preset number of iterations is reached. Iterative training includes: randomly selecting time step i within time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy and denoised sample data obtained in each training iteration is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result. Based on the evaluation result, the optimal time step value is determined. The optimal time step value is used as the current number of times the sample is noisy, and the process returns to generate sample data x based on the current number of times the sample is noisy T. T The process involves repeating the steps until the optimal time step value remains constant; configuring the maximum number of noise additions to the diffusion model to match the optimal time step value, thus obtaining a trained diffusion model; and finally, generating an image using the trained diffusion model. This scheme utilizes the symmetry of the distribution to quantitatively evaluate the effect of noise addition and reduction at each step, thereby determining the maximum number of noise additions. This facilitates efficient allocation of computational resources and optimization of model performance, reduces model training and application time, and ultimately improves the reliability of the image generated by the model.
[0057] Example 2
[0058] Figure 3 The diagram illustrates a schematic of an image generation device. This embodiment relates to image generation using a diffusion model, such as... Figure 1 As shown, the device includes:
[0059] Module 11 is used to add noise based on the current number of noise additions T and the training image to generate sample data x. T ; Using sample data xT Construct a symmetrical distribution centered on the target.
[0060] Training module 12 is used for repeated iterative training until a preset number of iterations is reached. Iterative training includes: randomly selecting time step i in time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy sample data and the denoised sample data obtained in each iteration of training is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result.
[0061] Processing module 13 is used to determine the optimal time step value based on the evaluation results; it uses the optimal time step value as the current number of times noise is added to the sample, and returns to execute the generation of sample data x based on the current number of times noise is added to the sample T. T The steps are repeated until the value of the optimal time step remains unchanged.
[0062] The generation module 14 is used to configure the maximum number of noise additions to the diffusion model to the value of the optimal time step, so as to obtain the trained diffusion model; and to generate an image using the trained diffusion model.
[0063] The method of adding noise is unrestricted; for example, it can be based on a noise-adding formula. As an example, a Markov chain with multiple time steps can be defined for noise addition. Noise is added to the training image at each time step. A Markov chain is a stochastic process where the state at time t+1 depends only on the state at time t and is independent of states between times t. In the diffusion model, by defining a Markov chain, the noise-adding process can be decomposed into a series of discrete steps, each depending only on the state of the previous step, where each state corresponds to data with varying degrees of added noise.
[0064] This scheme incorporates symmetric distribution considerations during model training. For different numbers of noise additions to samples, the optimal maximum number of noise additions is determined by evaluating the characteristics of the noise-added and noise-reduced samples within a symmetric distribution. Specifically, the model training process is illustrated with examples. In some examples, training module 12 is used to train the noise-added sample data x... i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The process includes: training the diffusion model to reduce noise based on the noisy sample data xi and time step i, obtaining the denoised sample data pT-i and the predicted noise; calculating the loss value of the predicted noise using a loss function based on the predicted noise and the actual noise; and updating the model parameters by performing gradient descent optimization based on the loss value.
[0065] Among them, the loss value and loss function are used to measure the difference between the samples generated by the model and the real data, and guide the training process of the model. By comparing the loss value, the parameters of the model are continuously adjusted so that the loss value gradually decreases, thereby making the generated results of the model closer and closer to the real data.
[0066] In some examples, the training module 12 is used to: obtain the probability density function or cumulative distribution function of the denoised sample data in a symmetric distribution; and obtain the probability density function or cumulative distribution function of the noisy sample data in a symmetric distribution; calculate the distribution difference between the probability density function or cumulative distribution function of the denoised sample data and the noisy sample data; and evaluate the sample data whose distribution difference exceeds a preset threshold as the evaluation result.
[0067] Specifically, the approximate equality of the probability density functions or cumulative distribution functions of the two data sets indicates that the denoised sample data is far from the distribution center. The probability density function (PDF) and cumulative distribution function (CDF) are important tools for describing probability distributions, helping to understand the distribution characteristics of sample data and guiding the training and generation process of diffusion models. To ensure the symmetry and sign of the denoised and noisy sample data, PDF and CDF can be used; Min||f(x) can be applied. i )-f(p T-i ||2 is used as an additional optimization term to ensure symmetry, while ||1-cdf(f,x) is used. i )-cdf(f,p T-i )||2 to ensure its positive or negative sign.
[0068] In some examples, training module 12 is used to: take the value of the time step corresponding to the sample data that appears most frequently in the evaluation results as the value of the optimal time step.
[0069] In this example, the preset threshold is determined through actual testing and analysis of the image generation effect. Specifically, firstly, evaluation metrics are determined; test data is collected by selecting a representative real image dataset for comparison with the generated image, and a large number of image samples, i.e., test data, are generated using a diffusion model; secondly, actual testing is conducted, namely, the subjective evaluation of the generated image by observers and the objective evaluation of the generated image by the evaluation metrics, and the value of each evaluation metric is calculated; finally, the objective evaluation values are statistically analyzed and combined with the subjective evaluation to determine the threshold.
[0070] In some examples, building block 11 is used to add standard Gaussian noise to the training image based on the current number of noise additions T for the samples. Specifically, there can be various noise addition strategies. As an example, standard Gaussian noise can be added to the training image. Standard Gaussian noise, also known as standard normally distributed noise, is a type of random noise with a specific probability distribution. In diffusion models, adding standard Gaussian noise can effectively learn the distribution of the data and generate high-quality samples.
[0071] It should be noted that this embodiment is a device embodiment corresponding to the aforementioned method embodiment, so for details of this embodiment, please refer to the relevant content of the aforementioned embodiment.
[0072] In the image generation apparatus provided in this embodiment, noise is first added based on the current number of sample noise additions T and the training image to generate sample data x. T ; Using sample data x T Construct a symmetrical distribution centered on the target; repeat iterative training until the preset number of iterations is reached. Iterative training includes: randomly selecting time step i within time steps 0-T to obtain noisy sample data x. i Based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy and denoised sample data obtained in each training iteration is calculated, and the distribution difference is evaluated based on the symmetric distribution to obtain the evaluation result. Based on the evaluation result, the optimal time step value is determined. The optimal time step value is used as the current number of times the sample is noisy, and the process returns to generate sample data x based on the current number of times the sample is noisy T. T The process involves repeating the steps until the optimal time step value remains constant; configuring the maximum number of noise additions to the diffusion model to match the optimal time step value, resulting in a trained diffusion model; and then using the trained diffusion model to generate images. This scheme utilizes the symmetry of the distribution to quantitatively evaluate the effect of noise addition and denoising at each step, thereby determining the maximum number of noise additions. This facilitates efficient allocation of computing resources and optimization of model performance, while reducing model training and application time, thus improving the reliability of the images generated by the model.
[0073] Example 3
[0074] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes:
[0075] The electronic device includes a processor 291 and a memory 292; it may also include a communication interface 293 and a bus 294. The processor 291, memory 292, and communication interface 293 can communicate with each other via the bus 294. The communication interface 293 can be used for information transmission. The processor 291 can invoke logical instructions stored in the memory 292 to execute the methods described in the example above.
[0076] Furthermore, the logical instructions in the aforementioned memory 292 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. As a computer-readable storage medium, memory 292 can be used to store software programs and computer-executable programs, such as the program instructions / modules corresponding to the methods in the embodiments of this application. Processor 291 executes functional applications and data processing by running the software programs, instructions, and modules stored in memory 292, thereby implementing the methods in the above method examples.
[0077] The memory 292 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 292 may include high-speed random access memory and may also include non-volatile memory.
[0078] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method in any of the embodiments.
[0079] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method in any of the embodiments.
[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0081] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.
[0082] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or in the form of software program modules.
[0083] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.
[0084] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0085] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. An image generation method, characterized in that, include: Noise is added based on the current number of noise additions T for the samples and the training image to generate sample data x. T ; using the sample data x T Construct a symmetrical distribution around the center; Repeated iterative training until a preset number of iterations is reached, wherein the iterative training includes: randomly selecting time step i in time steps 0-T to obtain noisy sample data x. i According to the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy sample data and the denoised sample data obtained in each iteration of training is calculated, and the distribution difference is evaluated based on the symmetrical distribution to obtain the evaluation result. Based on the evaluation results, determine the optimal time step value; use the optimal time step value as the current number of sample noise additions, and return to execute the step of generating sample data x based on the current number of sample noise additions T. T The steps are repeated until the value of the optimal time step remains unchanged; The maximum number of noise additions to the diffusion model is configured to the value of the optimal time step to obtain a trained diffusion model; an image is generated using the trained diffusion model.
2. The method according to claim 1, characterized in that, The data based on the noise-added sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i ,include: Based on the noise-added sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i and predict noise; Based on the predicted noise and the actual noise, the loss value of the predicted noise is calculated using a loss function; based on the loss value, the model parameters are updated by performing gradient descent optimization.
3. The method according to claim 1, characterized in that, The distribution difference between the noisy sample data and the denoised sample data obtained in each training iteration is calculated. The difference in the symmetric distribution is then evaluated to obtain an evaluation result, including: Obtain the probability density function or cumulative distribution function of the noise-reduced sample data in the symmetrical distribution; and obtain the probability density function or cumulative distribution function of the noise-added sample data in the symmetrical distribution; calculate the distribution difference between the probability density function or cumulative distribution function of the noise-reduced sample data and the noise-added sample data; and evaluate the sample data whose distribution difference exceeds a preset threshold as the evaluation result.
4. The method according to claim 1, characterized in that, Determining the optimal time step value based on the evaluation results includes: The value of the time step corresponding to the sample data that appears most frequently in the evaluation results will be taken as the value of the optimal time step.
5. The method according to claim 1, characterized in that, The noise addition based on the current number of noise additions T for the samples and the training image includes: Standard Gaussian noise is added to the training image based on the current number of times noise is added to the sample, T.
6. The method according to any one of claims 1-5, characterized in that, include: Input a noisy image; The noisy image is input into the trained diffusion model for prediction and noise reduction until the maximum number of noise reduction operations is reached. The image currently output by the diffusion model is then used as the generated image.
7. An image generation apparatus, characterized in that, include: The module is used to add noise to the current sample based on the number of noise additions T and the training image, generating sample data x. T ; using the sample data x T Construct a symmetrical distribution around the center; The training module is used to repeatedly iterate until a preset number of iterations is reached. The iterative training includes: randomly selecting time step i from time step 0-T to obtain noisy sample data x. i According to the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i The distribution difference between the noisy sample data and the denoised sample data obtained in each iteration of training is calculated, and the distribution difference is evaluated based on the symmetrical distribution to obtain the evaluation result. The processing module is used to determine the optimal time step value based on the evaluation result; use the optimal time step value as the current number of sample noise additions, and return to execute the step of generating sample data x based on the current number of sample noise additions T. T The steps are repeated until the value of the optimal time step remains unchanged; The generation module is used to configure the maximum number of noise additions to the diffusion model to the value of the optimal time step, thereby obtaining a trained diffusion model; and to generate an image using the trained diffusion model.
8. The apparatus according to claim 7, characterized in that, The training module is based on the noisy sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i ,include: Based on the noise-added sample data x i The diffusion model is trained for denoising at time step i to obtain denoised sample data p. T-i and predict noise; Based on the predicted noise and the actual noise, the loss value of the predicted noise is calculated using a loss function; based on the loss value, the model parameters are updated by performing gradient descent optimization.
9. The apparatus according to claim 7, characterized in that, The training module calculates the distribution difference between the noisy sample data and the denoised sample data obtained in each training iteration, and evaluates the symmetrical distribution difference based on the symmetrical distribution to obtain an evaluation result, including: Obtain the probability density function or cumulative distribution function of the noise-reduced sample data in the symmetrical distribution; and obtain the probability density function or cumulative distribution function of the noise-added sample data in the symmetrical distribution; calculate the distribution difference between the probability density function or cumulative distribution function of the noise-reduced sample data and the noise-added sample data; and evaluate the sample data whose distribution difference exceeds a preset threshold as the evaluation result.
10. The apparatus according to claim 7, characterized in that, The training module determines the optimal time step value based on the evaluation results, including: The value of the time step corresponding to the sample data that appears most frequently in the evaluation results will be taken as the value of the optimal time step.
11. The apparatus according to claim 7, characterized in that, The construction module adds noise based on the current number of noise additions T for the samples and the training image, including: Standard Gaussian noise is added to the training image based on the current number of times noise is added to the sample, T.
12. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.