Efficient and stable generative model based on noise scheduling and accelerated sampling

Abstract

The invention provides an efficient and stable generative model based on noise scheduling and accelerated sampling, and relates to the technical field of artificial intelligence generative models. According to the efficient and stable generation model based on noise scheduling and accelerated sampling, the distribution proportion between the image and the diffusion factor is controlled, and the growth rate of noise is dynamically adjusted in different stages of the diffusion step, so that the diffusion factor grows faster in the early stage and gradually tends to be stable in the middle and later stages; the influence of too fast noise accumulation on the generation effect is avoided, in addition, the quality of the generated image is optimized by fusing the statistical characteristics of the diffusion factors, and modeling is carried out through the statistical characteristics of the diffusion factors. According to the method, the Gnoise-sigmoid function is used for controlling the increasing rate of the noise in the diffusion process, and the problem of instability of traditional linear or cosine noise distribution in the generation process is solved. Through smoothing noise distribution, rapid diffusion in an early stage and stable transition in a middle and later stage, excessive accumulation of noise is reduced, and the quality of the generated image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence generation models, and specifically to an efficient and stable generation model based on noise scheduling and accelerated sampling. Background Art

[0002] In recent years, generative artificial intelligence models have achieved remarkable results in various fields, especially in the field of image generation. As one of the representative models, the variational autoencoder (VAE) is mainly used for unsupervised learning and generation tasks in deep learning. Unlike traditional autoencoders, VAE introduces the concept of variational inference and approximates the true posterior distribution by maximizing the lower bound of evidence. This enables VAE to generate diverse and continuous samples, thus achieving better results in generation tasks. The advantage of VAE lies in the diversity of its generated samples and training stability. It can effectively avoid the common mode collapse problem in Generative Adversarial Networks (GAN). However, due to its reliance on the assumption of Gaussian distribution, the samples generated by VAE may be relatively blurred, so it is not as effective as GAN in some high-resolution image generation tasks. In many application scenarios, VAE is widely used in image generation, data compression, anomaly detection and other fields.

[0003] GAN is a deep learning model proposed by Ian Goodfellow et al. in 2014. Its core idea is to generate new and realistic data by pitting two neural networks, the generator and the discriminator, against each other. The goal of the generator is to generate samples that are similar to the distribution of real data, while the discriminator tries to distinguish between generated samples and real samples. The training process of GAN is a zero-sum game in game theory: the generator continuously optimizes itself to deceive the discriminator; while the discriminator continuously improves its recognition ability to correctly distinguish between real and fake samples. This adversarial process will eventually reach a Nash equilibrium state, that is, the samples generated by the generator are indistinguishable from real data. GAN has a wide range of applications in image generation, image restoration, style transfer, data enhancement and other fields. However, GAN is prone to problems such as mode collapse and training instability during training, resulting in insufficient diversity of generated samples or training failure. To solve these problems, many variant models have emerged, such as conditional GAN, DCGAN (Deep Convolutional GAN) and WGAN (Wasserstein GAN) to improve training stability and generation quality.

[0004] Recent research results on generative models are mainly focused on denoising diffusion models. The most representative one is the Denoising Diffusion Probabilistic Models (DDPM) proposed by Ho et al. in 2020. The model designs a weighted variational bound based on the matching relationship between the diffusion probability model and the Langevin dynamics denoising score. Its training process is to gradually weight and fuse the samples and diffusion factors according to the proportional coefficient related to the time step to form a Markov chain. In addition, a noise predictor is trained to predict the diffusion factor at the current time step. The sampling process can be regarded as the inverse process of the Markov chain, starting from random noise, taking the current noise image and time step as input, and the noise predictor outputs the diffusion factor at the current time step, thereby gradually denoising the noise image into a clear image. The advantage of DDPM is that the generated image quality is very high, especially in some scenes with complex data distribution, it has strong expressiveness, and the training is stable, avoiding the mode collapse problem in GAN.

[0005] The core principle of DDPM is to gradually add noise to the data and then remove the noise step by step through the reverse process in the generation phase to restore the original data from the noise. This step-by-step generation strategy ensures the stability of the generation process and can generate highly realistic samples, especially in scenarios with complex data distribution. However, the main drawback of DDPM is its extremely low sampling efficiency.

[0006] Specifically, DDPM relies on the inverse Markov chain for multi-step denoising, and in order to satisfy the mathematical logic of the model, the diffusion step T is usually set to thousands of times. This means that T denoising operations are required to generate one sample. The computational complexity of each denoising operation is small, but tens of thousands of iterations will greatly increase the generation time, especially in the scenario of high-definition image generation. This defect is particularly obvious. In contrast, GAN only needs one forward propagation to generate samples. Through adversarial training of the generator and the discriminator, the generator only needs one calculation in the inference stage to generate high-quality samples. Therefore, in terms of generation speed, GAN is far superior to DDPM.

[0007] In addition, although the reverse sampling process of the diffusion process of DDPM can effectively ensure the quality and diversity of the generated images, it takes a lot of time to generate high-resolution images, which restricts its widespread adoption in practical applications. Therefore, although DDPM has obvious advantages in generation quality and stability, especially in complex data distribution and high-resolution scenes, its disadvantage in generation speed makes it face greater challenges in practical applications, especially compared with efficient generation models such as GAN. Summary of the invention

[0008] 1. Technical issues to be resolved

[0009] In view of the deficiencies in the prior art, the present invention provides an efficient and stable generation model based on noise scheduling and accelerated sampling, which solves the defects and deficiencies in the prior art.

[0010] (II) Technical solution

[0011] To achieve the above objectives, the present invention is implemented through the following technical solutions: an efficient and stable generation model based on noise scheduling and accelerated sampling, including the following specific contents:

[0012] 1) In DDPM, the optimization problem is transformed from minimizing the KL divergence between the generated distribution and the true distribution to maximizing the log-likelihood:

[0013] θ * =argminKL[p θ (x0)||q(x0)]=argmaxlogp θ (x)

[0014] where θ * When an image is generated by a network model with θ as a parameter, its data distribution p θ (x0) is the optimal solution of network parameters with the highest degree of fit to the real image data distribution q(x), and the joint distribution p θ (x 0:T ) is defined as the inverse denoising process, starting from p(x T )=N(x T ; 0,I):

[0015]

[0016] p θ (x0)=∫p θ (x 0:T )dx 1:T

[0017] Similar to other latent variable models, the optimal parameter θ of the DDPM network model is solved by maximizing the variational lower bound of the Jensen inequality:

[0018]

[0019] 2) q(x1,…x T |x0) is defined as the forward process. Given the initial real image data x0, Gaussian noise is gradually added to it to obtain x1, x2...x T After enough forward diffusion processes T times, the real image data will be submerged by Gaussian noise. TIt is considered to be an isotropically independent Gaussian noise image, where:

[0020]

[0021] Among them, β t is a set of linear hyperparameters related to the number of time steps. Considering that the noise added in each diffusion process follows a standard Gaussian distribution, the forward process has the following properties:

[0022]

[0023] At this time, x t It can be regarded as the combination of the real image data x0 and the standard Gaussian noise according to the weight coefficient related to the time step t obtained by fusion;

[0024] 3) The denoising process is from x T The process of gradually recovering to x0 can be found by transforming the variational lower bound in the formula;

[0025]

[0026] The sampling process is achieved by fitting the posterior conditional probability of the diffusion process:

[0027]

[0028] Among them, ε t is the x obtained by diffusion at time t t The diffusion factor contained in , and then, the optimization goal of DDPM is converted to training a neural network with a parameter θ as a noise predictor, and then denoising the original image data x0 through T iterations.

[0029] Preferably, a Gnoise-sigmoid function is designed to process the growth of noise more smoothly, and the formula is as follows:

[0030]

[0031] During the diffusion process, the distribution ratio between the image and the diffusion factor is controlled. Specifically, at different stages of the diffusion step, the growth rate of the noise is dynamically adjusted so that the diffusion factor grows faster in the early stage and gradually stabilizes in the middle and late stages, thereby avoiding the influence of too fast noise accumulation on the generation effect. In addition, the quality of the generated image is optimized by integrating the statistical characteristics of the diffusion factor. By modeling the statistical characteristics of the diffusion factor, the noise distribution in the diffusion and denoising process can be better regulated, and the noise superposition phenomenon in high-dimensional space can be reduced, thereby improving the denoising accuracy.

[0032] In addition, in image processing tasks, image pixel values ​​are usually normalized to the range of [-1, 1]. The original pixel value (0 to 255) is mapped to [-1, 1]. The normalization formula is as follows:

[0033]

[0034] Under the above normalization method, the normalized difference corresponding to the difference of each pixel value is: 1 / 127.5, and the step size of each increment after normalization is 1 / 127.5. Since the noise at the beginning of the diffusion process is too subtle, it will make it difficult for the noise prediction network to accurately predict ε, so it is necessary to introduce an offset s so that The offset is designed to be 0.005, which is smaller than 1 / 127.5 of the pixel value. In addition, the statistical characteristics of the diffusion factor in each time step are integrated into the diffusion weight coefficient. In order to help the model better capture the details of the diffusion factor data distribution and the discrete characteristics of the diffusion process during the training process, at the same time, increase the diversity and randomness of noise prediction, and make the samples obtained in the denoising process more diverse.

[0035] Preferably, reanalyze the formula to maximize ELBO, the denoising process p θ (x t-1 |x t )The distribution to be fitted is q(x t-1 |x t ,x0), q(x t |x t-1 ,x0)=q(x t |x t-1 ) is due to the Markov property of the diffusion process. Step sampling is achieved by de-Markovizing the diffusion process. At this time, the statistical distribution of the denoising process is p(x s |x t ), the diffusion process distribution to be fitted is:

[0036]

[0037] Where s<t,q(x s |x t ),q(x t |x0) still has the Markov property of the diffusion process, that is:

[0038]

[0039] At this time, the evidence lower bound formula is:

[0040]

[0041] According to the formula q(x t|x s ,x0)=q(x t |x s ,x0)·q(x s |x0) / q(x t |x0) converts the problem of solving the distribution to be fitted into solving the distribution q(x t |x s ,x0), considering the denoising process in DDPM, it satisfies:

[0042] p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),Σ θ (x t ,t))

[0043]

[0044] The statistical distribution of the synthesis step denoising process at time s is defined as:

[0045] p θ (x s |x t )=N(x s ;μ θ (x0,x t ,t),Σ θ (x0,x t ,t))

[0046] According to the denoising process of DDPM, p θ (x s |x t ) is x0,x t The linear combination of and ε, and then according to the above analysis, from x t Sampling to get x s :

[0047]

[0048] By giving different k values, different sampling processes will be obtained, and different s values ​​will obtain different sampling effects. In the estimation process and the derivation of the sampling formula, the diffusion process based on Markov properties is still used, and there is no need to retrain the model.

[0049] Preferably, a network model based on the U-net architecture is designed as a noise predictor in the image generation task, named U-attention. The design of this model combines temporal embedding, residual module, upsampling, downsampling and self-attention module to achieve accurate prediction of the diffusion factor in the input noisy image.

[0050] Preferably, the high-resolution features of the input image are gradually reduced in dimension using classic convolution operations, and multi-scale features are extracted. A residual module is introduced after each convolution layer of the encoder part to enhance the modeling capability of complex details in the image. The residual module retains the original features of the input and output through jump connections, ensuring that important information is not lost in the deep network, while alleviating the gradient vanishing problem.

[0051] Preferably, the core of U-Attention is the self-attention mechanism, which dynamically adjusts the feature weights of different positions to ensure that the network effectively captures the dependencies between the global context in the image. Under this mechanism, the model calculates the correlation of each position of the input feature map and learns the relationship between different regions in the image by generating query, key and value matrices.

[0052] Preferably, the U-Attention network also introduces a time embedding layer to fuse the time information in the diffusion process. The generation process of the diffusion model is an inverse diffusion process with a time step of T. The addition and removal of noise depends on the time step. Time embedding is added to the network, and the time step information is encoded into a vector and fused with the input feature map to guide the network to make the best noise prediction at different time steps. Time embedding converts the time step information into an embedding vector consistent with the dimension of the feature map through a fully connected layer. The vector is then multiplied element-by-element with the output of the convolutional layer to ensure that the network can adaptively adjust the prediction of noise according to the current diffusion stage.

[0053] Preferably, the decoder is responsible for gradually upsampling the multi-scale features extracted by the encoder and restoring them to an image of the same size as the input. During the decoding process, U-Attention directly passes the features captured in the encoding stage to the decoder by introducing a residual module and jump connection that are symmetrical to the encoder, thereby avoiding the loss of detail information during the upsampling process. At the same time, the self-attention mechanism strengthens the weights of important feature positions during the decoding process to ensure that the generated image details are richer and more accurate.

[0054] (III) Beneficial effects

[0055] The present invention provides an efficient and stable generation model based on noise scheduling and accelerated sampling. It has the following beneficial effects:

[0056] 1. In this invention, the Gnoise-sigmoid function is used to control the growth rate of noise during the diffusion process, solving the instability problem of traditional linear or cosine noise distribution during the generation process. By smoothing the noise distribution, rapid diffusion in the early stage and smooth transition in the middle and late stages, excessive noise accumulation is reduced and the quality of the generated image is improved.

[0057] 2. The present invention divides the sampling process into multiple stages, dynamically adjusts the number of sampling steps, and reduces redundant noise reduction operations. At the same time, multi-scale noise reduction is applied at different stages of generation to reduce the number of sampling times, thereby ensuring the quality of the generated image while greatly improving the generation speed. Compared with traditional step-by-step reverse sampling, this solution greatly reduces the time cost.

[0058] 3. The present invention introduces a self-attention mechanism in the encoding and decoding process of U-net, so that the network can capture long-distance dependencies in the image, thereby reconstructing image details more accurately. At the same time, the time embedding layer combines the information of the time step in the diffusion process to dynamically adjust the noise prediction, so that the network can make adaptive adjustments according to the noise distribution at each diffusion stage, improving the prediction accuracy and generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is the directed graph model diagram of DDPM of the present invention;

[0060] Figure 2 This is the architecture diagram of the U-Attention network model of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Example:

[0063] like Figure 1-Figure 2 As shown, the embodiment of the present invention provides an efficient and stable generation model based on noise scheduling and accelerated sampling, including the following specific contents:

[0064] 1) In DDPM, the optimization problem is transformed from minimizing the KL divergence between the generated distribution and the true distribution to maximizing the log-likelihood:

[0065] θ * =argminKL[p θ (x0)||q(x0)]=argmaxlogpθ (x)

[0066] where θ * When an image is generated by a network model with θ as a parameter, its data distribution p θ (x0) is the optimal solution of network parameters with the highest degree of fit to the real image data distribution q(x), and the joint distribution p θ (x 0:T ) is defined as the inverse denoising process, starting from p(x T )=N(x T ; 0,I):

[0067]

[0068] p θ (x0)=∫p θ (x 0:T )dx 1:T

[0069] Similar to other latent variable models (such as VAE), the optimal parameter θ of the DDPM network model is solved by maximizing the variational lower bound of the Jensen inequality:

[0070]

[0071] 2) q(x1,…x T |x0) is defined as the forward process. Given the initial real image data x0, Gaussian noise is gradually added to it to obtain x1, x2...x T , the process obviously satisfies the Markov property. After enough T forward diffusion processes, the real image data will be submerged by Gaussian noise. At this time, x T It is considered to be an isotropically independent Gaussian noise image, where:

[0072]

[0073] Among them, β t is a set of linear hyperparameters related to the number of time steps. Considering that the noise added in each diffusion process follows a standard Gaussian distribution, the forward process has the following properties:

[0074]

[0075] At this time, x t It can be regarded as the combination of the real image data x0 and the standard Gaussian noise according to the weight coefficient related to the time step t obtained by fusion;

[0076] 3) The denoising process is from x TThe process of gradually recovering to x0 still belongs to the Markov chain, which can be found by transforming the variational lower bound in the formula;

[0077]

[0078] The sampling process is achieved by fitting the posterior conditional probability of the diffusion process:

[0079]

[0080] Among them, ε t is the x obtained by diffusion at time t t The diffusion factor contained in , and then, the optimization goal of DDPM is converted to training a neural network with a parameter θ as a noise predictor, and then denoising the original image data x0 through T iterations.

[0081] First, the present invention proposes a noise diffusion scheme based on Gnoise-sigmoid to optimize the denoising process of DDPM. In the traditional diffusion model, the increase of noise and the allocation of diffusion factors are performed through linear or cosine functions, and these methods may cause the noise to change too fast or too slow at certain stages of the diffusion process, affecting the generation quality. In order to solve this problem, the present invention designs the Gnoise-sigmoid function to process the growth of noise more smoothly, and the formula is as follows:

[0082]

[0083] During the diffusion process, the distribution ratio between the image and the diffusion factor is controlled. Specifically, at different stages of the diffusion step, the growth rate of the noise is dynamically adjusted so that the diffusion factor grows faster in the early stage and gradually stabilizes in the middle and late stages, thereby avoiding the influence of too fast noise accumulation on the generation effect. In addition, the quality of the generated image is optimized by integrating the statistical characteristics of the diffusion factor. By modeling the statistical characteristics of the diffusion factor, the noise distribution in the diffusion and denoising process can be better regulated, and the noise superposition phenomenon in high-dimensional space can be reduced, thereby improving the denoising accuracy.

[0084] In addition, in image processing tasks, image pixel values ​​are usually normalized to the range of [-1, 1]. The original pixel value (0 to 255) is mapped to [-1, 1]. The normalization formula is as follows:

[0085]

[0086] Under the above normalization method, the normalized difference corresponding to the difference of each pixel value is: 1 / 127.5, which means that the step size of each increment after normalization is 1 / 127.5. Since the noise at the beginning of the diffusion process is too subtle, it will make it difficult for the noise prediction network to accurately predict ε, so it is necessary to introduce an offset s so that Less than 1 / 127.5 of the pixel value. In the present invention, the offset size is designed to be 0.005. In addition, the statistical characteristics of the diffusion factor in each time step are integrated into the diffusion weight coefficient. In order to help the model better capture the details of the diffusion factor data distribution and the discrete characteristics of the diffusion process during the training process, thereby improving the training stability and generation quality of the model, and at the same time, increasing the diversity and randomness of noise prediction, making the samples obtained in the denoising process more diverse.

[0087] Reanalyze the maximization of ELBO in the formula, the denoising process p θ (x t-1 |x t )The distribution to be fitted is q(x t-1 |x t ,x0), q(x t |x t-1 ,x0)=q(x t |x t-1 ) is due to the Markov property of the diffusion process. Step sampling is achieved by de-Markovizing the diffusion process. At this time, the statistical distribution of the denoising process is p(x s |x t ), the diffusion process distribution to be fitted is:

[0088]

[0089] Where s<t,q(x s |x t ),q(x t |x0) still has the Markov property of the diffusion process, that is:

[0090]

[0091] At this time, the evidence lower bound formula is:

[0092]

[0093] According to the formula q(x t |x s ,x0)=q(x t |x s ,x0)·q(x s |x0) / q(x t|x0) converts the problem of solving the distribution to be fitted into solving the distribution q(x t |x s ,x0), considering the denoising process in DDPM, it satisfies:

[0094] p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),Σ θ (x t ,t))

[0095]

[0096] The statistical distribution of the synthesis step denoising process at time s is defined as:

[0097] p θ (x s |x t )=N(x s ;μ θ (x0,x t ,t),Σ θ (x0,x t ,t))

[0098] According to the denoising process of DDPM, p θ (x s |x t ) is x0,x t The linear combination of and ε, and then according to the above analysis, from x t Sampling to get x s :

[0099]

[0100] By giving different k values, different sampling processes will be obtained, and different s values ​​will obtain different sampling effects. In the estimation process and the derivation of the sampling formula, the diffusion process based on Markov properties is still used, and there is no need to retrain the model.

[0101] The present invention designs a network model based on the U-net architecture as a noise predictor in the image generation task, named U-attention. The design of the model combines time embedding, residual module, upsampling, downsampling and self-attention module to achieve accurate prediction of the diffusion factor in the input noise image; the high-resolution features of the input image are gradually reduced in dimension using the classic convolution operation, and multi-scale features are extracted. A residual module is introduced after each convolution layer of the encoder part to enhance the modeling ability of complex details in the image. The residual module retains the original features of the input and output through jump connections, ensuring that important information is not lost in the deep network, and at the same time alleviating the gradient disappearance problem; the core of U-Attention is the self-attention mechanism, which dynamically adjusts the feature weights of different positions to ensure that the network effectively captures the dependencies between the global contexts in the image. Under this mechanism, the model calculates the correlation of each position of the input feature map, and learns the relationship between different regions in the image by generating query, key and value matrices; the U-Attention network also introduces time embedding The generation process of the diffusion model is an inverse diffusion process of time step T. The addition and removal of noise depends on the time step. Therefore, the present invention adds time embedding to the network, encodes the time step information into a vector, and fuses it with the input feature map to guide the network to make the best noise prediction at different time steps. The time embedding converts the time step information into an embedding vector consistent with the dimension of the feature map through the fully connected layer. The vector is then multiplied element by element with the output of the convolution layer to ensure that the network can adaptively adjust the prediction of noise according to the current diffusion stage. The decoder part is responsible for gradually upsampling the multi-scale features extracted by the encoder and restoring them to an image of the same size as the input. During the decoding process, U-Attention directly passes the features captured in the encoding stage to the decoder by introducing a residual module and jump connection symmetrical to the encoder, avoiding the loss of detail information during the upsampling process. At the same time, the self-attention mechanism strengthens the weights of important feature positions during the decoding process to ensure that the generated image details are richer and more accurate.

[0102] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprising a reference structure" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0103] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An efficient and stable generation model based on noise scheduling and accelerated sampling, characterized by: The specific contents include: 1) In DDPM, the optimization problem is transformed from minimizing the KL divergence between the generated distribution and the true distribution to maximizing the log-likelihood: θ * argminKL[p θ (x0)||q(x0)]6argmaxlogp θ (x) where θ * When an image is generated by a network model with θ as a parameter, its data distribution p θ (x0) is the optimal solution of network parameters with the highest degree of fit to the real image data distribution q(x), and the joint distribution p θ (x 0:T ) is defined as the inverse denoising process, starting from p(x T )=N(x T ; 0,I): Similar to other latent variable models, the optimal parameter θ of the DDPM network model is solved by maximizing the variational lower bound of the Jensen inequality: 2) q(x1,...x in the ELBO formula T |x0) is defined as the forward process. Given the initial real image data x0, Gaussian noise is gradually added to it to obtain x1, x2...x T After enough forward diffusion processes T times, the real image data will be submerged by Gaussian noise. T It is considered to be an isotropically independent Gaussian noise image, where: Among them, β t is a set of linear hyperparameters related to the number of time steps. Considering that the noise added in each diffusion process follows a standard Gaussian distribution, the forward process has the following properties: At this time, x t It can be regarded as the combination of the real image data x0 and the standard Gaussian noise according to the weight coefficient related to the time step t obtained by fusion; 3) The denoising process is from x T The process of gradually recovering to x0 can be found by transforming the variational lower bound in the formula; The sampling process is achieved by fitting the posterior conditional probability of the diffusion process: Among them, ε t is the x obtained by diffusion at time t t The diffusion factor contained in , and then, the optimization goal of DDPM is converted to training a neural network with a parameter θ as a noise predictor, and then denoising the original image data x0 through T iterations.

2. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 1, characterized in that: The Gnoise-sigmoid function is designed to process the growth of noise more smoothly. The formula is as follows: During the diffusion process, the distribution ratio between the image and the diffusion factor is controlled. At different stages of the diffusion step, the growth rate of the noise is dynamically adjusted, so that the diffusion factor grows faster in the early stage and gradually stabilizes in the middle and late stages, avoiding the influence of too fast noise accumulation on the generation effect. In addition, the quality of the generated image is optimized by integrating the statistical characteristics of the diffusion factor. By modeling the statistical characteristics of the diffusion factor, the noise distribution in the diffusion and denoising process can be better controlled to reduce the noise superposition phenomenon in high-dimensional space. In addition, in image processing tasks, image pixel values ​​are usually normalized to the range of [-1, 1]. The original pixel value (0 to 255) is mapped to [-1, 1]. The normalization formula is as follows: Under the above normalization method, the normalized difference corresponding to the difference of each pixel value is: 1 / 127.5, and the step size of each increment after normalization is 1 / 127.

5. Since the noise at the beginning of the diffusion process is too subtle, it will make it difficult for the noise prediction network to accurately predict ε, so it is necessary to introduce an offset s so that β0 1 / 2 The offset is designed to be 0.005, which is smaller than 1 / 127.5 of the pixel value. In addition, the statistical characteristics of the diffusion factor in each time step are integrated into the diffusion weight coefficient. In order to help the model better capture the details of the diffusion factor data distribution and the discrete characteristics of the diffusion process during the training process, at the same time, increase the diversity and randomness of noise prediction, and make the samples obtained in the denoising process more diverse.

3. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 2, characterized in that: Reanalyze the maximization of ELBO in the formula, the denoising process p θ (x t-1 |x t )The distribution to be fitted is q(x t-1 |x t ,x0), q(x t |x t-1 ,x0)=q(x t |x t-1 ) is due to the Markov property of the diffusion process. Step sampling is achieved by de-Markovizing the diffusion process. At this time, the statistical distribution of the denoising process is p(x s |x t ), the diffusion process distribution to be fitted is: Where s<t,q(x s |x t ),q(x t |x0) still has the Markov property of the diffusion process, that is: At this time, the evidence lower bound formula is: According to the formula q(x t |x s ,x0)=q(x t |x s ,x0)·q(x s |x0) / q(x t |x0) converts the problem of solving the distribution to be fitted into solving the distribution q(x t |x s ,x0), considering the denoising process in DDPM, it satisfies: p θ (x t-1 |x t )=N(x t-1 ;μ θ (x t ,t),Σ θ (x t ,t)) The statistical distribution of the synthesis step denoising process at time s is defined as: p θ (x) s |x t )=N(x s ;μ θ (x0,x t ,t),Σ θ (x0,x t ,t)) According to the denoising process of DDPM, p θ (x s |x t ) is x0,x t The linear combination of and ε, and then according to the above analysis, from x t Sampling to get x s : By giving different k values, different sampling processes will be obtained, and different s values ​​will obtain different sampling effects. In the estimation process and the derivation of the sampling formula, the diffusion process based on Markov properties is still used, and there is no need to retrain the model.

4. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 3, characterized in that: A network model based on the U-net architecture is designed as a noise predictor in the image generation task, named U-attention. The design of this model combines temporal embedding, residual module, upsampling, downsampling and self-attention modules to achieve accurate prediction of the diffusion factor in the input noisy image.

5. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 4, characterized in that: The classic convolution operation is used to gradually reduce the dimension of the high-resolution features of the input image and extract multi-scale features. A residual module is introduced after each convolution layer of the encoder to enhance the modeling ability of complex details in the image. The residual module retains the original features of the input and output through jump connections, ensuring that important information is not lost in the deep network and alleviating the gradient vanishing problem.

6. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 5, characterized in that: The core of U-Attention is the self-attention mechanism, which dynamically adjusts the feature weights at different positions to ensure that the network effectively captures the dependencies between the global context in the image. Under this mechanism, the model calculates the correlation between each position of the input feature map and learns the relationship between different areas in the image by generating query, key and value matrices.

7. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 6, characterized in that: The U-Attention network also introduces a time embedding layer to fuse the time information in the diffusion process. The generation process of the diffusion model is an inverse diffusion process with a time step of T. The addition and removal of noise depends on the time step. Time embedding is added to the network to encode the time step information into a vector and fuse it with the input feature map to guide the network to make the best noise prediction at different time steps. Time embedding converts the time step information into an embedding vector consistent with the dimension of the feature map through a fully connected layer. The vector is then multiplied element-by-element with the output of the convolutional layer to ensure that the network can adaptively adjust the prediction of noise according to the current diffusion stage.

8. The efficient and stable generation model based on noise scheduling and accelerated sampling according to claim 7, characterized in that: The decoder is responsible for gradually upsampling the multi-scale features extracted by the encoder and restoring them to an image of the same size as the input. During the decoding process, U-Attention directly passes the features captured in the encoding stage to the decoder by introducing a residual module and jump connection that are symmetrical to the encoder, thus avoiding the loss of detail information during the upsampling process. At the same time, the self-attention mechanism strengthens the weights of important feature positions during the decoding process to ensure that the generated image details are richer and more accurate.

Images