Image feature analysis and generation method based on fast denoising diffusion probability model

By introducing a conditional generator and discriminator into the denoising diffusion probability model and optimizing the generation process using U-net and Transformer structures, the problems of low generation quality and efficiency are solved, achieving efficient and high-quality image generation and feature analysis.

CN115908187BActive Publication Date: 2026-01-27BEIHANG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211560705.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-07
Publication Date
2026-01-27
Estimated Expiration
2042-12-07

AI Technical Summary

Technical Problem

Existing deep generative models suffer from limitations in the quality of implicit generative models and low sampling efficiency in denoising diffusion probability models.

Method used

Based on the original denoising diffusion probability model, a conditional generator and a corresponding discriminator are introduced. The supplementary conditional generator is trained through generative adversarial methods to construct a fast denoising diffusion probability model. Image features are extracted using U-net and Transformer neural network structures to optimize the generation process.

Benefits of technology

It achieves high-quality and high-efficiency image generation, has the ability to analyze the features of noisy images, and can control the generated features by adjusting the noise features.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115908187B_ABST
    Figure CN115908187B_ABST
Patent Text Reader

Abstract

The application discloses an image feature analysis and generation method based on a fast denoising diffusion probability model, comprising the following steps: pre-processing an image generation data set to obtain a training data set; constructing an original denoising diffusion probability model, wherein a score predictor is contained, and a conditional generator and a corresponding discriminator are introduced to obtain a fast denoising diffusion probability model; training the score predictor through a score matching method, freezing the trained score predictor, and training the conditional generator and the discriminator by using an alternating training method of a generative adversarial network; updating the trained score predictor and the trained conditional generator to the fast denoising diffusion probability model, inputting a Gaussian white noise picture into the fast denoising diffusion probability model, and obtaining a real high-quality output image. The method can guarantee high-quality generation capability and significantly improve the efficiency of generation sampling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of image feature analysis and generation in computer vision, and relates to an image feature analysis and generation method based on a fast denoising diffusion probability model. Background Technology

[0002] With the support of modern deep learning, the field of computer vision has achieved remarkable results. With the advent of convolutional neural networks and the invention of residual network structures, deep learning has far surpassed traditional methods in tasks such as image classification, detection, and segmentation at various scales. In image generation, deep generative models such as generative adversarial networks and variational autoencoders can generate high-resolution images. However, their generation performance is still limited by the overly simplistic implicit generative models. Recently, denoising diffusion probability models have been able to produce higher-quality samples, but they require constructing a Markov generation process of nearly 1000 steps, thus incurring high time and computational costs. Summary of the Invention

[0003] The purpose of this invention is to overcome the limitations of implicit generation models in existing deep generative models, which restrict the quality of image generation, and the low sampling efficiency of denoising diffusion probability models. This invention provides an image feature analysis and generation method based on a fast denoising diffusion probability model. It introduces a conditional generator and a corresponding discriminator model into the original denoising diffusion probability model, and trains the supplementary conditional generator using a generative adversarial approach, enabling the joint model to achieve high-quality and high-efficiency image generation.

[0004] The technical solution adopted by this invention to solve the technical problem is as follows:

[0005] An image feature analysis and generation method based on a fast denoising diffusion probability model includes the following steps:

[0006] S1: Preprocess the image generation dataset to obtain the training dataset;

[0007] S2: Construct the original denoising diffusion probability model, which includes a score predictor, and introduce a condition generator and a corresponding discriminator to obtain a fast denoising diffusion probability model;

[0008] S3: Train the score predictor using the score matching method, freeze the trained score predictor, and train the condition generator and discriminator using the alternating training method of generative adversarial training.

[0009] S4: Update the trained score predictor and the trained condition generator to the fast denoising diffusion probability model. Input the Gaussian white noise image into the fast denoising diffusion probability model to obtain a real, high-quality output image.

[0010] Further, in step S2, the original denoising diffusion probability model includes a forward Gaussian diffusion kernel and a reverse Gaussian denoising kernel, wherein the forward Gaussian diffusion kernel is... Used to implement the forward diffusion process for adding noise to an image x0...x t ...x T ,in Represents a Gaussian distribution, β t Given a pre-defined noise scale, times t = 1, ..., T, the starting point x0 of the diffusion process is a noise-free image, and the intermediate state x... t The image contains noise at time t, and the endpoint x is the process endpoint. T The image is a Gaussian white noise image; the inverse Gaussian denoising kernel p θ (x t-1 |x t ) is used to implement the reverse denoising generation process x T ...x t ...x0, specifically

[0011]

[0012]

[0013] In the formula, σ t I is the standard deviation of the Gaussian denoising kernel; I is the identity matrix; f θ (x t ,t) represents denoising prediction; ∈ θ (x t ,t) is the fractional predictor; μ θ (x t ,t) is the mean.

[0014] Further, in step S2, the condition generator is... Introducing this into the inverse denoising generation process yields the inverse Gaussian denoising kernel after incorporating the conditional generator:

[0015]

[0016]

[0017] In the formula, μ θ,ω (x t ,t) is the mean after introducing the condition generator, and u represents the Gaussian noise obtained from sampling.

[0018] Furthermore, the score predictor ∈ θThe implementation is based on the U-net neural network structure, which is a symmetrical encoder-decoder structure. Jump connections are used in the corresponding stages of the encoder and decoder for feature information sharing. The encoder part of U-net uses cascaded ResBlocks to process the input noisy image x. t Dimensionality reduction and multi-scale visual features are extracted, while the cascaded ResBlock in the decoder part outputs a noisy image x based on the multi-scale visual features. t The noise part in the network is eliminated; the time t is encoded into a 256-dimensional vector using the temporal embedding method of the Transformer neural network structure, and then input into each ResBlock of the U-net network, so that the U-net can explicitly depend on time t.

[0019] Condition generator G ω The implementation is based on the U-net neural network structure used by the score predictor, which is a symmetrical encoder-decoder structure. An additional three-layer fully connected network is used to map the noise u into a 256-dimensional vector, which is then input into each ResBlock of the U-net, so that the U-net can also explicitly depend on the noise u. The encoder side extracts multi-scale visual features for denoising prediction, and the decoder side outputs the optimized denoising prediction based on the multi-scale features.

[0020] Discriminator D ψ The implementation is based on the U-net neural network structure, using a time-dependent ResBlock, with the input being the noisy image x at the current time t. t And the image x at the next moment t-1 t-1 The output part obtains real-valued discrimination scores through convolutional neural layers.

[0021] Furthermore, step S3, training a score predictor using a score matching method, includes the following sub-steps:

[0022] (1) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time t~U(t). Sample a batch of Gaussian noise from a standard Gaussian distribution with the same dimension as the images, denoted as...

[0023] (2) Substitute the variables t, x0, ∈ obtained above into the unweighted score matching squared error function, and calculate the mean of the function as the objective function L(θ) for this optimization.

[0024] (3) Optimize the neural network parameters θ by backpropagating the objective function L(θ);

[0025] (4) Repeat steps 1)-3) until the objective function converges or the maximum number of optimizations is reached.

[0026] 6. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 5, characterized in that the score matching objective function is:

[0027]

[0028] In the formula, This indicates that the mean of a function is calculated based on the variables.

[0029] Furthermore, step S3, which trains the condition generator and discriminator using an alternating training method for generative adversarial networks, includes the following sub-steps:

[0030] (1) The trained score matcher ∈ θ Freezing serves as the basis for subsequent processes;

[0031] (2) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time t~U(t). Based on x0, follow the Gaussian conditional distribution q(x0). t-1 A batch of x samples is taken from |x0) t-1 According to x t-1 Forward Gaussian diffusion of kernel q(x) t |x t-1 A batch of samples x from ) t According to x t The inverse Gaussian denoising kernel p after introducing a condition generator θ,ω (x t-1 |x t The predicted x is obtained by sampling in ) t-1 ;

[0032] (3) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute this into the discriminator parameter ψ function of the generative adversarial objective function, and calculate the mean of this function as the optimization objective of the discriminator;

[0033] (4) Optimize the discriminator parameter ψ by backpropagating the objective function about the discriminator parameter ψ;

[0034] (5) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute the value of the function with respect to the condition generator parameter ω into the adversarial objective function, and calculate the mean of the function as the optimization objective of the condition generator;

[0035] (6) Optimize the condition generator parameter ω once by backpropagating the objective function about the condition generator parameter ω;

[0036] (7) Repeat steps (2)–(6) until the generated adversarial objective function converges or the maximum number of optimizations is reached.

[0037] Furthermore, the generative adversarial objective function for the discriminator parameter ψ and the condition generator parameter ω is:

[0038]

[0039]

[0040] The advantages of this invention over the prior art are:

[0041] (1) This patent is based on a denoising diffusion probability model, which has a stronger image generation capability than commonly used generative models such as generative adversarial networks and variational autoencoders.

[0042] (2) This patent addresses the problem of long sampling time and low efficiency of the denoising diffusion probability model when generating images. It introduces a condition generator as an improvement, which makes the proposed fast denoising diffusion probability model retain the high-quality generation capability of the denoising diffusion probability model while greatly improving the generation sampling efficiency.

[0043] (3) The fast denoising diffusion probability model proposed in this patent has the ability to analyze the features of noisy images during the image generation process, and can also control the generated features by adjusting the noise features. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of generative adversarial training for a fast denoising diffusion probability model;

[0045] Figure 2 It uses a fast denoising diffusion probability model to randomly generate CIFAR10 images. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0047] This invention provides an image feature analysis and generation method based on a fast denoising diffusion probability model, comprising the following steps:

[0048] Step 1: Data preprocessing;

[0049] Using a deep learning framework, the image generation dataset undergoes preprocessing operations such as resizing, normalization, and horizontal flipping to obtain a training dataset suitable for training neural network models. The image generation dataset is selected from self-collected datasets or publicly available image datasets, such as CIFAR10 and CelebA. Deep learning toolkits are used to scale the dataset to 32×32, 64×64, or 128×128 resolution. The dataset data is then normalized to [-1,1], and finally, each sample in the dataset is randomly horizontally flipped as image preprocessing.

[0050] Specifically, in this embodiment, the publicly available dataset CIFAR10 is selected as an example. The images in CIFAR10 are scaled down to a resolution of 32×32, and each pixel is normalized to [-1,1]. Random horizontal flipping preprocessing is performed to obtain a training dataset suitable for training a neural network model.

[0051] Step 2: Theoretical construction of the fast denoising diffusion probability model;

[0052] In theory, the original denoising diffusion probability model constructs a trainable stochastic process that learns how to progressively denoise the data image into a data sample. Specifically, the original model first constructs a forward diffusion process of length T that continuously adds noise to the data image x0...x t ...x T The starting point of the diffusion process, x0, is a noise-free image in the dataset, and the intermediate state is x... t Let x be the random variable of the noisy image after adding noise, representing the endpoint of the diffusion process. T Approximately Gaussian white noise. The forward Gaussian diffusion nucleus constituting this forward process is... in Represents a Gaussian distribution, β t Let t = 1, ..., T be a predefined noise scale. The forward Gaussian diffusion kernel describes the property of continuously adding noise to the forward process. From the Gaussian properties of the forward process, the conditional posterior distribution can be directly obtained. in This means we can obtain noisy samples at time t through direct sampling, such as... The Gaussian noise ∈ can be sampled. Next, the original denoising diffusion probability model constructs a denoising generation process x along the reverse time path. T ...x t ...x0. The denoising process consists of a trainable inverse denoising kernel x. t-1 ~p θ (x t-1 |x t It consists of a noise reduction kernel p θ (x t-1 |x tIt has the following Gaussian form

[0053]

[0054]

[0055] In the formula, the mean μ θ (x t (t) represents the current state x. t and denoising prediction f θ (x t A linear combination of ,t), while the denoising prediction can be made by a fractional predictor ∈ θ (x t ,t) means, such as σ in the formula t The standard deviation of the Gaussian denoising kernel is typically set to β. t In the denoising kernel, the score predictor ∈ θ (x t ,t) By analyzing noisy images x t Image features to predict noisy variable x t Contains noise ∈ t This leads to f θ (x t ,t) can predict the denoised sample x0.

[0056] The original denoising diffusion probability model typically requires 1000 steps of denoising generation to obtain high-quality samples. Reducing this process to 4 steps results in blurry, low-quality samples. We found that this is because, in a 4-step denoising process, the score predictor ∈ θ (x t ,t) cannot predict noisy variables x very well. t Contains noise ∈ t This makes the denoising prediction f θ (x t The poor quality of the denoised sample x0 makes it impossible to obtain a high-quality denoised sample x0 using only a 4-step denoising process. Therefore, this invention will denoise the prediction f θ (x t ,t) Input condition generator (where u is the sampled Gaussian noise with a ratio variable x) t Lower dimensions (usually chosen as 100 dimensions) are used in denoising prediction f. θ (x t Based on the image features contained in ,t), a higher quality and more realistic denoising prediction is obtained, which in turn predicts the mean more accurately. This corresponds to the following inverse Gaussian denoising kernel after introducing a generator:

[0057]

[0058]

[0059] In the formula, μ θ,ω (x t (t) represents the mean after introducing the condition generator. Theoretically, the predicted mean is μ higher than the original mean. θ (x t ,t) is more accurate.

[0060] Step 3: Implement a fast denoising diffusion probability model using a deep learning framework;

[0061] The original denoised diffusion probability model utilizes a fractional predictor ∈ θ To achieve a trainable generative denoising process, the fast denoising diffusion probability model proposed in this invention introduces a new conditional generator G. ω A trainable, fast generative denoising process is constructed, and a corresponding discriminator D is introduced. ψ To train the condition generator G ω In implementation, this invention uses a neural network to represent the score predictor ε. θ Condition generator G ω Discriminator D ψ Based on the resolution or scale of the dataset, construct the corresponding neural network module.

[0062] Specifically, the score predictor ∈ θ The implementation is based on the U-net neural network structure, which is a symmetrical encoder-decoder structure. Jump connections are used in the corresponding stages of the encoder and decoder for feature information sharing. The encoder part of U-net uses cascaded residual blocks (ResBlocks) to process the input noisy image x. t Dimensionality reduction and multi-scale visual features are extracted, while the cascaded ResBlock in the decoder part outputs a noisy image x based on the multi-scale visual features. t The noise component in the network. However, unlike the ordinary U-net, the input of the score predictor also includes time t. Therefore, we use the temporal embedding method of the Transformer neural network structure to encode time t into a 256-dimensional vector and input it into each ResBlock of the U-net network, so that the U-net can explicitly depend on time t.

[0063] Condition generator G ω The implementation uses a U-net structure similar to the score predictor, but adds random noise u as input. Therefore, we use an additional three-layer fully connected network to map the noise u into a 256-dimensional vector and input it into each ResBlock of the U-net, allowing the U-net to explicitly depend on the noise u. Conditional generator G ω The denoised prediction f obtained by the fractional predictor θAs input, the encoder extracts multi-scale visual features for denoising prediction, and the decoder outputs higher quality and more accurate denoising prediction based on the multi-scale features.

[0064] Discriminator D ψ The implementation is similar to the encoder part of the score predictor U-net, using a time-dependent ResBlock. However, the input is a noisy image x at the current time t. t And the image x at the next moment t-1 t-1 That is, function D ψ (x t-1 ,x t ,t). We will x t-1 and x t The inputs are connected together to form a neural network with 6 channels, and the output part obtains real-valued discrimination scores through convolutional neural layers.

[0065] Step 4: Train a fast denoising diffusion probability model;

[0066] The training of the fast denoising diffusion probability model consists of two stages. The first stage trains the score predictor network of the original denoising diffusion probability model. θ The following score-matching training process is adopted.

[0067] 1) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time intervals t~U(t). Sample a batch of Gaussian noise from a standard Gaussian distribution with the same dimension as the images, denoted as...

[0068] 2) The variables t, x0, obtained above, ∈ Substitute the values ​​into the unweighted score matching squared error function and calculate the mean of the function as the objective function L(θ) for this optimization.

[0069] 3) Optimize the neural network parameters θ by backpropagating the objective function L(θ);

[0070] 4) Repeat steps 1)-3) until the objective function converges or the maximum number of optimizations is reached.

[0071] The unweighted score matching objective function is specifically as follows:

[0072]

[0073] Expectation in the formula This indicates the operation of calculating the mean of the function based on the variables in step 2).

[0074] The second stage involves freezing the trained score matcher. θBelow, the conditional generator network G is introduced to train the fast denoising diffusion probability model. ω and discriminator network D ψ The following alternating training process of generating adversarial responses is adopted.

[0075] 1) Transfer the score matcher trained in the first stage to... θ Freezing serves as the basis for subsequent processes;

[0076] 2) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time t~U(t). Based on x0, follow the Gaussian conditional distribution q(x0). t-1 A batch of x samples is taken from |x0) t-1 According to x t-1 Forward Gaussian diffusion of kernel q(x) t |x t-1 A batch of samples x from ) t According to x t The inverse Gaussian denoising kernel p after introducing a condition generator θ,ω (x t-1 |x t The predicted x is obtained by sampling in ) t-1 ;

[0077] 3) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute this into the discriminator parameter ψ function of the generative adversarial objective function, and calculate the mean of this function as the optimization objective of the discriminator;

[0078] 4) Optimize the discriminator parameters ψ by backpropagating the objective function with respect to the discriminator parameters ψ;

[0079] 5) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute the value of the function with respect to the condition generator parameter ω into the adversarial objective function, and calculate the mean of the function as the optimization objective of the condition generator;

[0080] 6) Optimize the condition generator parameter ω by backpropagating the objective function with respect to the condition generator parameter ω;

[0081] 7) Repeat steps 2)–6) until the adversarial objective function converges or the maximum number of optimization iterations is reached.

[0082] The generative adversarial objective function for the discriminator parameter ψ and the condition generator parameter ω is:

[0083]

[0084]

[0085] The relevant probability distribution in the formula can be Gaussian or uniform, both of which allow for direct sampling, as shown in step 2. Intuitively, as... Figure 1 As shown, we will define the current state x t Input frozen score predictor ∈ θ A coarse denoising prediction x′ is obtained through calculation. o Then the rough prediction x′ o Input condition generator G ω We obtain a finely denoised prediction x′0′, and then input a new inverse Gaussian denoising kernel p. θ,ω (x t-1 |x t ) to obtain the predicted x t-1 We introduce a discriminator D. ψ To guide the condition generator G ω Train the discriminator to distinguish real x t-1 and the predicted x t-1 Next, the conditional generator is trained to provide better denoised predictions, thereby improving the predicted x. t-1 With real x t-1 near.

[0086] Step 5: Generate new samples using a fast denoising diffusion probability model;

[0087] The score predictor trained in the first stage ∈ θ The second-stage trained condition generator G ω Substitute the new inverse Gaussian denoising kernel p θ,ω (x t-1 |x t This constitutes a fast denoising generation process:

[0088]

[0089] In the formula, p(x) T ) represents the standard Gaussian distribution p θ (x 0:T ) represents the joint distribution of the process.

[0090] At each denoising step t in the newly generated denoising process, the score predictor ∈ θ Used for analyzing noisy images. t Based on the image features, a preliminary denoising prediction x′ is given. o The initial denoising prediction quality is poor. Next, the condition generator G... ω The poor denoising prediction x′ oTransforming into a high-quality denoised prediction x′0′, making the denoised kernel p θ,ω (x t-1 |x t It has a more accurate mean. This generative denoising process can transform random Gaussian noise into realistic, high-quality image samples, and the number of denoising steps is reduced to T=4.

[0091] Comparative Example

[0092] We used IS and FID as the generation quality metrics on the CIFAR10 dataset. The fast denoising diffusion probability model proposed in this invention achieves an IS score of 8.51 and an FID score of 11.41 on CIFAR10, outperforming recently proposed baseline models. The comparative results on CIFAR10 are shown in Table 1. Randomly generated images are shown below. Figure 2 As shown.

[0093] Table 1 Comparison of CIFAR10 generated task effects

[0094]

[0095] The generative models being compared are:

[0096] 1.NCSN model (see Generative modeling by estimating gradients of thedata distribution. Advances in Neural Information Processing Systems, 2019.)

[0097] 2.DiffuseVAE model (see Diffusevae: Efficient, controllable and high-fidelity generation from low-dimensional latents. ArXiv, abs / 2201.00308, 2022.)

[0098] 3.AutoGAN model (see Autogan: Neural architecture search for generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, pp. 3223–3233, 2019.)

[0099] 4. SNGAN model (See Spectral normalization for generative adversarial networks. In International Conference on Learning Representations, 2018.)

[0100] 5. Glow model (See Glow: Generative flow with invertible 1x1 convolutions. In Advances in neural information processing systems, 2018.)

[0101] 6. PixelCNN model (See Pixel recurrent neural networks. International Conference on Machine Learning, 2016.)

[0102] 7. NVAE model (See NVAE: A deep hierarchical variational autoencoder. In Advances in neural information processing systems, 2020.)

[0103] 8. IGEBM model (See Implicit generation and modeling with energy based models. In Advances in Neural Information Processing Systems, 2019.)

[0104] 9. VAEBM model (See Vaebm: A symbiosis between variational autoencoders and energy-based models. In International Conference on Learning Representations, 2021.)

Claims

1. A method for image feature analysis and generation based on a fast denoising diffusion probability model, characterized in that, Includes the following steps: S1: Preprocess the image generation dataset to obtain the training dataset; S2: Construct the original denoising diffusion probability model, which includes a score predictor, and introduce a condition generator and a corresponding discriminator to obtain a fast denoising diffusion probability model; S3: Train the score predictor using the score matching method, freeze the trained score predictor, and train the condition generator and discriminator using the alternating training method of generative adversarial training. S4: Update the trained score predictor and the trained condition generator to the fast denoising diffusion probability model. Input the Gaussian white noise image into the fast denoising diffusion probability model to obtain a real, high-quality output image. In step S2, the original denoising diffusion probability model includes a forward Gaussian diffusion kernel and a reverse Gaussian denoising kernel, wherein the forward Gaussian diffusion kernel is... Used to implement the forward diffusion process of adding noise to an image x0…x t …x T ,in Represents a Gaussian distribution, β t Given a pre-defined noise scale, times t = 1, ..., T, the starting point x0 of the diffusion process is a noise-free image, and the intermediate state x... t The image contains noise at time t, and the endpoint x is the process endpoint. T The image is a Gaussian white noise image; the inverse Gaussian denoising kernel p θ (x t-1 |x t ) is used to implement the reverse denoising generation process x T …x t …x0, specifically In the formula, σ t I is the standard deviation of the Gaussian denoising kernel; I is the identity matrix; f θ (x t ,t) represents denoising prediction; ∈ θ (x t ,t) is the fractional predictor; μ θ (x t (t) represents the mean; In step S2, the condition generator is G. ω (u,f θ (x t ,t),t), Introducing this into the inverse denoising generation process yields the inverse Gaussian denoising kernel after incorporating the conditional generator: In the formula, μ θ,ω (x t ,t) is the mean after introducing the condition generator, and u represents the Gaussian noise obtained from sampling.

2. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 1, characterized in that, The score predictor ∈ θ The implementation is based on the U-net neural network structure, which is a symmetrical encoder-decoder structure. In the corresponding stages of the encoder and decoder, a jump connection is used to share feature information. The U-net encoder section uses cascaded ResBlocks to process the input noisy image x t Dimensionality reduction and multi-scale visual features are extracted, while the cascaded ResBlock in the decoder part outputs a noisy image x based on the multi-scale visual features. t The noise part in the network is eliminated; the time t is encoded into a 256-dimensional vector using the temporal embedding method of the Transformer neural network structure, and then input into each ResBlock of the U-net network, so that the U-net can explicitly depend on time t. Condition generator G ω The implementation is based on the U-net neural network structure used by the score predictor, which is a symmetrical encoder-decoder structure. An additional three-layer fully connected network is used to map the noise u into a 256-dimensional vector, which is then input into each ResBlock of the U-net, so that the U-net can also explicitly depend on the noise u. The encoder side extracts multi-scale visual features for denoising prediction, and the decoder side outputs the optimized denoising prediction based on the multi-scale features. Discriminator D ψ The implementation is based on the U-net neural network structure, using a time-dependent ResBlock, with the input being the noisy image x at the current time t. t And the image x at the next moment t-1 t-1 The output part obtains real-valued discrimination scores through convolutional neural layers.

3. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 2, characterized in that, Step S3, which trains a score predictor using a score matching method, includes the following sub-steps: (1) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time t~U(t). Sample a batch of Gaussian noise from a standard Gaussian distribution with the same dimension as the images, denoted as... (2) Substitute the variables t, x0, ∈ obtained above into the unweighted score matching squared error function, and calculate the mean of the function as the objective function L(θ) for this optimization. (3) Optimize the neural network parameters θ by backpropagating the objective function L(θ); (4) Repeat steps 1)-3) until the objective function converges or the maximum number of optimizations is reached.

4. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 3, characterized in that, The objective function is: In the formula, This indicates that the mean of a function is calculated based on the variables.

5. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 4, characterized in that, Step S3, which trains the condition generator and discriminator using an alternating training method for generative adversarial networks, includes the following sub-steps: (1) The trained score matcher ∈ θ Freezing serves as the basis for subsequent processes; (2) Randomly select a batch of noise-free images from the training dataset, denoted as x0~q(x0). Average a batch of integers from [1,T] as time t~U(t). Based on x0, follow the Gaussian conditional distribution q(x0). t-1 A batch of x samples is taken from |x0) t-1 According to x t-1 Forward Gaussian diffusion of kernel q(x) t |x t-1 A batch of samples x from ) t According to x t The inverse Gaussian denoising kernel p after introducing a condition generator θ,ω (x t-1 |x t The predicted x is obtained by sampling in ) t-1 ; (3) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute this into the discriminator parameter ψ function of the generative adversarial objective function, and calculate the mean of this function as the optimization objective of the discriminator; (4) Optimize the discriminator parameter ψ by backpropagating the objective function about the discriminator parameter ψ; (5) The variables t, x0, x obtained above t-1 ,x t ,x t-1 Substitute the value of the function with respect to the condition generator parameter ω into the adversarial objective function, and calculate the mean of the function as the optimization objective of the condition generator; (6) Optimize the condition generator parameter ω once by backpropagating the objective function about the condition generator parameter ω; (7) Repeat steps (2)–(6) until the generated adversarial objective function converges or the maximum number of optimizations is reached.

6. The image feature analysis and generation method based on a fast denoising diffusion probability model according to claim 5, characterized in that, The generative adversarial objective function for the discriminator parameter ψ and the condition generator parameter ω is:

Citation Information

Patent Citations

  • Generative model training method and device, sample generation method and computing equipment

    CN113822320A

  • Generative model training method and device, noise scale generation method and computing equipment

    CN113822321A