A SAR image condition generation method and device based on statistical diffusion model

Abstract

The present invention discloses a method and device for generating SAR image conditions based on a statistical diffusion model, which relates to the field of image generation technology. The method comprises: a forward process of acquiring data to be processed and gradually adding noise to the data to generate noise data to be processed; and a reverse process of gradually sampling the noise data to be processed, gradually estimating and removing noise from the noise data to generate new samples with conditions. The method adds preset conditions to generate new samples with conditions. The method can generate high-resolution, high-quality, and clear SAR images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image generation, and in particular relates to a method and device for generating SAR image conditions based on a statistical diffusion model. Background Art

[0002] Synthetic Aperture Radar (SAR) is a high-resolution imaging radar. SAR actively transmits electromagnetic waves toward the ground and then interacts with the reflected electromagnetic waves from the scene, thereby describing the ground or target observations. SAR offers advantages such as multi-polarization, all-day, multi-band, and all-weather capability, and is unaffected by environmental factors such as weather and light. Consequently, SAR has been widely used in meteorological observation, land surveying, resource exploration, and military applications, attracting extensive attention and research from scholars both domestically and internationally. With the continuous development of SAR, research on SAR technology has also increased globally. Currently, SAR is primarily used in image recognition and target detection. However, image recognition and target detection require a large number of training samples to ensure good accuracy. However, SAR images are expensive and limited in number, making the acquisition of large numbers of SAR images challenging. Therefore, developing effective and low-cost SAR image generation methods has become a research hotspot for many scholars.

[0003] Traditional SAR generation modeling methods, most of which use SAR image feature information to reconstruct images, often lack similarity to the original images. This is primarily due to inaccuracies in azimuth, target outlines, and shadows. Furthermore, traditional methods are expensive to construct SAR images, and the resulting images lack diversity. Consequently, traditional methods face numerous difficulties in the field of SAR image generation, hindering their development. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method and device for generating SAR image conditions based on a statistical diffusion model. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] In a first aspect, the present invention provides a method for generating SAR image conditions based on a statistical diffusion model, comprising:

[0006] In the forward process, the data to be processed is obtained, noise is gradually added to the data to be processed, and noise data to be processed is generated;

[0007] The reverse process is to gradually sample the noise data to be processed, gradually estimate and remove the noise in the noise data to be processed, and generate new samples with conditions;

[0008] Among them, preset conditions are added to generate new samples with conditions.

[0009] In a second aspect, the present invention further provides a SAR image condition generation device based on a statistical diffusion model, comprising: a forward processing module for acquiring data to be processed, gradually adding noise to the data to be processed, and generating noise data to be processed;

[0010] A reverse processing module is used to gradually sample the noise data to be processed, gradually estimate and remove the noise in the noise data to be processed, and generate new samples with conditions;

[0011] Among them, preset conditions are added to generate new samples with conditions.

[0012] Beneficial effects of the present invention:

[0013] The present invention provides a method and device for generating SAR image conditions based on a statistical diffusion model. These methods generate high-resolution, high-quality, and clear SAR images, providing high-quality images for SAR target detection and recognition. This method addresses the severe overfitting problem often encountered in training deep neural networks due to insufficient SAR sample numbers, which reduces the network's generalization ability.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the probability diffusion model provided by the existing technology;

[0016] Figure 2 This is a flow chart of a method for generating SAR image conditions based on a statistical diffusion model provided by an embodiment of the present invention;

[0017] Figure 3 This is a schematic diagram of a SAR image condition generation method based on a statistical diffusion model provided by an embodiment of the present invention;

[0018] Figure 4 is another schematic diagram of a SAR image condition generation method based on a statistical diffusion model provided by an embodiment of the present invention;

[0019] Figure 5 This is a schematic diagram of a simulation experiment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0021] With the rapid development of deep generative networks (GANs), image generation methods based on these networks have become the mainstream. Variational autoencoders (VAEs) and generative adversarial networks (GANs) are two widely used models for image generation. VAEs, as generative models, effectively generate SAR images with statistical properties and physical consistency by constructing a latent variable space. GANs, through game learning between a generator and a discriminator, can generate high-resolution, visual SAR images, addressing the shortcomings of traditional generative methods in removing detailed textures and target features.

[0022] The above two methods address the issue of insufficient SAR image samples to a certain extent, significantly improving SAR image interpretation. While the VAE model has a complete Bayesian interpretation, it encodes the image into a Gaussian latent vector z, with each component being independently distributed. This approach is not suitable for fitting arbitrarily complex distributions. Therefore, no matter how the hyperparameters are adjusted, KL(p(x,z),q(x,z)) cannot be zero. This formula represents the difference between the distribution learned by the encoder (p(x,z)) and the prior distribution q(x,z). Therefore, the VAE model can only produce a rough, average result, which is one of the reasons why the images generated by the VAE model tend to be blurry. Furthermore, the VAE model over-focuses on the reconstruction loss and ignores the smoothness of the latent space, resulting in poor quality generated samples. While the GAN model can generate high-quality, realistic data, due to its relatively complex design and the need to achieve a balance between the generator and discriminator during training, it is difficult to train and prone to failure. Furthermore, the design of the generator and discriminator in the GAN model can result in insufficient diversity in the generated images, meaning that the generated samples are too similar and lack diversity.

[0023] In view of this, the present invention provides a conditional generation method for SAR images based on a statistical diffusion model. It adopts multiple conditional generation models to generate SAR images, thereby achieving the generation of high-quality SAR images under specific conditions. It solves the problems in the prior art such as insufficient number of SAR image samples, blurred generated images, overly similar generated samples, and lack of diversity.

[0024] Before introducing the technical solution adopted by the present invention, it is necessary to first understand the Denoising Diffusion Probabilistic Models (DDMP). DDMP includes a forward process and a reverse process. The forward process is also called a diffusion process. Both the forward process and the reverse process are parameterized Markov chains. The reverse process is used to generate data, such as Figure 1 As shown, Figure 1 It is a schematic diagram of the probability diffusion model provided by the existing technology.

[0025] like Figure 1 As shown in the figure, the DDPM forward process is a process of continuously adding noise, x0 is the original image, ε represents the noise sampled from the standard normal distribution, α t Represents linear interpolation from large to small, which is used to control the degree of noise addition of the model. The forward process is expressed as:

[0026]

[0027] Among them, x t represents the image after t steps of noise addition, x t-1 represents the image after t-1 steps of noise addition, i represents the index of the step, and N(0,I) represents the normal distribution.

[0028] The original image is continuously denoised, and after a finite number of steps T, the image x T It becomes a standard normal distribution for the reverse process to perform denoising operations.

[0029] The DDPM reverse process is a gradual denoising process, such as Figure 1 As shown, DDPM continuously removes noise from the standard normal distribution and finally generates an image; according to the Bayesian formula, given x0 and x t In the case of , formula (1.1) can be expressed as:

[0030]

[0031] According to formula (1.2), p(x t |x0) satisfies the Gaussian distribution, so we can reversely infer that p(x t-1 |x t ,x0) satisfies:

[0032]

[0033] That is, given x0 and x t In the case of xt-1 Obeys a Gaussian distribution, the mean of which is related to x0, but in generating x t-1 Therefore, DDPM uses a neural network to predict x0, that is, the following formula (1.5), ε θ (x t ,t) represents the noise predicted by the neural network, Represents the predicted x0, which is expressed as:

[0034]

[0035] From the above equations (1.4) and (1.5), it can be actually expressed as:

[0036]

[0037] Thus, by repeatedly t Sample x t-1 , and finally generate an image, due to the prediction The prediction is made through a trained neural network, and the noise prediction also relies on the trained neural network. DDPM retains a strong generation capability.

[0038] The existing DDPM training process is the forward process, training the model to predict noise; the utilization process is the reverse process, as described above. This invention extends DDPM, which generates noise unconditionally, while the present invention offers four conditional generation schemes based on DDPM.

[0039] See Figure 2 , Figure 2 This is a flow chart of a method for generating SAR image conditions based on a statistical diffusion model provided by an embodiment of the present invention. The method for generating SAR image conditions based on a statistical diffusion model provided by the present invention includes:

[0040] S101, forward process, obtaining data to be processed, gradually adding noise to the data to be processed, and generating noise data to be processed.

[0041] S102, reverse process, step by step sampling the noise data to be processed, and step by step estimating and removing the noise in the noise data to be processed, to generate new samples with conditions;

[0042] Among them, preset conditions are added to generate new samples with conditions.

[0043] This paper provides a conditional SAR image generation method based on a statistical diffusion model, generating high-resolution, high-quality, and clear SAR images for SAR target detection and recognition. This method addresses the severe overfitting problem often encountered in training deep neural networks due to insufficient SAR samples, which reduces the network's generalization ability.

[0044] In an optional embodiment of the present invention, see Figure 3 , Figure 3 This is a schematic diagram of a SAR image condition generation method based on a statistical diffusion model provided by an embodiment of the present invention. The main contents of Solution 1 include:

[0045] Step-by-step sampling of the noise data to be processed, and step-by-step estimation and removal of the noise in the noise data to be processed, to generate new samples with conditions, including:

[0046] Gradually sample the noise data to be processed and input it into the trained convolutional neural network, gradually estimate and remove the noise in the noise data to be processed, and generate unconditional new samples;

[0047] Use the trained classifier to classify unconditional new samples and generate new samples with conditions.

[0048] In an optional embodiment of the present invention, the main contents of Solution 2 include:

[0049] Stepwise sampling of the noise data to be processed includes:

[0050] When sampling the noise data to be processed, a gradient offset is added so that the mean of the sampled distribution is closer to the mean of the distribution of the required conditional category. The expression of the gradient offset is:

[0051]

[0052] Among them, s represents the gradient scaling parameter, β t-1 represents the noise intensity at time step t-1 in the forward process, β t represents the noise intensity at time step t in the forward process, often also called the diffusion coefficient, x t represents the time step, and y represents the category information.

[0053] It should be noted that the reverse process satisfies the Gaussian distribution, and the mean is the center point of the Gaussian distribution, which is also the core of reverse generation. The mean of the required generated category is the center point of the target distribution. The closer the two center points are, the better the generation effect.

[0054] In an optional embodiment of the present invention, stepwise sampling the noise data to be processed includes:

[0055] Given a known time step x t And category information y, get the data state x of the previous time step t-1 The expression of the conditional probability distribution is:

[0056]

[0057] Among them, x t By x t-1 Add noise to get p(y|x t-1 ,x t )=p(y|x t-1 ), the previous state x t-1 The conditional probability distribution of is updated as:

[0058]

[0059] according to Expressed as a first-order approximation from a Taylor expansion, p(y|x t ) About x t Expand so that you can use x t Approximate description x t-1 The distribution around it is:

[0060]

[0061] Among them, the above formula is expressed as a Gaussian distribution, defined from x t Sample x t-1 The distribution process of x t Represents the data state of the current time step, σ t Indicates that the noise intensity associated with the time step in the reverse process is correlated with the noise intensity in the forward process;

[0062] Get the data state x of the previous time step t-1 The distribution of the conditional probability of is expressed as:

[0063]

[0064] in, is x t After inputting into the preset classifier, x t The derivative shows that Ι represents the identity matrix, which is used for the covariance matrix, indicating that each dimension is independent and the noise is evenly distributed.

[0065] In an optional embodiment of the present invention, the method further includes: training the forward process and the reverse process; wherein,

[0066] Training the forward process includes using the preset data as the original data, gradually adding noise to the original data, and generating noise data with a standard normal distribution;

[0067] Training the reverse process involves taking the noisy data as initial data, training the initial convolutional neural network with the goal of minimizing the difference between the predicted noise and the actual added noise, and gradually removing the noise to reconstruct the original data.

[0068] In an optional embodiment of the present invention, see Figure 4 , Figure 4 This is another schematic diagram of a method for generating SAR image conditions based on a statistical diffusion model provided by an embodiment of the present invention. The main contents of Solution 3 include:

[0069] Training the forward process also includes:

[0070] Encode the time steps and class information at the same time;

[0071] The initial convolutional neural network is trained using the encoded time steps and the encoded category information.

[0072] In an optional embodiment of the present invention, the data state x of the previous time step is obtained. t-1 The distribution of the conditional probability of is expressed as:

[0073]

[0074]

[0075] in, Indicates the data status of the current time step preset after adding category information.

[0076] Specifically, in this embodiment, the time step and category information are encoded and input together into the convolutional neural network to be trained;

[0077] From x T Start iterating the above process to x0;

[0078] Use convolutional neural networks to predict the noise at the current time step t, denoise it based on the noise prediction, and sample the next time step x t-1 ;

[0079] The above process is constrained by the category information. At time step t=0, the denoising process is completed, and due to the constraint of the category information, an image that meets the conditions is generated.

[0080] It should be noted that in scheme three, the category information is encoded at the beginning of the forward process, and it carries the category information throughout the entire diffusion process (forward and backward). In the forward process, the model does not rely on the category information, and only relies on the category information for guidance and image generation in the backward diffusion process.

[0081] In an optional embodiment of the present invention, the main contents of Solution 4 include:

[0082] Training the reverse process further includes:

[0083] The initial convolutional neural network is trained using the preset data set and the noise data, with the goal of minimizing the difference between the predicted noise and the actual added noise. The noise is gradually removed to reconstruct the original data, and the convolutional neural network parameters are obtained to construct a trained convolutional neural network.

[0084] Specifically, in this embodiment, first, the DDPM model is used as a pre-training model, and the forward process (noise addition process) is frozen, which means that the parameters of the forward process will not be updated during subsequent training; secondly, at the beginning of the reverse process, a specific data set (with specific category information) is input into the convolutional neural network to be trained together with the noise data; finally, during the denoising process, the reverse process network is optimized according to the current noise state and category information, and images that conform to specific categories are gradually denoised and generated.

[0085] It should be noted that Solution 3 does not freeze the forward process. The training model is optimized for both forward and backward training. This requires greater computing resources and takes a long time to train. However, if the training resources are sufficient, it can provide stronger control over details. Solution 4 only requires training the backward process, and the forward process is the forward process of DDPM. This can save computing resources and time costs.

[0086] In an optional embodiment of the present invention, training the initial convolutional neural network includes:

[0087] A threshold is set in the convolutional neural network to filter the output of the convolutional neural network and remove images with poor generation effect.

[0088] Based on the same inventive concept, the present invention further provides a SAR image condition generation device based on a statistical diffusion model, which is used to implement the SAR image condition generation method based on a statistical diffusion model provided in the above embodiment of the present invention. The method embodiment is referred to above and will not be described in detail here. The device includes:

[0089] The forward processing module is used to obtain the data to be processed, gradually add noise to the data to be processed, and generate noise data to be processed;

[0090] A reverse processing module is used to gradually sample the noise data to be processed, and gradually estimate and remove the noise in the noise data to be processed to generate new samples with conditions;

[0091] Among them, preset conditions are added to generate new samples with conditions.

[0092] In an optional embodiment of the present invention, the effect of the SAR image condition generation method based on the statistical diffusion model provided in the above embodiment is verified through simulation experiments, specifically:

[0093] 1. FID scores of the four conditional generative models.

[0094] Common evaluation metrics in the generative image generation field include the Inception Score (IS) and the Fréchet Inception Distance (FID). However, since the IS uses an Inception model pre-trained on the natural image dataset ImageNet to calculate the conditional distribution of generated images, and SAR images are not natural images, the IS is not suitable for evaluating SAR image models in this paper. In contrast, the FID metric compares the feature vectors obtained by inputting the original image and the generated image into the Inception model to obtain the FID score. Therefore, most current SAR image generation models are evaluated using the FID metric.

[0095] However, the FID metric also has the following drawbacks: the FID score can only effectively measure the distance between samples when the number of samples is extremely large or the number of samples is the same. When the number of samples is small, the covariance matrix fluctuates greatly and cannot well represent the distance between samples.

[0096] To ensure a consistent sample size, we selected 428 samples (the minimum number of categories in the MSTAR dataset) to evaluate the four conditional generation schemes. Because the Classifier-Free scheme requires large GPU memory and crops images to (96x96), while the other three schemes crop images to (128x128), we only evaluated the performance of the Classifier-Free, Classifier-Guided, and Pre-Trained conditional generation schemes. Table 1 shows the resulting FID scores, where the black section indicates the lowest FID score, indicating the best-performing model.

[0097] Table 1 FID scores of three conditional generation models

[0098] Category Name Classifier pre-trained Classifier-Guidance 2S1 108.326 113.9692 124.4123 BMP2 93.2471 98.29862 98.65277 BRDM_2 73.9459 74.69507 90.906 BTR60 135.2398 129.214 152.8392 BTR70 112.9692 111.26 120.9701 D7 85.1499 92.71053 99.25594 T62 92.4519 96.06115 98.98671 T72 92.157 100.1628 118.0358 ZIL131 91.414 97.41465 105.1324 ZSU_23_4 87.9022 96.10545 93.20193

[0099] 2. Evaluation of generative models based on AlexNet.

[0100] SAR target detection and recognition are important research areas in SAR image interpretation. To ensure accuracy, target detection and recognition require a large number of training samples. This section will incorporate Diffusion-generated SAR images into classifier training and verify the research significance of this invention by evaluating the improved classifier performance. Furthermore, the performance improvement will be used to indirectly evaluate three conditional generation schemes, excluding the Classifier-Free scheme.

[0101] Since the data generated by Diffusion in this invention is trained by the SOC dataset in the MSTAR dataset, we will use the EOC dataset in the MSTAR dataset as the training sample of the classifier. Figure 5 As shown in Figure 2, we split the EOC dataset into training and test sets and merged the SOC dataset with the EOC training set to create a new training set. Images generated by the three conditional generation schemes were injected into the training set at certain ratios to determine whether injecting generated images improves classifier performance. The injection ratio was based on the amount of data in the SOC dataset and was interpolated using linear interpolation from 0 to 1 with a step size of 0.1.

[0102] AlexNet was used as the classifier network, with a batch size of 32 and an initial learning rate of 0.01. The training rate was reduced by one-tenth of the original value every 20 epochs, for a total of 50 epochs. Table 2 shows the experimental results. As can be seen from Table 2, when no image injection was performed, the classifier accuracy was 0.9358. Injecting images generated under all three conditions significantly improved classifier performance. Conditional classifier generation without a threshold uses conditional classifier generation, where the threshold is set to the mean of the ten-dimensional vector of the image after passing through the classifier. This shows that a properly set threshold can improve the generation quality of the conditional classifier generation model. The pre-trained conditional generation model also achieved good results. Therefore, based on the evaluation results of the classifier models, conditional classifier generation with a threshold achieved the best results, followed by pre-trained conditional generation, the classifier-guaidance classifier, and conditional generation without a threshold. Furthermore, the injection of images generated under all three schemes significantly improved classifier performance. Therefore, using diffusion to augment SAR images contributes to the development of SAR image interpretation.

[0103] Table 2 Classifier accuracy after injecting generated images

[0104]

[0105] It should be noted that, in this document, relational terms such as first and second are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that an article or device comprising a list of elements includes not only those elements but also other elements not explicitly listed. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the article or device comprising the element. Terms such as "connected" or "connected" are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. References to orientations or positional relationships, such as "upper," "lower," "left," and "right," are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate description and simplify the description of the present invention. They do not indicate or imply that the device or element referred to must have, be constructed, or operate in a specific orientation, and are therefore not to be construed as limiting the present invention.

[0106] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0107] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A SAR image conditional generation method based on a statistical diffusion model, characterized in that: include: In the forward process, data to be processed is obtained, and noise is gradually added to the data to be processed to generate noise data to be processed; The reverse process is to gradually sample the noise data to be processed, gradually estimate and remove the noise in the noise data to be processed, and generate new samples with conditions, including: gradually sampling the noise data to be processed, and inputting it into the trained convolutional neural network, gradually estimating and removing the noise in the noise data to be processed, and generating new unconditional samples; using the trained classifier to classify the new unconditional samples, and generating new samples with conditions; wherein, The stepwise sampling of the noise data to be processed includes: During the sampling process of the noise data to be processed, a gradient offset is added so that the mean of the sampled distribution is close to the mean of the distribution of the required conditional category; wherein the expression of the gradient offset is: ； in, represents the gradient scaling parameter, represents the time step in the forward process The noise intensity, represents the time step in the forward process The noise intensity, Represents the data state of the current time step, Indicates category information. It will After inputting into the preset classifier The derivative is obtained; According to the known data state of the current time step and category information , get the data status of the previous time step The expression of the conditional probability distribution is: ； in, Depend on Add noise to get , the data state of the previous time step The conditional probability distribution of is updated as: ； according to ,get: ； in, Represents the data state of the current time step, represents the noise intensity associated with the time step in the reverse process; Get the data status of the previous time step The distribution of the conditional probability of is expressed as: ； in, It will After inputting into the preset classifier The derivative is, represents the identity matrix, Indicates linear interpolation from large to small, Indicates the data status of the current time step preset after adding category information; Among them, preset conditions are added to generate new samples with conditions.

2. The SAR image condition generation method based on the statistical diffusion model according to claim 1, characterized in that: Also includes: The forward process and the reverse process are trained; wherein, Training the forward process includes using preset data as original data, gradually adding noise to the original data, and generating noise data of standard normal distribution; Training the reverse process includes taking the noisy data as initial data, training the initial convolutional neural network with the goal of minimizing the difference between the predicted noise and the actual added noise, and gradually removing the noise to reconstruct the original data.

3. The SAR image condition generation method based on the statistical diffusion model according to claim 2, characterized in that: Training the forward process further includes: Encode the time steps and class information at the same time; The initial convolutional neural network is trained using the encoded time steps and the encoded category information.

4. The SAR image condition generation method based on the statistical diffusion model according to claim 3, characterized in that: Get the data status of the previous time step The distribution of the conditional probability of is expressed as: ； ； in, Indicates the data status of the current time step preset after adding category information.

5. The SAR image condition generation method based on the statistical diffusion model according to claim 2, characterized in that: Training the reverse process further includes: The initial convolutional neural network is trained using the preset data set and the noise data, with the goal of minimizing the difference between the predicted noise and the actual added noise. The noise is gradually removed to reconstruct the original data, and the convolutional neural network parameters are obtained to construct a trained convolutional neural network.

6. The SAR image condition generation method based on the statistical diffusion model according to claim 2, characterized in that: Training the initial convolutional neural network includes: A threshold is set in the convolutional neural network to filter the output of the convolutional neural network and remove images with poor generation effect.

7. A SAR image condition generation device based on a statistical diffusion model, used to implement the SAR image condition generation method based on a statistical diffusion model according to any one of claims 1 to 6, characterized in that: include: A forward processing module is used to obtain data to be processed, and gradually add noise to the data to be processed to generate noise data to be processed; The reverse processing module is used to gradually sample the noise data to be processed, gradually estimate and remove the noise in the noise data to be processed, and generate new samples with conditions, including: gradually sampling the noise data to be processed and inputting it into a trained convolutional neural network, gradually estimating and removing the noise in the noise data to be processed, and generating unconditional new samples; using the trained classifier to classify the unconditional new samples to generate new samples with conditions; wherein, The stepwise sampling of the noise data to be processed includes: During the sampling process of the noise data to be processed, a gradient offset is added so that the mean of the sampled distribution is close to the mean of the distribution of the required conditional category; wherein the expression of the gradient offset is: ； in, represents the gradient scaling parameter, represents the time step in the forward process The noise intensity, represents the time step in the forward process The noise intensity, Represents the data state of the current time step, Indicates category information. It will After inputting into the preset classifier The derivative is obtained; According to the known data state of the current time step and category information , get the data status of the previous time step The expression of the conditional probability distribution is: ； in, Depend on Add noise to get , the data state of the previous time step The conditional probability distribution of is updated as: ； according to ,get: ； in, Represents the data state of the current time step, represents the noise intensity associated with the time step in the reverse process; Get the data status of the previous time step The distribution of the conditional probability of is expressed as: ； in, It will After inputting into the preset classifier The derivative is, represents the identity matrix, Represents linear interpolation from large to small, Indicates the data status of the current time step preset after adding category information; Among them, preset conditions are added to generate new samples with conditions.

Images