Semantic remote sensing image generation method based on de-noising diffusion probability model

Abstract

The invention discloses a semantic remote sensing image generation method based on a de-noising diffusion probability model. The method comprises the following steps: preparing data; constructing a de-noising diffusion model, and respectively constructing a forward diffusion structure and a reverse reasoning de-noising structure; in the training process, Gaussian noise is gradually added according to the initial remote sensing influence, and a noise target image is generated; and performing reverse reasoning sampling based on the target noise image and the semantic layout image to generate remote sensing influence. According to the method, through an improved conditional denoising network structure, a noise image is input into an encoder, meanwhile, semantic layout is embedded into the decoder through multi-layer space self-adaptive normalization operation, and the generation process of a semantic remote sensing image is fitted. By introducing a sampling strategy without classifier guidance, the semantic correlation is remarkably enhanced while the fidelity of the generated image is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence content generation and remote sensing image intelligent interpretation technology, and more specifically, to a semantic remote sensing image generation method based on a denoising diffusion probability model. Background Art

[0002] The acquisition and processing of remote sensing images is an important technical means in the fields of earth observation, environmental monitoring, disaster emergency response, etc. However, data-driven remote sensing image analysis methods usually rely on a large amount of labeled data to support complex tasks such as image classification, target detection, and semantic segmentation. The acquisition of these labeled data is often time-consuming and laborious, especially when high-quality semantic annotation is required, which further increases the cost and difficulty of data acquisition. Therefore, how to efficiently and cost-effectively acquire remote sensing samples is particularly important for intelligent interpretation.

[0003] Although existing data augmentation methods can increase the diversity of training data to a certain extent, they still have the problem of being unable to change high-level semantic information when generating high-quality remote sensing image data. Traditional deep learning methods mainly rely on generative adversarial networks (GANs). Although they have achieved some success, their poor generation quality and diversity in complex scenes limit their promotion in practical applications. In addition, how to effectively use semantic information and noisy images for conditional generation during the generation process remains a difficult problem to be solved. Therefore, exploring new generative models to improve the quality and diversity of generated images is of great significance for the intelligent interpretation of remote sensing images.

[0004] In this context, diffusion model, as an emerging generative model technology, has gradually attracted widespread attention due to its remarkable performance in image generation and editing tasks. Denoising diffusion probability model (DDPM), as a generative model based on maximum likelihood estimation, has demonstrated its great potential in image generation tasks. Compared with the traditional GAN ​​model, DDPM generates high-quality images from random noise through a gradual denoising process, which can effectively capture the complex semantic information in the image. In particular, the introduction of controllable diffusion model allows researchers to accurately control the specific content and structure of the image during the generation process, thereby generating synthetic data that is highly relevant to the target task. However, the existing DDPM model has certain limitations in processing semantic information, especially in complex remote sensing scenes, and the quality and semantic relevance of the generated results need to be improved. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to provide a semantic remote sensing image generation method based on a denoising diffusion probability model, overcome the limitations of existing generative adversarial networks in generating high-fidelity and high-diversity images in complex scenes, and fully consider the joint effect of noisy images and semantic layout in the process of semantic remote sensing image generation.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A semantic remote sensing image generation method based on a denoising diffusion probability model comprises the following steps:

[0008] (1) Data preparation: obtaining semantic layout images and corresponding high-quality remote sensing images for training;

[0009] (2) Construct a denoising diffusion model, and construct a forward diffusion structure and a reverse reasoning denoising structure respectively;

[0010] (3) During the training process, Gaussian noise is gradually added to the initial remote sensing effect to generate a noisy target image;

[0011] (4) Perform reverse reasoning sampling based on the target noise image and the semantic layout image to generate remote sensing impact.

[0012] In the above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model, in step (2), the forward diffusion structure converts the semantic layout image into a noise image by gradually adding Gaussian noise. Formula (1) describes the process of adding noise at each step:

[0013]

[0014] Among them, x t is the image or data at step t, α t The coefficient that controls the amount of noise added is usually related to the time step t, η is the normal distribution, and I is the unit matrix, which represents the covariance matrix of the noise.

[0015] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model uses the U-Net network to predict noise in the reverse reasoning denoising structure to generate a remote sensing image that is highly corresponding to the semantic layout image. The formula is:

[0016] p θ (xt-1|xt)=η(xt-1; μ θ (xt,t),∑ θ (xt,t)) (2);

[0017] Among them, μ θ is the mean function learned by the model parameter θ, which represents the estimate at the t-1 step after denoising, ∑θ (x t ,t) is the variance function learned by the model parameter θ, which represents the uncertainty after denoising, η is the normal distribution, and p θ is the inverse distribution based on the model parameters θ.

[0018] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model gradually adds noise to the image through the Markov chain during the process of generating the forward diffusion structure.

[0019] In the process of image generation, the Markov chain can gradually change state and add a small amount of noise at each step, making the generation process smoother and more controllable. The Markov chain is random, and by randomly selecting the path of state change, it can help the model produce more diverse image results during the generation process. The use of Markov chains can simplify the complex generation process, so that the steps of adding noise can be carried out in an orderly and step-by-step manner, rather than processing the entire image at once. By gradually adding noise to the image through the Markov chain, the original feature information of the image will be gradually lost, thereby increasing the increase of the noise target image. As the time step increases, the image will gradually approach the Gaussian noise distribution.

[0020] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model generates a reverse reasoning denoising structure, starting from a pure noise image, combining the semantic layout image conditions, gradually removing noise to generate the final remote sensing image.

[0021] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model adopts a multi-layer spatial adaptive normalization operation in the process of generating the reverse reasoning denoising structure, and embeds the semantic layout image information into the decoder of the model through the multi-layer spatial adaptive normalization operation. The formula is as follows:

[0022] f i+1 =γ i (x)·Norm(f i )+β i (x) (3);

[0023] Among them, f i is the input feature map of the i-th layer, Norm(·) refers to the parameter-free normalization operation, γ i (x) and β i (x) are the spatial adaptation weights and biases learned from the semantic layout x, respectively.

[0024] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model adopts the classifier unguided sampling strategy in the reverse reasoning denoising process, and improves the quality and semantic relevance of the generated image by adjusting the variance and mean of the model output. The formula is as follows:

[0025]

[0026] in, is the bootstrapped adjusted mean; it is obtained by adding an additional bootstrapped term to the original mean; μ θ (y t |x) is the mean estimated by the original model, indicating that under condition x, the noise image y t , s is the guiding factor, which controls the trade-off between the quality and diversity of the generated images. When s increases, the quality of the generated images may improve, but the diversity may decrease; ∑ θ (y t |x) is the variance of the model estimate; is the conditional distribution p(x|y t ) relative to y t gradient.

[0027] The above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model and the unguided sampling strategy of the classifier adjust the noise removal process in the generation process by combining the prediction results of the conditional and unconditional models to further improve the detail fidelity and semantic consistency of the generated image.

[0028] A conditional model refers to a model that combines the input semantic layout (semantic map) during the sampling process to guide the denoising process of the generated image, so that the generated image maintains a high degree of semantic consistency with the input semantic layout. This conditional model can ensure that the generated image reflects specific semantic information, thereby enhancing the semantic relevance of the image. The unconditional model generates images based solely on noise without semantic layout conditions, allowing the generated images to have more degrees of freedom and diversity. It does not require specific input conditions, so it can generate more diverse results. In the classifier unguided sampling strategy, by combining the conditional model and the unconditional model, the system can dynamically adjust the quality and diversity of the generated images by interpolating the output results of the two.

[0029] In the above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model, in step (3), during the training process, in the forward diffusion process, the model gradually disturbs the original clear image into a noisy target image close to the standard Gaussian distribution by gradually adding Gaussian noise to the input semantic layout image; this process is carried out using a Markov chain, and the semantic information is gradually lost by continuously superimposing noise.

[0030] In the above-mentioned semantic remote sensing image generation method based on the denoising diffusion probability model, in step (4), in the reverse denoising process, the model gradually removes noise based on the disturbed noisy target image and the input semantic layout image to restore high-quality semantic remote sensing images.

[0031] The technical solution of the present invention achieves the following beneficial technical effects:

[0032] 1. This method uses an improved conditional denoising network structure to input the noisy image into the encoder, and embeds the semantic layout into the decoder through a multi-layer spatial adaptive normalization operation to fit the generation process of semantic remote sensing images. By introducing a sampling strategy without classifier guidance, the present invention significantly enhances the semantic relevance while improving the fidelity of the generated image.

[0033] 2. This paper proposes a new semantic remote sensing image generation method based on a denoising diffusion probability model. Unlike traditional methods, our framework processes noisy images in the encoder and embeds the semantic layout into the decoder through multi-layer spatial adaptive normalization operations. This method not only improves the quality of the generated image, but also maintains the accuracy of the semantic layout during the generation process. In addition, we introduced a classifier-free guided sampling strategy, which further improves the fidelity and semantic consistency of the generated results by interpolating the scores of the unconditional model. In the classifier-free guided sampling strategy, by combining the conditional model and the unconditional model, the system can dynamically adjust the quality and diversity of the generated image by interpolating the output results of the two.

[0034] 3. Traditional image generation methods based on generative adversarial networks (GANs) often have uncertainties in generation quality and semantic relevance due to the limitations of high fidelity and diversity in complex scenes. The application effect of generative models designed based on different network structures in semantic remote sensing images needs to be studied urgently. The benefit of the present invention is that it introduces a semantic remote sensing image generation method based on a denoising diffusion probability model (DDPM). By independently processing the noise image and the semantic layout information and combining multi-layer spatial adaptive normalization operations, the quality and semantic consistency of the generated image are improved. The reliability and effectiveness of the application of this method in complex scenes are confirmed by comparative verification with the existing GAN method. The present invention uses multiple benchmark data sets, combines the U-Net network structure and the classifier-free guided sampling strategy, and proposes a semantic remote sensing image generation method based on a denoising diffusion probability model. The experimental results verify its superiority in high fidelity and diversity generation, and can provide rich samples for the intelligent interpretation of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 , Flow chart of semantic remote sensing image generation technology based on denoising diffusion probability model;

[0036] Figure 2 , semantic remote sensing image generation model based on denoising diffusion probability model;

[0037] Figure 3 , multi-layer spatial adaptive normalization module;

[0038] Figure 4 , sampling strategy without classifier guidance;

[0039] Figure 5a , the training set results of the denoising diffusion probability model; Figure 5b ,Test set results of denoising diffusion probability model; Figure 5c , the semantic remote sensing image generation results of the denoised diffusion probability model;

[0040] Figure 6 , Remote sensing image semantic segmentation results: the first row a is the original image, the second row b is the true value, the third row c is the original data segmentation result, and the fourth row d is the original data + generated data generation result;

[0041] Figure 7a is the original data distribution in remote sensing image data; Figure 7b The data distribution after controlling the category distribution in the generation process in remote sensing image data. DETAILED DESCRIPTION

[0042] First, a semantic remote sensing image generation method based on denoising diffusion probability model is used to generate high-quality remote sensing images, such as Figure 1 shown.

[0043] 1. Dataset preparation: obtain semantic layout images and corresponding high-quality remote sensing images for training.

[0044] The dataset uses commercial procurement images of Guangdong Province in 2023 (resolution 0.5 meters) as input, and uses samples accurately collected and interpreted by humans as labels. The samples are cut into small blocks of 512×512 pixels, with a total area of ​​1965.7km 2 A total of 16,467 samples were obtained in the region, and the entire dataset was divided into a training set (accounting for 80% of the total sample size) and a test set (accounting for 20%) to support the training and validation of deep learning models.

[0045] 2. Construct a denoising diffusion probability model.

[0046] The construction of the denoising diffusion probability model is divided into two main parts: the forward diffusion process and the reverse denoising process.

[0047] (1) The forward diffusion structure transforms the semantic layout image into a noisy image by gradually adding Gaussian noise. The following formula describes how to add noise at each step:

[0048]

[0049] Among them, x t is the image or data at step t, α t The coefficient that controls the amount of noise added is usually related to the time step t, η is the normal distribution, and I is the unit matrix, which represents the covariance matrix of the noise.

[0050] In the forward diffusion process, the model gradually disturbs the original clear image into a noisy image close to the standard Gaussian distribution by gradually adding Gaussian noise to the input semantic layout image; this process is carried out using a Markov chain, which gradually loses semantic information by continuously superimposing noise.

[0051] (2) In the reverse reasoning denoising structure, the U-Net network is used to predict noise and generate a remote sensing image that is highly corresponding to the semantic layout image. The formula is:

[0052] p θ (xt-1|xt)=η(xt-1; μ θ (xt,t),∑ θ (xt,t)) (2);

[0053] Among them, μ θ is the mean function learned by the model parameter θ, which represents the estimate at the t-1 step after denoising, ∑ θ (x t ,t) is the variance function learned by the model parameter θ, which represents the uncertainty after denoising, η is the normal distribution, and p θ is the inverse distribution based on the model parameters θ.

[0054] In the inverse denoising process, the model gradually removes noise based on the disturbed noise image and combines it with the input semantic layout image to restore high-quality semantic remote sensing images.

[0055] (2-1) Adopt multi-layer spatial adaptive normalization operation (SPADE).

[0056] In order to make full use of the semantic layout information and improve the quality and semantic relevance of the generated image, a multi-layer spatial adaptive normalization operation (SPADE) is used in the inverse denoising process. Specifically, the noisy image is input into the encoder of the U-Net structure, and the semantic layout information is processed by SPADE ( Figure 3 ) is embedded into the decoder to ensure that the generated image accurately reflects the semantic layout of the input.

[0057] In order to make full use of the semantic layout information and improve the quality and semantic relevance of the generated images, a multi-layer spatial adaptive normalization operation is used in the reverse reasoning denoising process to embed the semantic layout information into the decoder of the model, so that the generated image can more accurately reflect the input semantic layout. The formula is as follows:

[0058] f i+1 =γ i (x)·Norm(f i )+β i (x)

[0059] Among them, f i is the input feature map of the i-th layer, Norm(·) refers to the parameter-free normalization operation, γ i (x) and β i (x) are the spatial adaptation weights and biases learned from the semantic layout x, respectively.

[0060] (2-2) Introducing a sampling strategy without classifier guidance

[0061] We also introduce a sampling strategy without classifier guidance ( Figure 4 ), by combining the prediction results of the conditional and unconditional models, the noise removal process in the generation process is adjusted to further improve the detail fidelity and semantic consistency of the generated images.

[0062] like Figure 4 As shown, two denoising networks are used, with the same structure (shared weights) but accepting different inputs. The first denoising network accepts noisy images with semantic information and generates conditional noise (conditional model). The second denoising network accepts noisy images without semantic labels and generates unconditional noise (unconditional model).

[0063] In the process of reverse reasoning denoising, the classifier unguided sampling strategy is adopted to improve the quality and semantic relevance of the generated image by adjusting the variance and mean of the model output. The formula is as follows:

[0064]

[0065] in, is the bootstrapped adjusted mean. It is obtained by adding an additional bootstrapped term to the original mean, μ θ (y t |x) is the mean estimated by the original model, indicating that under condition x, the noise image y t The mean of , s is the guiding factor, which controls the trade-off between the quality and diversity of the generated images. When s increases, the quality of the generated images may improve, but the diversity may decrease, ∑ θ (y t|x) is the variance of the model estimate, is the conditional distribution p(xy t ) relative to y t gradient.

[0066] 3. During the training process, Gaussian noise is gradually added with the initial remote sensing effect to generate a noisy target image;

[0067] 4. Perform reverse reasoning sampling based on the target noise image and the semantic layout image to generate remote sensing impact.

[0068] During the forward diffusion and reverse denoising process, the model gradually optimizes the generated results through multiple rounds of iterations, and ultimately generates high-quality remote sensing images while maintaining the accuracy of semantic information.

[0069] The training of the entire denoising diffusion probability model uses the training set constructed in step 1. It is iteratively optimized through a large amount of sample data. In each iteration, the model adjusts the weights according to the difference between the real samples and the generated samples to continuously improve the accuracy and quality of the generated results. After the model training is completed, the test set is used to evaluate the generation performance of the model and verify its applicability and stability in different scenarios and conditions.

[0070] 2. Evaluation of Remote Sensing Image Generation Results

[0071] 1. Effectiveness and reliability evaluation

[0072] In order to comprehensively evaluate the effectiveness and reliability of the semantic remote sensing image generation method based on the denoising diffusion probability model proposed in this invention, we comprehensively analyze the generation results from both qualitative and quantitative perspectives.

[0073] In terms of qualitative evaluation, the semantic remote sensing image generation method based on the denoising diffusion probability model of the present invention is analyzed in combination with the training set, test set and generation results shown in Figure 5. The first column in the figure shows the actual remote sensing images in the training set and their corresponding semantic segmentation masks. These images contain rich terrain features and complex scene information. The second column is the test set, which also shows the real remote sensing images and their semantic segmentation masks, mainly used to verify the generation ability of the model. The third column is the generation result, which shows the remote sensing image generated by the model after the semantic layout input of the given test set.

[0074] By comparison, we can see that the generated results are highly consistent with the real remote sensing images visually. The specific analysis is as follows:

[0075] Detail restoration: The generated results accurately restore the details of buildings, roads, rivers, etc., especially in the complex terrain edges and structural details, the model shows a high resolution. The edges of roads and buildings are clear and coherent, which is consistent with the input of semantic layout.

[0076] Color consistency: The color distribution of the generated image is similar to that of the real image, and it can well simulate the natural colors and light and shadow effects of different objects, such as the reflection of the river surface and the green layering of vegetation. This shows that the model successfully generates remote sensing images with a natural feel while maintaining semantic information.

[0077] Structural fidelity: Whether it is a large area of ​​water or a complex building complex in the test set, the generated results accurately retain the structure of the objects in the original image. Especially in areas like parks or densely built areas, the model is able to maintain the arrangement and shape of buildings over a large range.

[0078] In terms of quantitative evaluation, the quality of the generated results is compared and analyzed in combination with Table 1.

[0079] Table 1 Qualitative evaluation of generated results

[0080]

[0081] Table 1 shows the performance of the generated images of two different models (GauGAN and the model of the present invention) in terms of the two indicators FID and LPIPS. FID (Fréchet Inception Distance): FID is an indicator to measure the difference in statistical distribution between the generated image and the real image. The lower the value, the closer the generated image is to the real image. As can be seen from the table, the FID value of the model of the present invention is 139.3, which is significantly lower than 148.7 of GauGAN. This shows that the denoising diffusion probability model of the present invention is superior to GauGAN in the overall quality of the generated images, and the distribution of the generated images is closer to the real images. LPIPS (Learned Perceptual Image Patch Similarity): LPIPS is used to measure the diversity of generated images. The higher the value, the greater the diversity between the generated images. From the data in the table, it can be seen that the LPIPS value of the model of the present invention is 0.835, which is higher than 0.706 of GauGAN, indicating that the model of the present invention generates more diverse images while maintaining semantic consistency.

[0082] Through the comprehensive analysis of these two indicators, it can be concluded that the denoising diffusion probability model of the present invention shows higher image quality and diversity in semantic remote sensing image generation, and is superior to traditional generative adversarial networks (such as GauGAN) in terms of adaptability and robustness in complex scenes. This further verifies the effectiveness and potential of the method of the present invention in practical applications.

[0083] 2. Generate results for semantic segmentation of remote sensing images

[0084] In the field of remote sensing image processing, semantic segmentation tasks usually rely on a large amount of high-quality annotated data. However, obtaining sufficient annotated data often faces high costs and time consumption, which to a certain extent limits the performance improvement of the semantic segmentation model. In order to solve this problem, the present invention proposes a semantic remote sensing image generation method based on a denoising diffusion probability model, which aims to enhance the data set by generating synthetic images and their corresponding semantic labels, thereby improving the performance of the semantic segmentation model. 16467 samples in the study area are used to generate a corresponding number of samples for data enhancement, and the results are compared qualitatively and quantitatively.

[0085] Figure 6 The results of semantic segmentation of remote sensing images are shown, comparing the effects of segmentation using original data and combined with generated data. The comparison between c and d shows the difference in the results of segmentation using only original data and combined with generated data. It can be clearly seen that the segmentation result (d) combined with generated data performs better in detail processing and boundary recognition. Specifically, the edges of buildings and roads in d are clearer and more complete, while the segmentation results in c are blurred or inaccurate in these areas. The introduction of generated data significantly improves the model's ability to recognize complex structures and small objects, and enhances the accuracy and consistency of the segmentation results.

[0086] The quantitative results of remote sensing image semantic segmentation are analyzed in combination with Table 2, where “Baseline” represents the model using only the original dataset, and “Baseline+Aug” represents the model using the enhanced dataset (including generated data).

[0087] Table 2 Quantitative results of remote sensing image semantic segmentation

[0088]

[0089] The model using the augmented dataset (Baseline+Aug) significantly outperforms the model using only the original dataset (Baseline) in all key indicators. Specifically, FWIoU increased from 59.20% to 77.52%, accuracy increased from 74.19% to 87.19%, and Kappa coefficient increased from 65.64% to 83.36%. These results show that by introducing data generated based on the denoising diffusion probability model for augmentation, the performance of the model in the semantic segmentation task has been comprehensively improved.

[0090] In actual remote sensing image data, the distribution of objects of different categories is often uneven, and the number of samples of some categories is small. In the generation process, by controlling the category distribution in the generation process, more samples belonging to scarce categories can be generated, thereby balancing the category distribution in the dataset and improving the performance of the model on scarce categories.

[0091] As can be seen from Figure 7, there is an obvious imbalance in the category distribution in the original data (a), and the number of samples in some categories such as forest land and cultivated land is much larger than that in other categories. After combining the generated data (b), by controlling the category distribution in the generation process, the number of samples in the scarce categories is significantly increased, such as the samples in categories such as photovoltaic panels and golf courses are effectively expanded. This shows that the semantic remote sensing image generation method based on the denoising diffusion probability model can balance the distribution of different categories in the data set, thereby improving the performance of the model on scarce categories and enhancing the overall classification accuracy.

[0092] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the claims of this patent application.

Claims

1. A semantic remote sensing image generation method based on a denoising diffusion probability model, characterized in that: The steps include: (1) Data preparation: obtaining semantic layout images and corresponding high-quality remote sensing images for training; (2) Construct a denoising diffusion model, and construct a forward diffusion structure and a reverse reasoning denoising structure respectively; (3) During the training process, Gaussian noise is gradually added to the initial remote sensing effect to generate a noisy target image; (4) Perform reverse reasoning sampling based on the target noise image and the semantic layout image to generate remote sensing impact.

2. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 1, characterized in that: In step (2), the forward diffusion structure transforms the semantic layout image into a noisy image by gradually adding Gaussian noise. Formula (1) describes the process of adding noise at each step: Among them, x t is the image or data at step t, α t The coefficient that controls the amount of noise added is usually related to the time step t, η is the normal distribution, and I is the unit matrix, which represents the covariance matrix of the noise.

3. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 1, characterized in that: In the reverse reasoning denoising structure, the U-Net network is used to predict noise and generate a remote sensing image that is highly corresponding to the semantic layout image. The formula is: p θ (x t-1 |x t )=η(x t-1 ;μ θ (x t ,t),∑ θ (x t ,t)) (2); Among them, μ θ is the mean function learned by the model parameter θ, which represents the estimate at the t-1 step after denoising, ∑ θ (x t ,t) is the variance function learned by the model parameter θ, which represents the uncertainty after denoising, η is the normal distribution, and p θ is the inverse distribution based on the model parameters θ.

4. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 2, characterized in that: In the process of generating the forward diffusion structure, noise is gradually added to the image through a Markov chain.

5. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 3, characterized in that: In generating the reverse reasoning denoising structure, we start from a pure noise image and combine it with the semantic layout image condition to gradually remove the noise and generate the final remote sensing image.

6. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 3, characterized in that: In the process of generating the reverse reasoning denoising structure, a multi-layer spatial adaptive normalization operation is used to embed the semantic layout image information into the decoder of the model through a multi-layer spatial adaptive normalization operation. The formula is as follows: f i+1 =c i (x)·Norm(f i )+b i (x) (3); Among them, f i is the input feature map of the i-th layer, Norm(·) refers to the parameter-free normalization operation, γ i (x) and β i (x) are the spatial adaptation weights and biases learned from the semantic layout x, respectively.

7. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 3, characterized in that: In the process of reverse reasoning denoising, the classifier unguided sampling strategy is adopted to improve the quality and semantic relevance of the generated image by adjusting the variance and mean of the model output. The formula is as follows: in, is the bootstrapped adjusted mean; it is obtained by adding an additional bootstrapped term to the original mean; μ θ (y t |x) is the mean estimated by the original model, indicating that under condition x, the noise image y t , s is the guiding factor, which controls the trade-off between the quality and diversity of the generated images. When s increases, the quality of the generated images may improve, but the diversity may decrease; ∑ θ (y t |x) is the variance of the model estimate; is the conditional distribution p(x|y t ) relative to y t gradient.

8. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 7, characterized in that: The classifier unguided sampling strategy adjusts the noise removal process in the generation process by combining the prediction results of the conditional and unconditional models to further improve the detail fidelity and semantic consistency of the generated images.

9. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 1, characterized in that: In step (3), during the training process, in the forward diffusion process, the model gradually disturbs the original clear image into a noisy target image close to the standard Gaussian distribution by gradually adding Gaussian noise to the input semantic layout image; this process is carried out using a Markov chain, and by continuously superimposing noise, the semantic information is gradually lost.

10. The method for generating semantic remote sensing images based on a denoising diffusion probability model according to claim 1, characterized in that: In step (4), in the reverse denoising process, the model gradually removes noise based on the disturbed noisy target image and the input semantic layout image to restore high-quality semantic remote sensing images.

Images