Cirrus cloud image generation method based on non-classification guidance

Abstract

A cirrus cloud image generation method based on unclassified guidance solves the problems of unreliable remote sensing image defogging datasets and a limited number of cirrus cloud image features, belonging to the field of image generation technology. The invention comprises the following steps: obtaining a real cirrus cloud image, and obtaining a cirrus cloud image dataset after preprocessing; constructing a Markov chain using forward process diffusion, and gradually adding random noise that follows a Gaussian distribution to the cirrus cloud image in the cirrus cloud image dataset, so that the cirrus cloud image is noisy to a pure noise image; constructing a training dataset based on the pure noise image and the corresponding real noise; establishing a noise prediction model using the pure noise image as input and the corresponding noise as output, and training using the training dataset; inputting the randomly generated noise image into the trained noise prediction model, and performing reverse denoising on the image using the predicted noise combined with the posterior probability. After a set number of denoising steps, a cirrus cloud image with real cirrus cloud features is obtained.

Description

Technical Field

[0001] The invention relates to a cirrus cloud image generation method based on non-classification guidance, belonging to the technical field of image generation. Background Art

[0002] In recent years, with the rapid development of generative models, a large number of high-quality image generation models have emerged, with DALL E, MidJourney, and Stable Diffusion being among the most representative. These models have demonstrated powerful generative capabilities in fields such as art creation, virtual reality, and medical image analysis, promoting the widespread application and continuous innovation of image generation technology. However, in the field of remote sensing imagery, particularly in remote sensing image dehazing research, the application of such technologies remains relatively limited, and high-quality datasets are still scarce. The lack of a unified open-source dataset has significantly limited the research and development of remote sensing image dehazing algorithms.

[0003] Currently, some researchers have achieved the synthesis of realistic non-uniform haze remote sensing images based on the wavelength dependence and spatial variation characteristics of haze. However, these methods are usually based on fixed cirrus cloud images, resulting in a lack of diversity in the generated remote sensing image dehazing datasets, which is unrealistic. Furthermore, the limited number of features in cirrus cloud images cannot fully meet the requirements of deep learning models for learning large-scale and diverse data. Summary of the Invention

[0004] Aiming at the problems of unreliable remote sensing image defogging datasets and limited number of cirrus image features, the present invention provides a cirrus image generation method based on non-classification guidance.

[0005] The present invention provides a method for generating cirrus cloud images based on non-classification guidance, comprising:

[0006] Step 1: Obtain a real cirrus cloud image;

[0007] Step 2: Preprocess the acquired real cirrus cloud images to obtain a cirrus cloud image dataset;

[0008] Step 3: Use forward diffusion to construct a Markov chain and gradually add random noise that follows a Gaussian distribution to the cirrus cloud images in the cirrus cloud image dataset, so that the cirrus cloud images are noisy to pure noise images. Then, a training dataset is constructed based on the pure noise images and the corresponding real noise.

[0009] Step 4: Take the pure noise image as input and the corresponding noise as output, establish a noise prediction model, and train the noise prediction model using the training data set;

[0010] Step 5: Input the randomly generated noise image into the trained noise prediction model. The predicted noise is combined with the posterior probability to perform reverse denoising on the image. After the set number of denoising steps, a cirrus cloud image with real cirrus cloud characteristics is obtained.

[0011] Preferably, in step 1, a real cirrus cloud image in the aerosol band with a spectral range of 1.360 to 1.390 μm is obtained.

[0012] Preferably, in step 2, preprocessing the acquired real cirrus cloud image includes:

[0013] Each real cirrus cloud image is cropped, and the cropped images are filtered to select images with clear cirrus clouds and obvious features. The filtered images X are normalized to obtain the preprocessed cirrus cloud image x0:

[0014]

[0015] As a preference, in step 3, random noises that obey a Gaussian distribution are gradually added to the cirrus cloud images in the cirrus cloud image dataset, x t It represents the cirrus cloud image after t steps of noise addition. The noise addition process is:

[0016]

[0017] Among them, x0 represents the preprocessed cirrus cloud image, x t-1 represents the cirrus cloud image after the previous noise addition, q(x t |x0) means that x is obtained under the given x0 t The probability of obeying Gaussian distribution, I represents the identity matrix, represents a Gaussian distribution, α t :=1-β t , β t represents the variance of the t-th forward process.

[0018] As a preferred approach, the noise prediction model adopts the UNet network architecture and combines it with the linear focus self-attention mechanism. The output of each position of the linear focus self-attention mechanism is:

[0019]

[0020] Among them, Q i represents the i-th vector in the query matrix Q, K j represents the jth vector in the key matrix K, V j Represents the jth vector in the value matrix, N represents the spatial dimension, and the self-attention function x is Q i or K j, ReLU(·) represents the ReLU activation function, x **p It means raising each element in x to the power of p.

[0021] Preferably, the noise predicted by the noise prediction model is:

[0022]

[0023] Among them, w is the guidance weight, which is used to control the balance between the realism and diversity of the generated images, ∈ θ (x t ,y) is the Gaussian noise predicted by the category information, ∈ θ (x t ) is the noise predicted without category information.

[0024] Preferably, the method for reverse denoising the cirrus cloud image by combining the predicted noise with the posterior probability is:

[0025]

[0026] Among them, x t-1 represents the cirrus image obtained after the current reverse denoising, x t represents the cirrus image obtained after the last reverse denoising, T is the maximum number of reverse denoising times, β t and T are hyperparameters, α t :=1-β t , represents the predicted noise, ∈ represents is a random noise term sampled from a standard normal distribution.

[0027] Preferably, the method further comprises:

[0028] Step 6: Based on the obtained cirrus cloud image, synthesize a remote sensing image dehazing dataset containing real cirrus cloud features:

[0029] Based on the atmospheric diffusion model, the generated cirrus cloud image is used to synthesize the real foggy remote sensing image on the haze-free remote sensing image. The haze synthesis model of the visible light channel j is:

[0030]

[0031] I j (x) represents the real foggy remote sensing image, J j (x) represents the remote sensing image without haze, λ1 represents the central wavelength of reference channel 1, λ j is the central wavelength of channel j, A j Represents the global atmospheric light value;

[0032] t1(x) represents the haze transmittance, t1(x) = 1-ωρ9(x), ρ9(x) represents the haze reflectance of channel nine, that is, the cirrus cloud image obtained. ω∈[0,1] represents the haze concentration;

[0033] γ(x)=a3(ωρ9(x)) 3 +a2(ωρ9(x)) 2 +a1(ωρ9(x))+a0, a0, a1, a2, a3 are coefficients.

[0034] The present invention demonstrates the beneficial effects of capturing the complex spatial structure and optical properties of cirrus cloud images using a diffusion model, generating realistic and diverse cirrus cloud images and constructing a feature-rich remote sensing image dehazing dataset. This not only significantly improves the quality and scale of the dataset, but also provides a unified evaluation standard for remote sensing image dehazing algorithms, further promoting the development of remote sensing image dehazing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a flow chart of the method of the present invention;

[0036] Figure 2 It is the forward process of the non-classified guided diffusion model;

[0037] Figure 3 It is the reverse denoising process of the non-classified guided diffusion model;

[0038] Figure 4 Cirrus cloud images generated for the unclassified guidance diffusion model;

[0039] Figure 5 This is a remote sensing image dehazing dataset made from cirrus images. DETAILED DESCRIPTION

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0041] The cirrus cloud image generation method based on non-classification guidance of this embodiment includes:

[0042] Step 1: Obtain a real cirrus cloud image;

[0043] Specifically, on the Earth Explorer website, 100 locations with different central longitudes and latitudes were selected around the world. Then, eligible Landsat 8 Earth observation satellite Level-1 data was selected, and real cirrus cloud images in Band 9 of the spectral range of 1.360 to 1.390 microns were downloaded from multispectral remote sensing images.

[0044] Step 2: Preprocess the acquired real cirrus cloud images to obtain a cirrus cloud image dataset;

[0045] Specifically, each cirrus cloud image is cropped to a standard 512×512 cirrus cloud image. The images are then screened to select those with clear cirrus cloud features. The screened image X is then normalized to the range [-1, 1] to obtain the preprocessed cirrus cloud image x0. The specific normalization process is calculated using the following formula:

[0046]

[0047] Finally, all normalized data are saved to build a cirrus cloud image dataset.

[0048] Step 3: Use forward diffusion to construct a Markov chain, and gradually add random noise that obeys the Gaussian distribution to the cirrus cloud image in the cirrus cloud image dataset to make it gradually degenerate, so that the cirrus cloud image is noisy to a pure noise image, and finally generate a pure noise image close to the Gaussian distribution. The training dataset is constructed based on the pure noise image and the corresponding real noise. Here, the diffusion step size t is set to 1000, and x0 represents the original real cirrus cloud image, x t It represents the image after t steps of noise addition. The noise addition process is expressed as follows:

[0049]

[0050] The variance of the forward process β t The maximum variance and minimum variance The result of interpolation is calculated as follows:

[0051]

[0052] Where v is the scaling factor, which is set to 0.3. In addition, a significant feature of the forward process is that it can be sampled at any time step t in a closed form. t , using the notation α t :=1-β t and So we have:

[0053]

[0054] That is, given x0, we get x t The probability of following a Gaussian distribution, the mean of which is The variance is I represents the identity matrix. When t=T, x T is pure Gaussian noise, and T is the maximum number of noise additions.

[0055] Step 4: Take the pure noise image as input and the corresponding noise as output, establish a noise prediction model, and train the noise prediction model using the training data set;

[0056] The noise prediction model performs noise prediction on the samples that are finally generated and close to the Gaussian distribution. The noise prediction model of this embodiment can be implemented using the RSCirrusNet model, which generates the predicted noise of the same size as the original cirrus cloud image. The RSCirrusNet model of this embodiment adopts the UNet network architecture and combines it with the linear focus self-attention mechanism. Compared with the traditional self-attention mechanism, the linear focus self-attention mechanism adopted by the RSCirrusNet model of this embodiment reduces the computational complexity from O(N 2 ) is reduced to O(N), which means that an approximation of the original similarity function is used:

[0057] Sim(Q,K)=φ(Q)φ(K) T

[0058] in:

[0059]

[0060] ReLU(·) represents the ReLU activation function, x **p represents raising each element in x to the power of p. The power p modifies the feature orientation of each query and key matrix, bringing similar query-key pairs closer together and making their features more distinct. This simultaneously pushes dissimilar query-key pairs further apart, reducing feature similarity. This restores the sharp attention distribution of the original Softmax function.

[0061] As can be seen, this method can change and adjust the feature direction of each query matrix Q and key matrix K, so that similar query-key pairs are closer and their features are more obvious; at the same time, dissimilar query-key pairs are pushed apart, reducing feature similarity.

[0062] Furthermore, the self-attention mechanism can be rewritten as:

[0063]

[0064] Q i represents the i-th vector in the query matrix Q, K j represents the jth vector in the key matrix K, V j represents the jth vector in the value matrix, N represents the spatial dimension, and x is Q i or K j .

[0065] Thus, this embodiment can change the calculation order from (QK T )V changes to Q(K T V):

[0066]

[0067] At this time, the computational complexity of marking is reduced to O(N).

[0068] In addition, another important factor that limits the expressive power of linear attention is feature diversity. In the traditional Transformer self-attention mechanism calculation process, the attention matrix can reach full rank. In the case of linear attention, due to the limitation of the channel dimension d, that is:

[0069] rank(φ(Q)φ(K) T )≤min{rank(φ(Q)),rank(φ(K))

[0070] ≤min{N,d},

[0071] In the vision problem, d represents the channel dimension, which is usually much smaller than N. This shows that the upper bound of the rank of the attention matrix is ​​limited by the lower channel dimension d, which indicates that many rows of the attention map are severely homogenized. To improve this situation, a simple and effective solution is adopted to address this limitation of linear attention, that is, a deep convolution branch is added when calculating self-attention. At this time, the output can be expressed as:

[0072] O=φ(Q)φ(K) T V+DWC(V)

[0073] It can be further expressed as:

[0074] O=(φ(Q)φ(K) T +M DWC )V=M eq V

[0075] Because M DWC It may be a full-rank matrix, which will effectively improve the upper bound of the attention matrix rank, that is, improve the feature diversity of the self-attention module.

[0076] After the RSCirrusNet model is built, it is trained using the training set and the noised samples x t , time step and category embedding information are input into the RSCirrusNet model, and the current prediction noise can be obtained. For the diffusion model with classification guidance, the conditional generation probability can be logarithmically decomposed using Bayes' theorem to obtain:

[0077]

[0078] As can be seen, the first part on the right side of the equation represents the gradient of the unclassified information generated by the RSCirrusNet model, while the second part represents the gradient of the classifier model. Since this implementation adopts a diffusion model without classification guidance, that is, no additional classification model is required, Further decomposition. First, according to the Bayesian formula:

[0079]

[0080] Then since p(y) does not contain x t , it can be decomposed into:

[0081]

[0082] Substituting this gradient into the gradient guided by the classifier, we get:

[0083]

[0084] Here, w is the guide weight, which is set to 1.8. It can be seen that a conditional generative model and an unconditional generative model are required here, but these two models can be represented by the same model. During training, only the condition of the unconditional model needs to be left blank. Then, a simplified loss evaluation function L is established based on the predicted noise output by the RSCirrusNet model and the sample noise image:

[0085]

[0086] The RSCirrusNet model continuously optimizes and adjusts the model's weight parameters according to the loss function until a converged prediction model is obtained.

[0087] Step 5: Input the randomly generated noise image into the trained noise prediction model. The predicted noise is combined with the posterior probability to perform reverse denoising on the image. After the set number of denoising steps, a cirrus cloud image with real cirrus cloud characteristics is obtained:

[0088] The posterior probability distribution is calculated based on the chain rule of multivariate conditional probability, Bayesian formula, and Gaussian distribution probability density function, and reverse denoising is performed. The posterior probability distribution calculation formula is:

[0089]

[0090] x t-1 represents the cirrus image obtained after the current reverse denoising, x t represents the cirrus cloud image obtained after the last reverse denoising;

[0091] Further according to the implicit diffusion model, we can calculate the random Gaussian noise image x t Under the premise of x t-1 The probability of following the mean is The variance is Gaussian distribution, where:

[0092]

[0093] Reparameterize the mean in the above formula and use x t , To indicate that:

[0094]

[0095] T is the maximum number of reverse denoising, β t and T are hyperparameters, α t :=1-β t , represents the predicted noise, ∈ represents is a random noise term sampled from a standard normal distribution.

[0096] By randomly generating noise data, the trained RSCirrusNet model is used in combination with the posterior probability for reverse denoising. After T cycles, x0 is obtained, and a cirrus cloud image with real texture features can be generated.

[0097] The main purpose of this implementation is to use the generated cirrus cloud images to synthesize a remote sensing image dehazing dataset. Before synthesis, two datasets are required: one containing 11,000 clear, fog-free remote sensing images, and one containing 3,000 multispectral remote sensing images in band 9.

[0098] On this basis, the cirrus cloud image generation method based on non-classification guidance of this embodiment is used to generate a cirrus cloud image with real texture features.

[0099] Furthermore, based on the obtained cirrus cloud images, a remote sensing image defogging dataset containing real cirrus cloud features is synthesized. Currently, atmospheric scattering models are widely used to synthesize foggy remote sensing images, namely:

[0100] I(x)=J(x)·t(x)+A·(1-t(x))

[0101] Where I(x) is the foggy image captured, J(x) is the corresponding fog-free image, A is the global atmospheric illumination value, and t(x) is the scene transmittance. When the atmosphere is homogeneous, t(x) can be generalized as:

[0102] t(x)=e -β·d(x)

[0103] Where β is the atmospheric scattering coefficient and d(x) is the scene depth. In multispectral remote sensing images, the true haze composition depends on the wavelength and haze conditions. Furthermore, since the field of view of a remote sensing image system typically covers a fairly large area, different locations in the perceived scene may experience different haze conditions. Therefore, the atmospheric scattering coefficient can be expressed as:

[0104] β(λ,γ(x))=c0λ -γ(x)

[0105] Where c0 is a constant, λ is the wavelength of the corresponding channel, and the exponent γ(x)∈[0,4] represents the haze condition. At this point, the haze transmittance t(x) can be re-expressed as:

[0106]

[0107] In addition, based on the correlation between channels, the transmittance of one channel with haze can be initialized as a reference value, and then the transmittance of other channels with haze can be further derived. Without loss of generality, the first channel can be set as the reference band. According to the linear relationship between lnt(x) and β(λ,γ(x)), we can further obtain,

[0108]

[0109] Where, λ1 represents the central wavelength of reference channel 1, λ j is the central wavelength of channel j, A j Represents the global atmospheric light value;

[0110] At this point, the haze imaging model of channel j can be expressed as:

[0111]

[0112] Among them, we can directly use λ j Set to its central wavelength, and set the global atmospheric light value A to the average intensity of the brightest 0.01% pixels in each channel of the remote sensing image. In addition, because the reflectance of channel nine, i.e., the generated cirrus cloud image with realistic texture features, can reflect the spatial non-uniformity of actual haze, it is used to generate the transmission map of channel 1, which is expressed as:

[0113] t1(x)=1-ωρ9(x)

[0114] Here, ρ9(x) represents the haze reflectance of channel nine, i.e., the normalized cirrus cloud image, and ω∈[0,1] represents the haze concentration. A larger ω value results in a smaller transmittance value, corresponding to a more severe haze condition. The final parameter γ(x) of the synthetic hazy remote sensing image is used to describe the non-uniform spatial distribution of haze concentration. Based on the relationship between the exponent γ(x) and the haze reflectance ρ, a cubic curve can be used for fitting, and its formula can be expressed as:

[0115] γ(x)=a3(ωρ9(x)) 3 +a2(ωρ9(x)) 2 +a1(ωρ9(x))+a0

[0116] Among them, a0, a1, a2, and a3 are coefficients. In this embodiment, a0 = 6.537, a1 = -27.465, a2 = 41.224, and a3 = -21.547. It should be noted that γ(x) should be limited to [0, 4] to avoid abnormal values. Through the above parameterization method, the present invention can obtain the haze-free remote sensing image J j Synthesize a real foggy remote sensing image I j .

[0117] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be employed in conjunction with other described embodiments.

Claims

1. A cirrus cloud image generation method based on non-classification guidance, characterized in that: include: Step 1: Obtain a real cirrus cloud image; Step 2: Preprocess the acquired real cirrus cloud images to obtain a cirrus cloud image dataset; Step 3: Use forward diffusion to construct a Markov chain and gradually add random noise that follows a Gaussian distribution to the cirrus cloud images in the cirrus cloud image dataset, so that the cirrus cloud images are noisy to pure noise images. Then, a training dataset is constructed based on the pure noise images and the corresponding real noise. Step 4: Take the pure noise image as input and the corresponding noise as output, establish a noise prediction model, and train the noise prediction model using the training data set; Step 5: Input the randomly generated noise image into the trained noise prediction model. The predicted noise is combined with the posterior probability to perform reverse denoising on the image. After the set number of denoising steps, a cirrus cloud image with real cirrus cloud characteristics is obtained. Step 6: Based on the obtained cirrus cloud image, synthesize a remote sensing image dehazing dataset containing real cirrus cloud features: Based on the atmospheric diffusion model, the generated cirrus cloud image is used to synthesize the real foggy remote sensing image on the haze-free remote sensing image. The haze synthesis model of the visible light channel j is: I j (x) represents the real foggy remote sensing image, J j (x) represents the remote sensing image without haze, λ1 represents the central wavelength of reference channel 1, λ j is the central wavelength of visible light channel j, A j Represents the global atmospheric light value; t1(x) represents the haze transmittance, t1(x) = 1-ωρ9(x), ρ9(x) represents the haze reflectance of channel nine, that is, the cirrus cloud image obtained, ω∈[0,1] represents the haze concentration; γ(x)=a3(ωρ9(x)) 3 +a2(ωρ9(x)) 2 +a1(ωρ9(x))+a0, a0, a1, a2, a3 are coefficients.

2. The cirrus cloud image generation method based on non-classification guidance according to claim 1, characterized in that: In step 1, a real cirrus cloud image in the aerosol band with a spectral range of 1.360 to 1.390 μm is obtained.

3. The cirrus cloud image generation method based on non-classification guidance according to claim 1, characterized in that: In step 2, the acquired real cirrus cloud image is preprocessed, including: Each real cirrus cloud image is cropped, and the cropped images are filtered to select images with clear cirrus clouds and obvious features. The filtered images X are normalized to obtain the preprocessed cirrus cloud image x0:

4. The method for generating cirrus cloud images based on non-classified guidance according to claim 1, characterized in that: In step 3, random noise that obeys Gaussian distribution is gradually added to the cirrus cloud images in the cirrus cloud image dataset, x t It represents the cirrus cloud image after t steps of noise addition. The noise addition process is: Among them, x0 represents the preprocessed cirrus cloud image, x t-1 represents the cirrus cloud image after the previous noise addition, q(x t |x0) means that x is obtained under the given x0 t The probability of obeying Gaussian distribution, I represents the identity matrix, represents a Gaussian distribution, α t :=1-β t , β t represents the variance of the t-th forward process.

5. The method for generating cirrus cloud images based on non-classified guidance according to claim 1, characterized in that: The noise prediction model adopts the UNet network architecture and combines it with the linear focus self-attention mechanism. The output of each position of the linear focus self-attention mechanism is: Among them, Q i represents the i-th vector in the query matrix Q, K j represents the jth vector in the key matrix K, V j represents the jth vector in the value matrix, N represents the spatial dimension, and the self-attention function φ p (x) = f p (ReLU(x)), x is Q i or K j , ReLU(·) represents the ReLU activation function, x **p It means raising each element in x to the power of p.

6. The method for generating cirrus cloud images based on non-classified guidance according to claim 5, characterized in that: The noise predicted by the noise prediction model is: Among them, w is the guidance weight, which is used to control the balance between the realism and diversity of the generated images, ∈ θ (x t ,y) is the Gaussian noise predicted by the category information, ∈ θ (x t ) is the noise predicted without category information.

7. The method for generating cirrus cloud images based on non-classified guidance according to claim 1, characterized in that: The method of reverse denoising the cirrus image by combining the predicted noise with the posterior probability is as follows: Among them, x t-1 represents the cirrus image obtained after the current reverse denoising, x t represents the cirrus image obtained after the last reverse denoising, T is the maximum number of reverse denoising times, β t and T are hyperparameters, α t :=1-β t , represents the predicted noise, ∈ represents is a random noise term sampled from a standard normal distribution.

Images