High-precision spectral remote sensing atmospheric correction and cloud removal processing method fusing multiple time phases, atmospheric parameters and ground reflectivity

Abstract

The invention discloses a high-precision spectral remote sensing atmospheric correction and cloud removal processing method fusing multiple time phases, atmospheric parameters and ground reflectivity, and relates to the technical field of computer vision, deep learning image processing and remote sensing. Comprising the following steps: acquiring an atmosphere top reflectance and surface reflectance model of a Sentinel-2 satellite remote sensing image in a Guilin region, and constructing a data set by atmosphere data; constructing a diffusion model for multispectral remote sensing image processing; constructing a condition control and coding structure to carry out multispectral parameter and atmospheric parameter processing; and establishing a unified model to carry out atmospheric correction and cloud removal processing on the spectral satellite remote sensing image. According to the method, multi-temporal multi-spectral parameters, atmospheric aerosol optical thickness, atmospheric water vapor and ground reflectivity remote sensing characteristic parameters and data are fully fused, and a brand new unified model is established to realize atmospheric correction and cloud removal processing of a high-precision spectral satellite remote sensing image; and atmospheric correction and cloud removal processing of the high-precision spectrum satellite remote sensing image are realized.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision, deep learning image processing and remote sensing technology, and more specifically to a high-precision spectral remote sensing atmospheric correction and cloud removal processing method that integrates multi-temporal phases, atmospheric parameters and ground reflectivity. Background Art

[0002] Atmospheric aerosols and clouds are the core factors that affect the acquisition of high-quality satellite remote sensing image data. Therefore, accurate atmospheric correction and declouding of satellite remote sensing images are the most critical prerequisite steps for developing satellite remote sensing applications.

[0003] At present, atmospheric correction and cloud removal methods for satellite remote sensing images are mainly divided into two categories: traditional remote sensing model methods and deep learning methods. Traditional remote sensing model algorithms usually involve complex mathematical models of atmospheric radiation transmission, complex meteorological environmental parameters, and matching ground synchronous measurement data and related basic geographic data. Therefore, it is extremely difficult and limited to complete a detailed description of ground and atmospheric patterns. Traditional deep learning methods usually use computer vision methods, but these methods cannot effectively utilize important atmospheric and multispectral characteristics and ground reflectivity parameters and models, and therefore cannot significantly improve the data quality of spectral satellite remote sensing images.

[0004] Therefore, how to propose a high-precision spectral remote sensing atmospheric correction and cloud removal processing method that integrates multi-temporal, atmospheric parameters and ground reflectivity, fully integrate multi-temporal and multi-spectral parameters, atmospheric parameters (aerosol optical thickness and water vapor) and remote sensing characteristic parameters and data such as ground reflectivity, establish a new unified model, and realize atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a high-precision spectral remote sensing atmospheric correction and cloud removal processing method that integrates multi-temporal, atmospheric parameters and ground reflectivity, fully integrates multi-temporal multi-spectral parameters, atmospheric parameters (aerosol optical thickness and water vapor) and remote sensing characteristic parameters and data such as ground reflectivity, establishes a new unified model, and realizes atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images. In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0006] A high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectance, comprising:

[0007] Obtain the top of atmosphere reflectance, surface reflectance model and atmospheric data of Sentinel-2 satellite remote sensing images of Guilin area to build a dataset;

[0008] Constructing diffusion models for multispectral remote sensing image processing;

[0009] Construct condition control and coding structures for multi-spectral and atmospheric parameter processing;

[0010] A unified model is established to perform atmospheric correction and cloud removal on high-precision spectral satellite remote sensing images.

[0011] Optionally, use Sentinel-2 satellite remote sensing images, and use the images of the B2 to B8A, B11 and B12 bands in the Sentinel-2L1C product to resample to a spatial resolution of 60 meters using the nearest pixel upsampling method and the Average downsampling method; use the sen2cor tool to invert the atmospheric optical thickness and water vapor content data based on the Dense Dark Vegetation algorithm and the Atmospheric Pre-corrected Differential Absorption algorithm as additional input conditions; use the L2A product as the standard value GroundTruth for model atmospheric correction.

[0012] Optionally, collect 3000 images for atmospheric correction and 500 sets of images for cloud removal, with each image having a size of 512x512 pixels;

[0013] Atmospheric correction data include atmospheric top reflectance TOA, surface reflectance SR, atmospheric aerosol optical thickness AOT and water vapor WVP; cloud removal data include multi-temporal panchromatic images and cloud masks. The data are divided into training set and validation set in a ratio of 8:2 for model training and evaluation respectively.

[0014] All 12 bands of Sentinel-2 images, true color images, and 5 to 30 images with a size of 512pix × 512pix cropped from random areas on AOD and WVP data were used as the data set;

[0015] The random cropping operation skips the image boundary area, and the cropped area includes complete reflectance, atmospheric aerosol optical thickness AOT and water vapor WVP data.

[0016] Optionally, the diffusion model is a DDPM denoising diffusion model. In the diffusion model, two processes are included: a forward process and a reverse process, wherein the forward process is also called a diffusion process. Both processes are implemented based on a parameterized Markov chain and are used to represent a random transition process from one state to another in a state space.

[0017] The forward process adds noise to the sample image and transforms the distribution of the original sample data x0 into a standard Gaussian distribution x T,Through multiple iterations, the noise gradually propagates and diffuses throughout the whole sample;

[0018] The reverse process is used to generate data samples. By performing several steps of denoising operations on the standard Gaussian distribution data, it is gradually restored to the distribution of the real sample data. The goal of the reverse process is to reversely trace the path of noise addition and gradually restore the Gaussian distribution data to the distribution of the original sample data. The samples generated by the reverse process are used to evaluate the performance of the denoising algorithm.

[0019] Optionally, the diffusion process is to gradually add Gaussian noise to the image data until the content of the input image data completely becomes random noise. The forward process, i.e., the diffusion process, is to accumulate the input data x at each step T times. i In the process of adding Gaussian noise, the diffusion process of DDPM is expressed as:

[0020]

[0021] Optionally, the reverse process of the diffusion model is to use a neural network to infer the noise, train the neural network to infer the distribution after adding noise, and restore the data sampled from the standard Gaussian distribution back to the distribution of the original sample data. The reverse diffusion process is expressed as follows:

[0022]

[0023] Optionally, in the diffusion process of the DDPM denoising diffusion model, the image is gradually converted into random noise, and the image feature representation gradually approaches the Gaussian distribution. The DDPM denoising diffusion model uses Markov chains to model the evolution of the image. Markov chains are a random process that describes the transition process of an image from one state to another. Neural networks are used to represent the probability distribution of image features. The UNet structure is used to extract and reconstruct image data features, and additionally a set of conditions, such as time steps and reference images, are given. The conditional generation process relies on neural network training;

[0024] The diffusion model is trained by neural network to obtain the mean μ Θ (X t ,t) and variance ∑ Θ (X t ,t), the DDPM denoising diffusion model obtains its variational upper bound L through variational inference of the maximum likelihood estimate. The variational upper bound is obtained using KL divergence. The optimization method based on the variational upper bound of the model is expressed as:

[0025]

[0026] Optionally, the diffusion model builds image generation capabilities based on denoising and restoration processes in image tasks;

[0027] The conditional guidance method of the conditional diffusion model without classifier guidance is used to design the conditional guidance of aerosol and spectral data for the model during the training of the neural network model. The reverse process p(X t-1 |X t ) The original image is inferred through the Gaussian noise image. If y is used as the model condition, the reverse process is t and y, and infer X t-1 , the formula is as follows: p(X t-1 |X t ,y).

[0028] Optionally, the reverse denoising process of the diffusion model uses a UNet-like neural network encoder to predict the noise distribution based on the noise image and the time step guidance condition. UNet is used for image semantic segmentation and image generation tasks, and has an encoding-decoding structure. Through the process of downsampling and upsampling, it extracts features and retains the large-scale information of the image;

[0029] The structure of UNet is divided into the left part and the right part. The left part is the encoding process, which is used for downsampling and feature extraction, reducing the spatial size of the feature map while retaining important features. While extracting features, the image size is gradually reduced; the right part is the decoding process, which is used for upsampling and feature fusion. Upsampling operations are performed at each level. After each upsampling operation, the obtained feature map is superimposed and connected with the corresponding feature map on the left to fuse multi-scale feature information, restore the image size, and increase the number of channels.

[0030] The UNet network contains an output layer that uses a convolutional network module to adjust the output structure to match the format of the target output.

[0031] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a high-precision spectral remote sensing atmospheric correction and cloud removal processing method that integrates multiple temporal phases, atmospheric parameters and ground reflectivity, which has the following beneficial effects:

[0032] The present invention proposes a high-precision spectral remote sensing atmospheric correction and cloud removal processing method that integrates multi-temporal, atmospheric parameters and ground reflectivity, including: obtaining the atmospheric top reflectivity, surface reflectivity model and atmospheric data of Sentinel-2 satellite remote sensing images in Guilin area to construct a data set; constructing a diffusion model for multi-spectral remote sensing image processing; constructing a conditional control and coding structure to process multi-spectral parameters and atmospheric parameters; and establishing a unified model to perform atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images. The present invention fully integrates multi-temporal multi-spectral parameters, atmospheric aerosol optical thickness, atmospheric water vapor and ground reflectivity and other remote sensing characteristic parameters and data, and establishes a new unified model to realize atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images, thereby realizing atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images. By redesigning the reverse denoising and conditional guidance network structure of the model, a unified neural network model is used to process atmospheric correction and cloud removal, and the effective fusion of key factors of image remote sensing data quality such as multi-spectral and aerosol is realized. The method of the present invention completes the accuracy evaluation on Sentinel-2 multi-spectral remote sensing image data. In the atmospheric correction task of full-color images. Compared with other models and methods, the method adopted by the present invention achieves the best evaluation results. The present invention selects the multispectral satellite remote sensing images of the Guilin area with unique and diverse geographical environment and complex atmospheric conditions as the research object. The satellite remote sensing data contains a wealth of research characteristics that can be used to realize atmospheric correction and cloud removal tasks, providing a typical and representative experimental scene for the present invention. The present invention first uses the top of the atmosphere reflectance and surface reflectance model and data of Sentinel-2 in the Guilin area to create four data sets: true color and multispectral fusion data sets after atmospheric correction containing aerosol parameters, and single-phase and multi-phase cloud removal data sets. The present invention realizes a brand-new conditional guided diffusion model applicable to multispectral parameters and atmospheric parameters by redesigning the conditional control and encoding parts. The method of the present invention achieved significantly excellent evaluation results after completing the atmospheric correction and cloud removal processing of true color and multispectral. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 A data sampling time distribution diagram provided by the present invention;

[0035] Figure 2 The overall flow chart of the experiment provided by the present invention;

[0036] Figure 3 A diagram of the steps for making a data set provided by the present invention;

[0037] Figure 4 A flow chart of the model application provided by the present invention;

[0038] Figure 5 The overall result diagram of the model provided by the present invention;

[0039] Figure 6 A diagram showing the processing results of the atmospheric correction model provided by the present invention;

[0040] Figure 7 This is a diagram of the cloud removal model processing results provided by the present invention;

[0041] Figure 8 A schematic diagram showing the difference between the atmospheric correction of the urban area model provided by the present invention and the true value;

[0042] Fig. 9 A schematic diagram showing the difference between the atmospheric correction of the vegetation coverage area model provided by the present invention and the true value;

[0043] Fig.10 This is a schematic diagram comparing the difference between the atmospheric correction and the true value of the water body area model provided by the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The embodiment of the present invention discloses a high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multiple temporal phases, atmospheric parameters and ground reflectivity, comprising:

[0046] Acquire the top of atmosphere reflectance, surface reflectance and atmospheric data of Sentinel-2 satellite remote sensing images of Guilin area to construct a dataset;

[0047] Constructing diffusion models for multispectral remote sensing image processing;

[0048] Construct condition control and coding structures for multi-spectral and atmospheric parameter processing;

[0049] A unified model is established to perform atmospheric correction and cloud removal on high-precision spectral satellite remote sensing images.

[0050] In a specific implementation, a high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal, atmospheric parameters and ground reflectivity is proposed, which fully integrates remote sensing characteristic parameters and data such as multi-temporal multi-spectral parameters, atmospheric aerosol optical thickness, atmospheric water vapor and ground reflectivity, and establishes a new unified model to realize the atmospheric correction and cloud removal processing method of high-precision spectral satellite remote sensing images, and the steps include:

[0051] Obtain the top of atmosphere reflectivity and surface reflectivity model and atmospheric data of Sentinel-2 in Guilin area to build a dataset;

[0052] Construct a diffusion model designed for multispectral remote sensing image processing;

[0053] Build new condition control and encoding parts to realize new processing applicable to multi-spectral parameters and atmospheric parameters;

[0054] A new unified model was established to realize atmospheric correction and cloud removal of high-precision spectral satellite remote sensing images.

[0055] Furthermore, using Sentinel-2 satellite remote sensing images, the images of the B2 to B8A, B11 and B12 bands in the Sentinel-2L1C product were resampled to a spatial resolution of 60 meters using the Nearest Neighbour upsampling method and the Average downsampling method. The sen2cor tool was used to invert the AOT (atmospheric optical thickness) and WVP (water vapor content) data based on the Dense Dark Vegetation algorithm and the Atmospheric Pre-corrected Differential Absorption algorithm (APDA) method as additional input conditions. The L2A product was used as the standard value Ground Truth for model atmospheric correction.

[0056] Furthermore, about 3,000 images were collected for atmospheric correction and about 500 sets of images were collected for cloud removal. The size of each image is 512x512 pixels. The atmospheric correction data includes TOA (top of atmosphere reflectance), SR (surface reflectance), AOT (aerosol optical thickness) and WVP (water vapor). The cloud removal data includes multi-temporal panchromatic images and cloud masks. These data are divided into training sets and validation sets in a ratio of 8:2 for model training and evaluation respectively. 5 to 30 images with a size of 512pix×512pix are cropped from random areas of all 12 bands of Sentinel-2 images, true color images, and AOD and WVP data as the data set. The random cropping operation skips the image boundary area, so the cropped area includes complete reflectance, AOT and WVP data.

[0057] Furthermore, the basic model of the diffusion model, DDPM (Denoising Diffusion Probabilistic Models), was proposed in 2020 and has been widely used in different tasks in the field of computer vision, such as text image generation and image restoration. The diffusion model is similar to other generative models, such as GAN (Generative Adversarial Network) or VAE (Variational Auto-Encoder), which generates target data samples from random noise (DDPM uses Gaussian noise). In the diffusion model, there are two processes: the forward process and the reverse process, where the forward process is also called the diffusion process. Both processes are based on parameterized Markov chains, which are used to represent the random transition process from one state to another in the state space. The forward process adds noise to the sample image. This process gradually transforms the distribution of the original sample data x0 into a standard Gaussian distribution x T . Through multiple iterations, the noise gradually propagates and diffuses throughout the sample. The reverse process can be used to generate data samples. It performs several steps of denoising operations on the standard Gaussian distribution data to gradually restore it to the distribution of the real sample data. The goal of the reverse process is to reversely trace the path of noise addition and gradually restore the Gaussian distribution data to the distribution of the original sample data. The samples generated by the reverse process can be used to evaluate the performance of the denoising algorithm or perform other related tasks.

[0058] Furthermore, the specific operation of the diffusion process is to gradually add Gaussian noise to the image data until the content of the input image data completely becomes random noise. The forward process, i.e., the diffusion process, accumulates the input data x at each step T times. i The process of adding Gaussian noise, the diffusion process of DDPM can be expressed as:

[0059]

[0060] Furthermore, the reverse process of the diffusion model uses a neural network to infer noise. By training the neural network to infer the distribution after adding noise, this study can restore the data sampled from the standard Gaussian distribution back to the distribution of the original sample data. In the reverse diffusion process, it is specifically expressed as follows:

[0061]

[0062] Furthermore, in the diffusion process of DDPM (denoising diffusion model), the image is gradually converted into random noise, and the image feature representation gradually approaches the Gaussian distribution. DDPM uses Markov chain to model the evolution of the image. Markov chain is a random process that describes the transition process of the image from one state to another. A neural network is used to represent the probability distribution of image features. The UNet structure is generally used to extract and reconstruct image data features, and additionally, by giving a set of conditions, such as time step and reference image, the conditional generation (denoising) process depends on neural network training. The diffusion model obtains a suitable mean μ through neural network training. Θ (X t ,t) and variance ∑ Θ 9X t ,t). The model uses variational inference to obtain the variational upper bound L of the maximum likelihood estimate. The variational upper bound is obtained using KL divergence. The optimization method of the model based on the variational upper bound can be expressed as:

[0063]

[0064] Furthermore, the diffusion model obtains image generation capabilities based on the denoising and restoration process design in the image task. However, because the original image is diffused into Gaussian noise in the ordinary diffusion model, its original information is almost completely lost, and the model output is random. This cannot meet the requirements of image atmospheric correction. This study uses the Classifier-Free GuidanceDiffusionModel (conditional diffusion model without classifier guidance) conditional guidance method. When training the neural network model, the conditional guidance of aerosol and spectral data is designed for the model. The reverse process p(X t-1 |X t ) can be used to infer the original image through the Gaussian noise image. If y is used as the model condition, the reverse process becomes t and y, and infer X t-1 , the formula is as follows:

[0065] p(X t-1 |X t ,y).

[0066] Furthermore, the denoising model constructs the reverse denoising process of the usual diffusion model using a UNet-like neural network encoding to predict the noise distribution based on the guiding conditions such as the noise image and the time step. UNet is a commonly used convolutional neural network structure, which is mainly used for image semantic segmentation and image generation tasks. It has an "encoding-decoding" structure, which can extract features and retain large-scale information of the image through the process of downsampling and upsampling. The structure of UNet is divided into a left part and a right part. The left part is the encoding process, which is used for downsampling and feature extraction. Reduce the spatial size of the feature map while retaining important features. In this way, the image size can be gradually reduced while extracting features. The right part is the decoding process, which is mainly used for upsampling and feature fusion. Upsampling operations are performed at each level. After each upsampling, the obtained feature map is superimposed and connected with the corresponding feature map on the left to fuse multi-scale feature information. In this way, the size of the image can be gradually restored and the number of channels can be increased. Finally, the UNet network usually contains an output layer, and a convolutional network module is used to adjust the output structure to match the format of the target output.

[0067] In a specific embodiment, a high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectivity comprises the following steps:

[0068] 1) Use Sentinel-2 satellite remote sensing image data to build a dataset;

[0069] 2) Construct a diffusion model to perform atmospheric correction and cloud removal on remote sensing images;

[0070] 3) Design a conditional control and encoding method suitable for multispectral and atmospheric parameters so that the model can be guided by atmospheric and multispectral conditions;

[0071] 4) Use unified diffusion model training to achieve atmospheric correction and cloud removal of high-precision spectral satellite remote sensing images.

[0072] Furthermore, the step 1) uses Sentinel-2 satellite remote sensing images, and uses the images of the B2 to B8A, B11 and B12 bands in the Sentinel-2L1C product to be resampled to a spatial resolution of 60 meters using the nearest neighbor upsampling method and the average downsampling method. The sen2cor tool is used to invert the AOT (atmospheric optical thickness) and WVP (water vapor content) data based on the DenseDark Vegetation algorithm and the Atmospheric Pre-corrected Differential Absorption algorithm (APDA) method as additional input conditions. The L2A product is used as the standard value Ground Truth for model atmospheric correction. The data set collects about 3,000 images for atmospheric correction and about 500 groups of images for cloud removal. The size of each image is 512x512 pixels. The atmospheric correction data includes TOA (atmospheric top reflectance), SR (surface reflectance), as well as AOT (atmospheric aerosol optical thickness) and WVP (water vapor). The cloud removal data includes multi-temporal panchromatic images and cloud masks. These data are divided into training and validation sets in a ratio of 8:2 for model training and evaluation respectively. 5 to 30 images with a size of 512pix×512pix are randomly cropped from all 12 bands of Sentinel-2 images, true color images, and AOD and WVP data as the data set. The random cropping operation skips the image boundary area, so the cropped area includes complete reflectance, AOT and WVP data.

[0073] Furthermore, the step 2) constructs a diffusion model to perform atmospheric correction and cloud removal of remote sensing images, including the following steps: ① construct a downsampling module of UNet; ② construct an upsampling module of UNet; ③ construct a conditional guidance module; ④ determine the model training strategy and loss. A convolution module is used in the input part of the encoding network to process the upper layer input. A deep network is used to process time embedding. The downsampling module adds a residual and attention mechanism module. The features are then output through the downsampling module. The upsampling module uses the Features Decode module to fuse the upsampling results of the previous step and the features of the jump connection of the encoding part, and uses the Conv2DTranspose module for upsampling. The DDPM model adjusts the network prediction noise instead of the distribution mean, and uses the MSE (Mean Square Error) of the posterior Gaussian noise and the model prediction noise as the loss.

[0074] Furthermore, the step 3) designs conditional control and encoding methods, and adds a conditional encoding module to the model. This module accepts multispectral remote sensing images and aerosol parameters as external guidance conditions. The conditional encoding module uses a three-layer deep jump convolution module to learn features, and splices the conditional features with the model backbone network through a downsampling module.

[0075] Furthermore, the step 4) uses a new unified model, uses the data set constructed in step 1), and uses the same neural network to train and implement atmospheric correction and cloud removal processing of high-precision spectral satellite remote sensing images, including the following steps: ① Training the atmospheric correction data set of multispectral remote sensing images to implement the atmospheric correction model; ② Using the cloud removal data set to train the remote sensing image cloud removal model; ③ Using the Sentinel-2TOA image input step 1 model to obtain the ground reflectance and then input step 2 model to obtain an image of a certain area in Guilin that has completed atmospheric correction and cloud removal.

[0076] The present invention constructs a brand-new atmospheric correction and cloud removal data set that integrates multispectral satellite remote sensing images and atmospheric parameters; compared with existing atmospheric correction and cloud removal methods, the present invention significantly improves the atmospheric correction and cloud removal effects; the present invention proposes a unified model that can obtain more accurate atmospheric correction results and more effective cloud removal results after integrating multispectral remote sensing images and atmospheric parameters, verifying the effectiveness and innovation of the present invention.

[0077] In a specific embodiment, the geographical coordinates of Guilin are N25°16′35.35″N and E110°17′7.52″E. Its total area is approximately 27,700 square kilometers. The core part of the urban area covers an area of ​​approximately 866 square kilometers. Guilin has a diverse topography, mainly consisting of hilly terrain and a series of small intermountain basins, while the urban area is located on a relatively large plain in the Hunan-Guangxi Corridor. The southern part of the urban area is a fragmented hilly plain, while the west, north and east extend to vast mountains. There are five rivers such as the Lijiang River crossing Guilin. The average annual temperature in Guilin is about 19.3°C, the average annual rainfall is as high as 1949.5 mm, the average evaporation fluctuates between 1490 and 1905 mm, and the average annual relative humidity is maintained between 73% and 79%. The area is often affected by high humidity and heavy precipitation, resulting in common smog, heavy cloud cover and other problems. Rainfall, fog, smog and seasonal weather changes significantly affect the availability of remote sensing images. Therefore, in-depth research on remote sensing image dehazing technology in this area is crucial for the effective analysis of image data.

[0078] The data collection time range of this embodiment is from January 2018 to September 2023. And the Sentinel-2 satellite data L1C product with cloud cover not exceeding 20% ​​is selected. After screening, 170 days of image data are obtained. The specific time distribution is as follows: Figure 1 shown.

[0079] This example designs a conditional guided diffusion model for processing multispectral remote sensing images based on Sentinel-2 remote sensing images. The goal of the model is to effectively complete the two tasks of multispectral image atmospheric correction and cloud removal at the same time. The specific process is as follows: Figure 2 As shown in Figure 1. Different from the common computer vision defogging task, this study adds aerosol data as an image generation condition in the remote sensing image defogging task. The goal of the model in remote sensing image processing is to obtain the real ground reflectivity information based on the remote sensing image and other atmospheric information. The model further removes clouds from the remote sensing image and uses the model to reconstruct the ground object information lost by the cloud. The complete process of the experiment is shown in Figure 1. Figure 3 As shown in the figure, the process includes the following steps: (1) data acquisition; (2) data set construction; (3) neural network model construction; (4) model training and result evaluation.

[0080] In a specific embodiment, in order to study the effect of applying aerosol information as auxiliary data to the atmospheric correction and cloud removal effect of the deep learning model, several data sets were constructed. After obtaining the Sentinel-2 remote sensing image products, the process of constructing the data set consists of three steps: (1) preliminary preprocessing and image mosaicking; (2) image cropping; and (3) data set formatting. The overall process of data processing and data set construction is as follows: Figure 4 The final dataset contains two categories and four sets of data, namely (1) Dataset 1: atmospheric correction of remote sensing images assisted by aerosol data, including two sets of data: panchromatic images and multispectral ground reflectance. Its purpose is to train the neural network model to output high-availability ground reflectance images; (2) Dataset 2: multi-temporal remote sensing image cloud removal, its purpose is to train the neural network model to output high-fidelity cloud-occluded area objects.

[0081] In a specific embodiment, the encoder-decoder structure based on UNet is used. Multispectral remote sensing images and atmospheric data are used as guiding conditions and a new denoising model is designed. The model designs a new encoder-decoder module based on the UNet model. The model also adds a conditional guiding path. The overall structure of the model is as follows Figure 5 As shown, the model mainly includes the following modules:

[0082] Conditional guidance module: In order to achieve the remote sensing image defogging task under conditional guidance, a conditional encoding module is added to the model. This module accepts multispectral remote sensing images and aerosol parameters as external guidance conditions. The conditional encoding module uses a three-layer deep jump convolution module to learn features, and splices the conditional features with the model backbone network through a downsampling module.

[0083] Downsampling module: A convolutional module is used in the input part of the encoding network to process the upper layer input. A deep network is used to process the time embedding. The downsampling module adds residual and attention mechanism modules. The features are then output through the downsampling module.

[0084] Upsampling module: The upsampling module uses the Features Decode module to fuse the upsampling result of the previous step and the features of the jump connection of the encoding part, and uses the Conv2DTranspose module for upsampling.

[0085] The present invention has completed the accuracy evaluation on Sentinel-2 multispectral remote sensing image data. The specific result diagram is as follows Figures 6 to 10 As shown, Figure 6 This is a graph of the atmospheric correction model processing results provided by the present invention. Figure 7 This is a diagram of the cloud removal model processing results provided by the present invention. Figure 8 This is a schematic diagram comparing the difference between the atmospheric correction of the urban area model provided by the present invention and the true value. Fig. 9 This is a schematic diagram comparing the atmospheric correction of the vegetation coverage area model provided by the present invention and the true value. Fig.10 Schematic diagram of the comparison between the atmospheric correction of the water body area model provided by the present invention and the true value. In the atmospheric correction task of panchromatic image, the MAE, SSIM and PSNR indicators of the present invention reached 0.0125, 0.9445 and 35.5221dB respectively. In the atmospheric correction task of multispectral image, the MAE, SSIM and PSNR indicators of the present invention reached 0.0174, 0.9499 and 36.3547dB respectively. The MAE, SSIM and PSNR indicators in the multi-temporal panchromatic image cloud removal task reached 0.0224, 0.9556 and 36.2122dB respectively. Compared with other models and methods, the method adopted by the present invention achieves the best evaluation results.

[0086] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts refer to the method part description. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

Claims

1. A high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectivity, characterized in that: include: Acquire the top of atmosphere reflectance, surface reflectance and atmospheric data of Sentinel-2 satellite remote sensing images of Guilin area to construct a dataset; Constructing diffusion models for multispectral remote sensing image processing; Construct condition control and coding structures for multi-spectral and atmospheric parameter processing; A unified model is established to perform atmospheric correction and cloud removal on high-precision spectral satellite remote sensing images.

2. According to claim 1, a high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectivity is characterized in that: Sentinel-2 satellite remote sensing images were used. The images of the B2 to B8A, B11 and B12 bands in the Sentinel-2L1C product were resampled to a spatial resolution of 60 meters using the nearest pixel upsampling method and the Average downsampling method. The sen2cor tool was used to invert the atmospheric optical thickness and water vapor content data based on the Dense Dark Vegetation algorithm and the Atmospheric Pre-corrected Differential Absorption algorithm as additional input conditions. The L2A product was used as the standard value Ground Truth for model atmospheric correction.

3. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method according to claim 1, which integrates multiple temporal phases, atmospheric parameters and ground reflectivity, is characterized in that: 3000 images were collected for atmospheric correction and 500 sets of images were collected for cloud removal. The size of each image was 512x512 pixels. Atmospheric correction data include atmospheric top reflectance TOA, surface reflectance SR, atmospheric aerosol optical thickness AOT and water vapor WVP; cloud removal data include multi-temporal panchromatic images and cloud masks. The data are divided into training set and validation set in a ratio of 8:2 for model training and evaluation respectively. All 12 bands of Sentinel-2 images, true color images, and 5 to 30 images with a size of 512pix × 512pix cropped from random areas on AOD and WVP data were used as the data set; The random cropping operation skips the image boundary area, and the cropped area includes complete reflectance, atmospheric aerosol optical thickness AOT and water vapor WVP data.

4. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method according to claim 1, which integrates multiple temporal phases, atmospheric parameters and ground reflectivity, is characterized in that: The diffusion model is the DDPM denoising diffusion model. In the diffusion model, there are two processes: the forward process and the reverse process. The forward process is also called the diffusion process. Both processes are implemented based on parameterized Markov chains to represent the random transition process from one state to another in the state space. The forward process adds noise to the sample image and transforms the distribution of the original sample data x0 into a standard Gaussian distribution x T ,Through multiple iterations, the noise gradually propagates and diffuses throughout the whole sample; The reverse process is used to generate data samples. By performing several steps of denoising operations on the standard Gaussian distribution data, it is gradually restored to the distribution of the real sample data. The goal of the reverse process is to reversely trace the path of noise addition and gradually restore the Gaussian distribution data to the distribution of the original sample data. The samples generated by the reverse process are used to evaluate the performance of the denoising algorithm.

5. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method according to claim 4, which integrates multiple temporal phases, atmospheric parameters and ground reflectivity, is characterized in that: The diffusion process is to gradually add Gaussian noise to the image data until the content of the input image data completely becomes random noise. The forward process, i.e., the diffusion process, is to accumulate the input data x at each step through T times. i In the process of adding Gaussian noise, the diffusion process of DDPM is expressed as:

6. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method according to claim 4, which integrates multiple temporal phases, atmospheric parameters and ground reflectivity, is characterized in that: The reverse process of the diffusion model is to use a neural network to infer the noise. By training the neural network to infer the distribution after adding noise, the data sampled from the standard Gaussian distribution is restored to the distribution of the original sample data. The reverse diffusion process is expressed as follows:

7. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectivity according to claim 4 is characterized in that: In the diffusion process of the DDPM denoising diffusion model, the image is gradually converted into random noise, and the image feature representation gradually approaches the Gaussian distribution. The DDPM denoising diffusion model uses Markov chains to model the evolution of images. Markov chains are a random process that describes the transition process of an image from one state to another. Neural networks are used to represent the probability distribution of image features. The UNet structure is used to extract and reconstruct image data features, and additionally, a set of conditions, such as time steps and reference images, are given. The conditional generation process relies on neural network training; The diffusion model is trained by neural network to obtain the mean μ Θ (X t , t) and variance ∑ Θ (Xt, t), the DDPM denoising diffusion model obtains its variational upper bound L through variational inference of the maximum likelihood estimate. The variational upper bound is obtained using KL divergence. The optimization method based on the variational upper bound of the model is expressed as:

8. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multiple temporal phases, atmospheric parameters and ground reflectivity according to claim 4 is characterized in that: The DDPM denoising diffusion model builds image generation capabilities based on denoising and restoration processes in image tasks; The conditional guidance method of the conditional diffusion model without classifier guidance is used to design the conditional guidance of aerosol and spectral data for the model during the training of the neural network model. The reverse process p(X t-1 |X t ) The original image is inferred through the Gaussian noise image. If y is used as the model condition, the reverse process is t and y, and infer X t-1 , the formula is as follows: p(X t-1 |X t , y).

9. The high-precision spectral remote sensing atmospheric correction and cloud removal processing method integrating multi-temporal phases, atmospheric parameters and ground reflectivity according to claim 4 is characterized in that: The reverse denoising process of the diffusion model uses a UNet-like neural network encoding to predict the noise distribution based on the noise image and the time step guidance condition. UNet is used for image semantic segmentation and image generation tasks. It has an encoding-decoding structure and extracts features and retains the large-scale information of the image through the process of downsampling and upsampling. The structure of UNet is divided into the left part and the right part. The left part is the encoding process, which is used for downsampling and feature extraction, reducing the spatial size of the feature map while retaining important features. While extracting features, the image size is gradually reduced; The right part is the decoding process, which is used for upsampling and feature fusion. Upsampling is performed at each level. After each upsampling, the obtained feature map is superimposed and connected with the corresponding feature map on the left to fuse multi-scale feature information, restore the image size, and increase the number of channels. The UNet network contains an output layer that uses a convolutional network module to adjust the output structure to match the format of the target output.

Images