Timing sequence remote sensing composite under-cloud reconstruction method and system based on observation modeling and radiation reconstruction network

Abstract

The invention discloses a time sequence remote sensing composite under-cloud reconstruction method and system based on observation modeling and a radiation reconstruction network, and belongs to the technical field of remote sensing, and the method comprises the steps: carrying out the mask processing of a composite cloud time sequence remote sensing image based on observation, and obtaining a real degraded sequence after mask; inputting the real degraded sequence into the trained radiation reconstruction network to obtain a cloudless surface reflectance sequence, and completing composite under-cloud reconstruction; wherein the training of the radiation reconstruction network comprises the following steps: constructing a simulated degradation-cloudless pairing data set; building a radiation reconstruction network; and training the constructed radiation reconstruction network by using the constructed simulated degradation-cloudless pairing data set to obtain a trained radiation reconstruction network. According to the method, joint processing of the space-time spectrum data is realized, the method can be effectively applied to joint processing of thin cloud radiation deviation correction and thick cloud information loss reconstruction, and finally reconstruction of the surface reflectance sequence under the composite cloud is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing, and particularly relates to a method and system for reconstructing time-series remote sensing under composite clouds based on observation modeling and radiation reconstruction network. Background Technique

[0002] Remote sensing technology has been widely applied in the fields of dynamic monitoring of the land surface, ecological environment protection, resource management, etc. However, the data acquisition process is often interfered by various factors. Factors such as cloud cover will cause spatio-temporal gaps in the data, affecting the integrity and availability of the data.

[0003] To solve this problem, a series of time-series reflectance reconstruction techniques have been developed based on the temporal correlation and spatial similarity of time-series data. The missing data is filled through mathematical models and algorithms to restore the true surface radiation information under the clouds. The existing methods can be divided into four categories: 1) Reconstruction techniques based on spatial information. This method originated from digital image processing and mainly relies on the spatial local or non-local features of the image itself to reconstruct the missing information. However, the main drawback of this type of method is the low accuracy of the restoration effect. The reconstruction results often show obvious seam marks and perform poorly in dealing with large-area continuous missing regions. 2) Reconstruction techniques based on spectral information. Such methods repair degraded data by using the complementary information of the spectrum, but their applicability is relatively limited, only applicable to hyperspectral or multispectral data, and the integrity of some band information needs to be ensured. 3) Reconstruction methods based on multi-temporal information. This method uses the correlation between images in different periods to fill the missing information. The key lies in selecting appropriate reference temporal data to ensure the effectiveness and accuracy of the reconstruction. 4) Reconstruction techniques based on multi-source information. The core idea of this method is to reconstruct the missing information by fusing data from different sources. However, the requirements for data are relatively high, increasing the complexity of its application.

[0004] In the thick cloud cover area of the composite cloud, the surface information is completely lost and needs to be reconstructed using relevant information; while in the thin cloud cover area, there is still some surface radiation information, and it is necessary to correct the radiation deviation caused by the cloud and mist. However, in the time-series reconstruction task, the existing several types of methods usually uniformly process multiple types of clouds, without considering the differences in the degradation types of different cloud types on the surface radiation, and lacking a systematic study on the radiation reconstruction problem under composite clouds. The existing cloud removal methods generally only focus on a certain type of degradation problem and lack a study on the overall process of composite cloud removal.

[0005] Therefore, it is necessary to design a method and system for reconstructing time-series remote sensing under composite clouds based on observation modeling and radiation reconstruction network to solve the above problems. Summary of the Invention

[0006] The object of the present invention is to provide a temporal remote sensing composite cloud under-reconstruction method based on observational modeling and radiation reconstruction network for problems such as coexistence of multiple types of degradations including thin cloud radiation deviation and thick cloud missing reconstruction, and high information redundancy in temporal data, realizing integrated intelligent processing of composite cloud degradation in temporal data, improving the utilization rate of remote sensing images, and promoting effective comparison and analysis of subsequent application research.

[0007] According to one aspect of the present specification, there is provided a temporal remote sensing composite cloud under-reconstruction method based on observational modeling and radiation reconstruction network, including:

[0008] Performing masking processing on the observational composite cloud temporal remote sensing image to obtain the masked true degradation sequence;

[0009] Inputting the true degradation sequence into the trained radiation reconstruction network to obtain the cloud-free surface reflectance sequence, completing the reconstruction under the composite cloud; wherein, the training of the radiation reconstruction network includes:

[0010] Constructing a simulated degradation-cloud-free paired dataset;

[0011] Building a radiation reconstruction network, including: a three-dimensional encoding module for converting the input data into a sequence with spatial position information and masking the thick cloud area; a Transformer module for capturing long-distance dependence relationships and performing multi-level feature fusion; an output module for inverse encoding the output features of the Transformer.

[0012] Training the built radiation reconstruction network using the constructed simulated degradation-cloud-free paired dataset to obtain the trained radiation reconstruction network.

[0013] Further, the three-dimensional encoding module further includes:

[0014] Performing patch embedding on the input data using three-dimensional convolution to realize joint modeling of time-space-spectrum features, and the dimension of the modeled sequence is (T, H, W, C), where T is the temporal length, H×W is the spatial resolution, and C is the number of channels.

[0015] Further, the construction of the simulated degradation-cloud-free paired dataset includes:

[0016] Obtaining a simulated cloud-free sequence, and setting the cloud-covered area of the observational composite cloud temporal remote sensing image as Nan value;

[0017] Based on the simulated cloud-free sequence, superimposing thin cloud radiation to obtain a simulated degradation sequence, and performing thick cloud masking processing to form spatio-temporally aligned "degradation-cloud-free" paired data.

[0018] Furthermore, the training of the radiation reconstruction network also includes being constrained by a joint loss function, specifically:

[0019] The L1 loss and L2 loss are adopted to constrain the radiation characteristics and texture details;

[0020] The edge optimization loss based on the Sobel operator is adopted to extract the edge gradient difference between the reconstructed sequence and the label sequence, for suppressing the block effect;

[0021] Define a dynamic weight adjustment matrix for assigning higher weights to the missing regions according to the thick cloud mask.

[0022] Furthermore, the expression of the joint loss function is:

[0023]

[0024] Among them, represents the reconstructed sequence, represents the label sequence, represents the total number of pixels in the sequence image, represents the th pixel in the sequence, represents the process of extracting edge texture by the Sobel operator, represents the weight matrix.

[0025] According to one aspect of this specification, a temporal remote sensing composite cloud-covered surface reconstruction system based on observation modeling and a radiation reconstruction network is provided, including:

[0026] A data acquisition module, configured to perform masking processing on the observed composite cloud temporal remote sensing image to obtain the masked true degradation sequence;

[0027] A composite cloud-covered surface reconstruction module, configured to input the true degradation sequence into the trained radiation reconstruction network to obtain the cloud-free surface reflectance sequence, and complete the reconstruction under the composite cloud; wherein, the training of the radiation reconstruction network includes:

[0028] Construct a simulated degradation-cloud-free paired dataset;

[0029] Build a radiation reconstruction network, including: a three-dimensional encoding module, configured to convert the input data into a three-dimensional sequence with spatial position information and perform masking processing on the thick cloud region; a Transformer module, configured to capture long-distance dependencies and perform multi-level feature fusion; an output module, configured to perform inverse encoding on the output features of the Transformer.

[0030] Use the constructed simulated degradation-cloud-free paired dataset to train the built radiation reconstruction network to obtain the trained radiation reconstruction network.

[0031] According to one aspect of the present specification, there is provided an electronic device, including a memory and a processor. The memory stores a computer program. It is characterized in that when the processor executes the computer program, the steps of the temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network are implemented.

[0032] According to one aspect of the present specification, there is provided a computer-readable storage medium, on which a computer program is stored. It is characterized in that when the computer program is executed by a processor, the steps of the temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network are implemented.

[0033] According to one aspect of the specification of the present invention, there is provided a computer program product containing instructions, which when running on a computer, causes the computer to execute the steps of the temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0035] 1. The radiation reconstruction network constructed by the present invention promotes the mapping learning from the degraded sequence to the cloud-free sequence, realizes the integrated modeling of spatio-temporal spectral multi-dimensional information of the temporal degraded data, can be effectively applied to the joint processing of thin cloud radiation deviation correction and thick cloud information missing reconstruction, and finally realizes the restoration of the surface reflectance sequence under clouds.

[0036] 2. Aiming at the problem of coexistence of multiple types of degradation such as thin cloud radiation deviation and thick cloud missing reconstruction, the present invention realizes the integrated intelligent processing of composite cloud degradation in temporal data, improves the utilization rate of remote sensing images, and promotes the effective comparison and analysis of subsequent application research. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0038] Figure 1 It is the overall process framework diagram of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] As Figure 1 shown, the embodiments of the present invention also provide the overall process of the time-series remote sensing composite cloud removal and reconstruction method based on observation modeling and radiation reconstruction network. Specifically, through Step 1, simulation of paired sequence synthesis, Step 2, training of the radiation reconstruction network, Step 3, masking of real degraded sequences, and Step 4, composite cloud removal, the input time-series data is finally processed into cloud-free and spatially seamless surface reflectance data.

[0041] Specifically, the embodiments of the present invention also provide the specific process of Step 1. The cloud-covered areas of the cloud-free sequence data are uniformly assigned Nan for identification, and only the surface radiation in the clear-sky areas is retained for subsequent calculation processes. The sequence images are uniformly cropped into image blocks of size size. All simulated cloud-free sequence images are saved in the same folder, which contains multiple sub-folders named by spatial serial numbers. Each sub-folder stores the data at T moments in the same spatial position and is numbered in chronological order. Correspondingly, the storage format of the simulated degraded sequence dataset is the same as that of the cloud-free sequence dataset. Their main difference is that the degraded sequence data is synthesized by superimposing thin cloud radiation on the cloud-free sequence data, and the missing simulation of the thick cloud area is achieved through masking technology. The incremental design of the thin cloud radiation is consistent with the distribution characteristics of the real data, and the value setting of the thick cloud missing simulation is 1 (refer to the patent CN119048915A, a method for constructing a thin cloud degradation dataset based on coupled learning, published on November 29, 2024). The images in the two datasets form a "degraded-cloud-free" pair, and the surface radiation values of the paired data are kept consistent.

[0042] Specifically, in the case of remote sensing image composite cloud degradation, the observed image of the sensor can be expressed by the following formula:

[0043]

[0044] Wherein, represents the time-series observed image sequence, represents the surface radiation sequence, represents the atmospheric radiation sequence, represents the thick cloud mask sequence, Denote the observation time. According to formula (1), the reconstruction of the surface under compound clouds in the time-series observation images can be divided into two parts. Among them, the thin cloud correction process can be expressed as , and then the surface information in the thin cloud-covered area and the cloud-free area are used together to guide the thick cloud reconstruction process as , refers to the regional reconstruction algorithm.

[0045] Specifically, in the embodiments of the present invention, the apparent reflectance data processed by radiometric calibration are used to train the network. The network input is the simulated degradation sequence , and the label data is the simulated cloud-free sequence . A simulated degradation-cloud-free paired dataset is constructed, including: The simulated cloud-free sequence is collected from the Landsat8 OLI sensor, and the cloud-covered area of the original image is set to Nan values to mask the calculation of this part of the radiation during the reconstruction process. The simulated degradation sequence is based on the simulated cloud-free sequence, synthesizes thin cloud radiation, and masks 60% of the images according to the true cloud coverage ratio to simulate the missing thick clouds. The simulated degradation-cloud-free sequence is a paired sequence, which can be used to guide the training of the radiation reconstruction network and for the time-series remote sensing compound cloud removal task.

[0046] Specifically, the image sequence needs to be masked in the thick cloud-covered area before being input into the network, so as to identify the area to be reconstructed. Specifically, the quality assessment (QA) band of the Landsat image contains classification information such as clouds, shadows, and water bodies. Using the cloud position information recorded in the third bit of the QA band, guide the masking of the sequence data and assign the thick cloud area as 1.

[0047] Specifically, the embodiments of the present invention construct a radiation reconstruction network based on Vision Transformer, which uses three-dimensional encoding to jointly process spatio-temporal spectral features. After the simulated degradation sequence is input, it mainly undergoes the following several parts of processing: 1) Encoding processing; including image encoding and position encoding. Through three-dimensional encoding, the input data is transformed into a sequence with spatial position information that can be processed by the Transformer, and the thick cloud area is masked to reduce the computational amount of the network. 2) Transformer module; including multi-layer perceptron and multi-head attention. This mechanism allows the model to focus on different parts of the input data, thereby capturing long-range dependencies, enhancing the expression ability of features, and improving the model's understanding ability of complex scenes through multi-level feature fusion. 3) Output module; perform inverse encoding on the output features of the Transformer to make them the same size as the label data, that is, the simulated cloud-free sequence.

[0048] Specifically, the three-dimensional encoding module jointly models spatio-temporal spectral features using separable convolutions. The dimension of the encoded sequence is (T, H, W, C), where T is the temporal length, H×W is the spatial resolution, and C is the number of channels.

[0049] Specifically, the embodiment of the present invention mainly guides the radiation reconstruction network by constraining the similarity between the network output and the label data. The radiation reconstruction network can achieve the correction of thin cloud radiation and the information reconstruction of the thick cloud area marked by the mask. To simultaneously consider the radiation characteristics and texture details of the restored sequence and efficiently constrain the network, a combined loss of L1 and L2 norms is set. And for the block effect of the Vision Transformer result, an edge optimization loss based on the Sobel operator is set to extract the edge gradient difference between the reconstructed sequence and the label sequence. In addition, the loss weight of the information missing area is increased to ensure that the information in the missing area is fully concerned during the reconstruction process, thereby improving the overall restoration quality. For the reconstructed sequence and the label sequence in terms of, the constraint function can be expressed by Equation (2):

[0050]

[0051] where, represents the total number of pixels in the sequence image, represents the th pixel in the sequence, represents the process of extracting edge textures by the Sobel operator, represents the weight matrix, which will be dynamically adjusted according to the mask M, as shown in Equation (3):

[0052]

[0053] where, The value of the corresponding pixel is consistent with the observation model. When the value is 1, it represents clear sky or thin cloud cover conditions, and when the value is 0, it represents thick cloud cover.

[0054] Specifically, the embodiment of the present invention optimizes the network parameters by minimizing the combined loss function. After training is completed, the masked real degraded sequence is input. The network weights will be used for the final removal of compound clouds in arbitrarily timed degraded images, and the reconstructed cloud-free surface reflectance sequence is output. The thick cloud mask of the real degraded sequence is generated by the cloud position information of the third bit of the Landsat QA band. The area with a mask value of 1 corresponds to the thick cloud covered area to be reconstructed. The final constraint target is The value is as small as possible and tends to be stable.

[0055] The implementation basis of each embodiment of the present invention is achieved through programmed processing by a device with processor functions. Therefore, in engineering practice, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this actual situation, on the basis of the above embodiments, an embodiment of the present invention provides a temporal remote sensing composite cloud under reconstruction system based on observation modeling and radiation reconstruction network, which is used to execute the temporal remote sensing composite cloud under reconstruction method based on observation modeling and radiation reconstruction network in the above method embodiment.

[0056] The system includes: an acquisition data module, configured to perform masking processing on the observed composite cloud temporal remote sensing image to obtain the masked real degradation sequence;

[0057] A composite cloud under reconstruction module, configured to input the real degradation sequence into the trained radiation reconstruction network to obtain a cloud-free surface reflectance sequence, and complete the reconstruction under the composite cloud; wherein, the training of the radiation reconstruction network includes:

[0058] Construct a simulated degradation-cloud-free paired dataset;

[0059] Build a radiation reconstruction network, including: a three-dimensional encoding module, configured to convert the input data into a three-dimensional sequence with spatial position information and perform masking processing on the thick cloud area; a Transformer module, configured to capture long-distance dependence relationships and perform multi-level feature fusion; an output module, configured to perform inverse encoding on the output features of the Transformer.

[0060] Use the constructed simulated degradation-cloud-free paired dataset to train the built radiation reconstruction network to obtain a trained radiation reconstruction network.

[0061] For the temporal remote sensing composite cloud under reconstruction system based on observation modeling and radiation reconstruction network provided by the embodiment of the present invention, in the case of coexistence of multiple types of degradations such as thin cloud radiation deviation and thick cloud missing reconstruction, and high information redundancy in temporal data, several modules are adopted, and through the radiation reconstruction network, it promotes the mapping learning from the degradation sequence to the cloud-free sequence, realizes the joint processing of spatio-temporal spectral data, can be effectively applied to the joint processing of thin cloud radiation deviation correction and thick cloud information missing reconstruction, and finally realizes the restoration of the cloud-free surface reflectance sequence.

[0062] Based on the same inventive concept as the foregoing embodiment, an embodiment of the present invention further provides an electronic device, including a memory and a processor, the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement a temporal remote sensing composite cloud under reconstruction method based on observation modeling and radiation reconstruction network as proposed in the above embodiment.

[0063] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it overcomes the coexistence of multiple types of degradations such as thin cloud radiation deviation and thick cloud missing reconstruction, as well as the high information redundancy in time-series data, realizes the integrated intelligent processing of composite cloud degradation in time-series data, improves the utilization rate of remote sensing images, and promotes the effective comparison and analysis of subsequent application research. The storage medium can be any non-volatile storage device such as a hard disk, a solid-state drive, a flash drive, an optical disc, etc., for storing computer program codes and necessary data files. The stored computer program includes: a data acquisition module, a reconstruction module under composite clouds.

[0064] An embodiment of the present invention also provides a computer program product containing instructions, which, when running on a computer, wholly or partially generates the method for reconstructing under a composite cloud in time-series remote sensing based on observation modeling and radiation reconstruction network proposed in the above embodiment. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0065] Finally, it should be pointed out that the above specific embodiments are only relatively representative examples of the present invention. Obviously, the present invention is not limited to the above specific embodiments and there can be many variations. Any simple modification, equivalent change, and modification made to the above specific embodiments based on the technical essence of the present invention should be considered to fall within the protection scope of the present invention.

Claims

1. A temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network, characterized in that, Including: Performing masking processing on the time-series remote sensing images of composite clouds based on observations to obtain the masked true degradation sequence; Inputting the true degradation sequence into the trained radiation reconstruction network to obtain the cloud-free surface reflectance sequence, and completing the reconstruction under composite clouds; wherein, the training of the radiation reconstruction network includes: Constructing a simulated degradation-cloud-free paired dataset; Building a radiation reconstruction network, including: a three-dimensional encoding module for converting the input data into a sequence with spatial position information and masking the thick cloud area; a Transformer module for capturing long-range dependencies and performing multi-level feature fusion; an output module for inverse encoding the output features of the Transformer; Training the built radiation reconstruction network using the constructed simulated degradation-cloud-free paired dataset to obtain the trained radiation reconstruction network.

2. The temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network according to claim 1, characterized in that The three-dimensional encoding module further includes: Performing patch embedding on the input data using three-dimensional convolution to achieve joint modeling of time-space-spectrum features. The dimension of the modeled sequence is (T, H, W, C), where T is the time series length, H×W is the spatial resolution, and C is the number of channels.

3. A temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network according to claim 1, characterized in that, The construction of the simulated degradation-cloud-free paired dataset includes: Obtaining a simulated cloud-free sequence and setting the cloud-covered area of the observed time-series remote sensing images of composite clouds as Nan values; Based on the simulated cloud-free sequence, superimposing thin cloud radiation to obtain a simulated degradation sequence, and performing thick cloud masking processing to form spatio-temporally aligned "degradation-cloud-free" paired data.

4. A temporal remote sensing composite cloud-bottom reconstruction method based on observation modeling and radiation reconstruction network according to claim 1, characterized in that The training of the radiation reconstruction network further includes being constrained by a joint loss function, specifically: Adopting L1 loss and L2 loss to constrain the radiation characteristics and texture details; Adopting an edge optimization loss based on the Sobel operator to extract the edge gradient difference between the reconstructed sequence and the label sequence for suppressing the block effect; Defining a dynamic weight adjustment matrix for assigning higher weights to the missing areas according to the thick cloud mask.

5. A temporal remote sensing composite cloud-down reconstruction method based on observation modeling and radiation reconstruction network according to claim 4, characterized in that The expression of the joint loss function is: , Among them, represents the reconstructed sequence, represents the label sequence, represents the total number of pixels in the sequence image, represents the th pixel in the sequence, represents the process of extracting edge texture by the Sobel operator, represents the weight matrix.

6. A temporal remote sensing composite cloud-covered reconstruction system based on observation modeling and radiation reconstruction network, characterized in that, Including: A data acquisition module for performing masking processing on the time-series remote sensing images of composite clouds based on observations to obtain the masked true degradation sequence; A reconstruction module under composite clouds for inputting the true degradation sequence into the trained radiation reconstruction network to obtain the cloud-free surface reflectance sequence and completing the reconstruction under composite clouds; wherein, the training of the radiation reconstruction network includes: Constructing a simulated degradation-cloud-free paired dataset; Building a radiation reconstruction network, including: a three-dimensional encoding module for converting the input data into a three-dimensional sequence with spatial position information and masking the thick cloud area; a Transformer module for capturing long-range dependencies and performing multi-level feature fusion; an output module for inverse encoding the output features of the Transformer; Training the built radiation reconstruction network using the constructed simulated degradation-cloud-free paired dataset to obtain the trained radiation reconstruction network.

7. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for reconstructing time-series remote sensing composite clouds under composite clouds based on observational modeling and radiation reconstruction network according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network according to any one of claims 1 to 5.

9. A computer program product comprising instructions, characterized in that, When it runs on a computer, it causes the computer to execute the steps of the temporal remote sensing composite reconstruction method under clouds based on observation modeling and radiation reconstruction network according to any one of claims 1 to 5.

Images