Ice and snow albedo data reconstruction method based on deep learning fusion multi-source data

Abstract

The invention provides an ice and snow albedo data reconstruction method based on deep learning fusion multi-source data, and relates to the technical field of remote sensing, and the method comprises the steps: obtaining albedo data and geographical auxiliary data; processing the albedo data by using a non-local space-time filtering method and a Whittaker filtering method to obtain space-time auxiliary data; an R-TNet deep learning neural network model is constructed; training an R-TNet deep learning neural network model through albedo data, geographical assistance data and space-time assistance data; obtaining albedo data to be recovered; and inputting the geographic auxiliary data and the space-time auxiliary data corresponding to the missing pixels of the albedo data to be recovered into the trained R-TNet deep learning neural network model, and recovering the albedo data to be recovered of the missing pixels. According to the technical scheme of the invention, the neural network is combined to deeply mine the internal nonlinear relationship of the variables, and the time-space continuous and accurate albedo data is effectively reconstructed.

Description

Technical Field

[0001] The present application relates to the field of remote sensing technology, and in particular to a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data. Background Art

[0002] Snow and ice albedo, as an important geographical parameter reflecting the earth's energy balance, plays an important role in global climate and heat balance. Due to the high albedo of snow and ice, its slight changes may form a positive feedback effect on the energy balance of the earth-atmosphere system, further intensifying global warming. The increase in snow and ice melting leading to a reduction in its mass is also closely related to the decrease in surface albedo and the rise in regional temperature. Therefore, it is of great significance to study the temporal and spatial changes of snow and ice albedo.

[0003] Optical satellite remote sensing provides a way to continuously observe albedo on a large spatial and temporal scale. In particular, the Moderately High Resolution Imaging Spectroradiometer (MODIS) albedo products have become a valuable resource in snow and ice research due to their high spatial and temporal resolution, long-term data records, and free access to data. However, studies have shown that cloud coverage on the global land surface can be as high as 55%, resulting in a large amount of missing data in MODIS products. In addition, in existing observation and prediction models, environmental factors related to albedo are often difficult to accurately measure and simulate, resulting in uncertainty in albedo data. Therefore, reconstructing temporally and spatially continuous and accurate albedo data is very important for climate change monitoring. Summary of the invention

[0004] The purpose of the present invention is to provide a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data in order to solve the problem of long-term continuous loss of albedo data.

[0005] The above-mentioned purpose of the present application is achieved through the following technical solutions: Step S1: Acquire albedo data, acquire geographic auxiliary data related to the albedo data, and unify the spatial resolutions of the albedo data and the geographic auxiliary data; Step S2: Processing the albedo data using the non-local spatiotemporal filtering method and the Whittaker filtering method to obtain a clear sky albedo data sequence at the missing pixels of the albedo data, and using it as spatiotemporal auxiliary data; Step S3: Build an R-TNet deep learning neural network model based on ResNet and Transformer; Step S4: training the R-TNet deep learning neural network model through albedo data, geographic auxiliary data and spatiotemporal auxiliary data; Step S5: Obtain the albedo data to be restored and the spatiotemporal auxiliary data corresponding to the albedo data to be restored; input the geographic auxiliary data and spatiotemporal auxiliary data corresponding to the missing pixels of the albedo data to be restored into the trained R-TNet deep learning neural network model to restore the albedo data to be restored of the missing pixels.

[0006] Optionally, step S1 includes: The geographic auxiliary data include: elevation data, solar zenith angle data, cloud cover data, cloud optical thickness data, brightness temperature data, surface temperature data, snow depth data and snow density data.

[0007] Optionally, step S2 includes: S21: Fill missing pixels in albedo data using spatiotemporal non-local filtering method; S22: Filter the filled albedo data using the Whittaker algorithm to generate a corrected clear sky albedo data sequence, which is used as spatiotemporal auxiliary data.

[0008] Optionally, step S21 includes: For pixels with a neighborhood window of Ω , calculate the weight value of the pixel , the calculation formula is as follows:

[0009] Pixel Recovery value The calculation formula is:

[0010] in, represents Gaussian kernel; and Represent the values ​​of similar pixels and target pixels respectively; Represents the geographic distance between the target pixel and similar pixels.

[0011] Optionally, step S3 includes: R-TNet deep learning neural network model, including: a residual network and a Transformer encoder.

[0012] Optionally, the residual network includes five residual blocks; each residual block includes a fully connected layer and a batch normalization layer, and uses a ReLU activation function for nonlinear transformation; The Transformer encoder consists of a multi-head attention mechanism layer, layer normalization, a ReLU activated feed-forward network, and a fully connected layer.

[0013] An electronic device includes a processor, a memory, a user interface and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data.

[0014] A computer-readable storage medium stores instructions. When the instructions are executed, a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data is performed.

[0015] The beneficial effects of the technical solution provided by this application are: Considering the temporal dependence of ice and snow albedo data, the spatiotemporal non-local filtering method and Whittaker filtering method are used to reconstruct the clear sky albedo data sequence under clear sky conditions and use it as spatiotemporal auxiliary data. The R-TNet deep learning neural network model is trained through albedo data, geographic auxiliary data and spatiotemporal auxiliary data. The variable data related to albedo are fully integrated, and the deep learning neural network is used to construct the relationship between different variables and albedo. The neural network is combined to deeply explore the intrinsic nonlinear relationship of the variables, and effectively reconstruct spatiotemporal continuous and accurate albedo data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present application will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 is a step diagram in an embodiment of the present application; Figure 2 is a network structure diagram in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific implementation methods of the present application are now described in detail with reference to the accompanying drawings.

[0018] An embodiment of the present application provides a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data.

[0019] Please refer to Figure 1 , Figure 1 This is a step diagram of a method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data in an embodiment of the present application, including: Step S1: Acquire albedo data, acquire geographic auxiliary data related to the albedo data, and unify the spatial resolutions of the albedo data and the geographic auxiliary data; Step S2: Processing the albedo data using the non-local spatiotemporal filtering method and the Whittaker filtering method to obtain a clear sky albedo data sequence at the missing pixels of the albedo data, and using it as spatiotemporal auxiliary data; Step S3: Build an R-TNet deep learning neural network model based on ResNet and Transformer; Step S4: training the R-TNet deep learning neural network model through albedo data, geographic auxiliary data and spatiotemporal auxiliary data; Step S5: Obtain the albedo data to be restored and the spatiotemporal auxiliary data corresponding to the albedo data to be restored; input the geographic auxiliary data and spatiotemporal auxiliary data corresponding to the missing pixels of the albedo data to be restored into the trained R-TNet deep learning neural network model to restore the albedo data to be restored of the missing pixels.

[0020] Step S1 includes: The geographic auxiliary data include: elevation data, solar zenith angle data, cloud cover data, cloud optical thickness data, brightness temperature data, surface temperature data, snow depth data and snow density data.

[0021] Specifically, the albedo data comes from the existing remote sensing dataset MOD10A1, with a spatial resolution of 500 meters. The remote sensing data of elevation comes from GDEMV2 30M, with a spatial resolution of 30 meters. The remote sensing data of solar zenith angle comes from the existing remote sensing dataset MOD09GA, with a spatial resolution of 1000 meters. The remote sensing data of cloud optical thickness, cloud amount and top of atmosphere brightness temperature come from the Pathfinder Atmospheres Extended (PATMOS-x) cloud product dataset, with a spatial resolution of 0.01°. The remote sensing data of surface temperature, snow depth and snow density come from the ERA5- land analysis dataset, with a spatial resolution of 0.1°. When unifying the spatial resolution of albedo data and geographic auxiliary data, the remote sensing data of elevation, solar zenith angle, cloud optical thickness, cloud amount, top of atmosphere brightness temperature, surface temperature, snow density and snow depth are resampled to the same spatial resolution as the remote sensing data of albedo.

[0022] Step S2 includes: S21: Fill missing pixels in albedo data using spatiotemporal non-local filtering method; S22: Filter the filled albedo data using the Whittaker algorithm to generate a corrected clear sky albedo data sequence, which is used as spatiotemporal auxiliary data.

[0023] Step S21 includes: For pixels with a neighborhood window of Ω , calculate the weight value of the pixel , the calculation formula is as follows:

[0024] Pixel Recovery value The calculation formula is:

[0025] in, represents Gaussian kernel; and Represent the values ​​of similar pixels and target pixels respectively; Represents the geographic distance between the target pixel and similar pixels.

[0026] Step S3 includes: R-TNet deep learning neural network model, including: a residual network and a Transformer encoder.

[0027] The residual network includes five residual blocks; each residual block includes a fully connected layer and a batch normalization layer, and uses a ReLU activation function for nonlinear transformation; The Transformer encoder consists of a multi-head attention mechanism layer, layer normalization, a ReLU activated feed-forward network, and a fully connected layer.

[0028] Specifically, the input data passes through the ResNet module, which includes five residual blocks. The output dimension of each block gradually increases from 32 to 128 and then decreases back to 32. Each residual block contains a fully-connected layer and a batch normalization layer, and uses the ReLU activation function for nonlinear transformation. After the ResNet module processes the data, the model introduces the encoder part of the Transformer, which first applies a multi-head attention mechanism layer, then normalizes the results of the multi-head attention layer (Add&Norm), and further processes it through a feedforward network (Feed Forward) containing ReLU activation. Finally, the final prediction value is output through a fully connected layer. The learning rate of the training model is set to 0.01, the batchsize is 32, the number of training generations is 400, the number of heads of the multi-head attention is 2, and num_layers is 2. The network weights are randomly initialized. After a lot of training optimization, when the network loss function value drops to a sufficiently low and stable level, the training model parameters are saved.

[0029] Specifically, Figure 2 As shown in the figure, the ResNet module (residual network) forms the basis of the model, which consists of several residual blocks. Each residual block has a linear layer inside, followed by batch normalization and ReLU activation function. The features processed by each block are combined with the input features, and the combined information is then sent to the Transform encoder. The Transformer encoder consists of several Transformer blocks, each of which contains a multi-head attention mechanism, layer normalization, a feedforward network, and a dropout layer.

[0030] The ResNet part forms the basis of the model and consists of several residual blocks. The core function of these residual blocks is to deeply extract data features. Its unique residual connection design effectively avoids the gradient vanishing problem that occurs during deep model training. It can be expressed by the following formula:

[0031] Where x is the input feature, H(x) is the output feature of the block, The representative weight is Convolution operation. Each residual block has a linear layer, followed by batch normalization and ReLU activation function. The features processed by each block are combined with the input features, and the combined information is sent to the Transform encoder to achieve more complex spatiotemporal feature extraction.

[0032] The Transformer encoder consists of several Transformer blocks, each of which contains a multi-head attention mechanism, layer normalization, a feedforward network, and a dropout layer. The core of this architecture is the multi-head attention mechanism, which measures the similarity between vectors through dot products, and further adjusts these similarity values ​​by scaling (usually dividing by the square root of the vector dimension) to prevent the gradient of the activation function from being too small; the Softmax function is applied to determine the weight distribution of the relationship between each vector, thereby achieving dynamic attention to information in sequence data processing. This attention mechanism is called scaled dot product attention and is expressed by the following formula:

[0033] in, Represent the query, key, and value matrices respectively. is the dimension of the key vector. The training process uses the mean square error (MSE) as the loss function, and the optimizer uses Adam to update parameters with high efficiency.

[0034] The present invention makes full use of variables related to albedo, introduces spatiotemporal auxiliary information to calculate the clear sky albedo sequence, and uses a deep learning neural network to construct a nonlinear relationship between different variables to restore the lost albedo data. In the R-TNet network, the preliminary features extracted by the ResNet module are then passed to the Transformer module for further processing and integration, thereby realizing the feature extraction and prediction tasks of the input data. The design of this structure not only utilizes the ability of resNet to maintain information flow in a deep hierarchical structure, but also combines the efficiency of transformer in processing long-distance dependencies in sequence data, so that the model performs well in processing diverse geographic auxiliary data. And good results have been achieved in the correlation coefficient (R), root mean square error (RMSE), and mean absolute error (MAE). The above method of the present invention finally reconstructs the spatiotemporal continuous Greenland albedo data.

[0035] In order to verify the performance of the R-TNet deep learning neural network model (R-TNet) based on ResNet and Transformer, it was compared with the currently popular machine learning and deep learning networks. The machine learning models include multivariate linear regression (MLR), K-NearestNeighbor (KNN), and random forest (RF), and the deep learning networks include residual networks (ResNet) and transformer.

[0036] Table 1. Quantitative evaluation of albedo data predicted by different models

[0037] As relatively basic models, K-nearest neighbor (KNN), linear regression, and random forest showed lower performance indicators. Among them, KNN performed the worst, indicating that it has weak processing capabilities for complex data. In contrast, although the performance of MLR and RF is improved compared to KNN, the accuracy is still not ideal. Deep learning models such as ResNet and Transformer showed higher performance. Among them, transformer performed better than ResNet, with an R value of 0.8264, RMSE and MAE of 0.0792 and 0.0455 respectively. Overall, the R-TNet model performed best in all evaluation indicators, with a correlation coefficient of 0.8775, RMSE of 0.0698, and MAE of 0.0442, showing its high efficiency and accuracy in processing multi-source complex data.

[0038] The present application also discloses an electronic device. Figure 3 , Figure 3The electronic device 500 may include: at least one processor 501 , at least one network interface 504 , a user interface 503 , a memory 505 , and at least one communication bus 502 .

[0039] The communication bus 502 is used to realize the connection and communication between these components.

[0040] The user interface 503 may include a display screen, and the optional user interface 503 may also include a standard wired interface or a wireless interface.

[0041] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0042] The present application also discloses a computer-readable storage medium storing a plurality of instructions suitable for loading by a processor to execute the above-mentioned method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data.

[0043] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0044] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data, characterized in that: The method comprises the following steps: Step S1: Acquire albedo data, acquire geographic auxiliary data related to the albedo data, and unify the spatial resolutions of the albedo data and the geographic auxiliary data; Step S2: Processing the albedo data using the non-local spatiotemporal filtering method and the Whittaker filtering method to obtain a clear sky albedo data sequence at the missing pixels of the albedo data, and using it as spatiotemporal auxiliary data; Step S3: Build an R-TNet deep learning neural network model based on ResNet and Transformer; Step S4: training the R-TNet deep learning neural network model through albedo data, geographic auxiliary data and spatiotemporal auxiliary data; Step S5: Obtain the albedo data to be restored and the spatiotemporal auxiliary data corresponding to the albedo data to be restored; input the geographic auxiliary data and spatiotemporal auxiliary data corresponding to the missing pixels of the albedo data to be restored into the trained R-TNet deep learning neural network model to restore the albedo data to be restored of the missing pixels.

2. The method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data as claimed in claim 1, characterized in that: Step S1 includes: The geographic auxiliary data include: elevation data, solar zenith angle data, cloud cover data, cloud optical thickness data, brightness temperature data, surface temperature data, snow depth data and snow density data.

3. The method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data as claimed in claim 1, characterized in that: Step S2 includes: S21: Fill missing pixels in albedo data using spatiotemporal non-local filtering method; S22: Filter the filled albedo data using the Whittaker algorithm to generate a corrected clear sky albedo data sequence, which is used as spatiotemporal auxiliary data.

4. The method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data as claimed in claim 3, characterized in that: Step S21 includes: For pixels with a neighborhood window of Ω , calculate the weight value of the pixel , the calculation formula is as follows: Pixel Recovery value The calculation formula is: in, represents Gaussian kernel; and Represent the values ​​of similar pixels and target pixels respectively; Represents the geographic distance between the target pixel and similar pixels.

5. The method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data as claimed in claim 1, characterized in that: Step S3 includes: R-TNet deep learning neural network model, including: a residual network and a Transformer encoder.

6. The method for reconstructing ice and snow albedo data based on deep learning and fusion of multi-source data as claimed in claim 5, characterized in that: The residual network includes five residual blocks; each residual block includes a fully connected layer and a batch normalization layer, and uses a ReLU activation function for nonlinear transformation; The Transformer encoder includes: a multi-head attention mechanism layer, layer normalization, a ReLU activated feedforward network FeedForwar, and a fully connected layer.

7. An electronic device, characterized in that: The electronic device comprises a processor (501), a memory (505), a user interface (503) and a network interface (504), wherein the memory (505) is used to store instructions, the user interface (503) and the network interface (504) are used to communicate with other devices, and the processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device executes the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a computer, the method according to any one of claims 1 to 6 is executed.

Images