MODIS (moderate resolution imaging spectroradiometer) surface temperature compensation method based on spatial-temporal feature fusion

Abstract

The invention discloses an MODIS (Moderate Resolution Imaging Spectroradiometer) surface temperature value compensation method based on spatial-temporal feature fusion. The value compensation method comprises the following steps: acquiring data to be subjected to value compensation; the data to be subjected to value compensation are input into a spatial-temporal feature fusion network model, a value compensation result is obtained, the spatial-temporal feature fusion network model is obtained through training of a training set, the training set is historical surface temperature data, the spatial-temporal feature fusion network model is constructed through a neural network, and a value compensation result is obtained; the neural network is used for extracting a one-dimensional time sequence feature and a three-dimensional space-time sequence feature, and performing feature fusion on the one-dimensional time sequence feature and the three-dimensional space-time sequence feature to obtain the value compensation result. According to the model, feature mining of a one-dimensional time sequence and a three-dimensional space-time sequence can be realized at the same time, and multi-dimensional information is effectively integrated based on a multi-dimensional feature fusion module.

Description

Technical Field

[0001] The invention belongs to the field of surface temperature data reconstruction, and in particular relates to a MODIS surface temperature compensation method based on spatiotemporal feature fusion. Background Art

[0002] Surface temperature is an important indicator of the Earth's surface characteristics, directly affecting climate change, ecosystems, crop growth, water resource management and many other fields. Changes in surface temperature not only reflect the dynamic changes in the natural environment, but are also closely related to human activities. MODIS can obtain a variety of important parameters such as surface temperature, vegetation cover, ocean and climate on a global scale at a daily frequency, with high spatial resolution (about 250 meters to 1 kilometer) and temporal resolution.

[0003] MODIS surface temperature data provides rich spatiotemporal information and is widely used in climate change research, ecological monitoring, agricultural production, urban heat island effect and other fields. However, due to factors such as climate conditions, sensor errors, and cloud cover, there are often missing values ​​or outliers in MODIS data, which brings challenges to data analysis and application. Therefore, it is particularly important to effectively supplement MODIS data.

[0004] The current surface temperature data compensation technology has many shortcomings, which to some extent restrict the accuracy and efficiency of compensation. The main defects of the existing technology are as follows:

[0005] (1) Limitations of traditional imputation methods: Traditional imputation techniques, when dealing with sparse data, usually assume that the data is spatially continuous. However, this assumption does not always hold, especially in the presence of significant noise or missing values. Traditional methods fail to effectively capture the complexity of the data, resulting in large deviations between the imputation results and the actual situation.

[0006] (2) Insufficient simple models: Many complementary models based on simple neural networks (such as basic RNN or 1D CNN) fail to fully capture the complexity of time series data. Such basic neural networks often encounter the problem of gradient vanishing or exploding when processing long sequence data, resulting in poor learning effects of the model over a longer time range. This limitation makes the model perform far less well than expected when complementary over a long time scale, affecting the reliability of complementary.

[0007] (3) Lack of spatiotemporal information fusion: Many current interpolation methods fail to effectively utilize spatial information when processing geographic data. Traditional time series models often only focus on changes in the time dimension and ignore temperature changes in surrounding pixels. For example, when interpolating MODIS surface temperature data, simple time series models fail to take into account the impact of other pixels in the neighborhood. This lack of spatial information fusion leads to unreasonable interpolation results.

[0008] Therefore, a MODIS surface temperature compensation method based on the fusion of spatiotemporal features is urgently needed to solve the above problems. Summary of the invention

[0009] To solve the above technical problems, the present invention proposes a MODIS surface temperature compensation method based on spatiotemporal feature fusion. The method uses one-dimensional time series and three-dimensional spatiotemporal series as input features, and fully considers the characteristic information of surface temperature in time and space. The model consists of two parallel branch modules and two multi-dimensional feature fusion modules. Each branch module is used to process different types of data, and has many significant advantages.

[0010] The present invention provides a MODIS surface temperature compensation method based on spatiotemporal feature fusion, including:

[0011] Get the value data to be supplemented;

[0012] The data to be supplemented is input into the spatiotemporal feature fusion network model to obtain the supplementation result, wherein the spatiotemporal feature fusion network model is obtained by training with a training set, the training set is historical surface temperature data, the spatiotemporal feature fusion network model is constructed using a neural network, and the neural network is used to extract one-dimensional time series features and three-dimensional spatiotemporal series features, and the one-dimensional time series features and the three-dimensional spatiotemporal series features are feature-fused to obtain the supplementation result.

[0013] Optionally, obtaining the training set includes:

[0014] Get raw surface temperature data;

[0015] The original surface temperature data is spliced, cropped and reprojected to obtain the training set.

[0016] Optionally, the spatiotemporal feature fusion network model includes: a one-dimensional time series feature extraction module, a three-dimensional spatiotemporal sequence feature extraction module, a spatiotemporal interaction fusion module and a multi-dimensional feature synthesis module;

[0017] The one-dimensional time series feature extraction module is used to extract one-dimensional feature information of the surface temperature;

[0018] The three-dimensional spatiotemporal sequence feature extraction module is used to extract three-dimensional feature information of surface temperature;

[0019] The spatiotemporal interactive fusion module is used to fuse the one-dimensional feature information and the three-dimensional feature information;

[0020] The multi-dimensional feature synthesis module is used to synthesize the fused one-dimensional feature information and three-dimensional feature information.

[0021] Optionally, extracting one-dimensional characteristic information of surface temperature includes:

[0022] Extracting the to-be-complemented value data using a 1D-CNN unit to perform one-dimensional time series feature extraction;

[0023] Using bidirectional N-Beats units to capture the basic patterns of the one-dimensional time series features and learn long-term dependencies;

[0024] The one-dimensional feature information is obtained according to the long-term dependency relationship.

[0025] Optionally, the N-Beats unit includes: an input layer, a basic block, a stacking layer and an output layer;

[0026] The input layer is used to input the one-dimensional time series feature;

[0027] The basic block is used to capture the information flow from the past to the present in the time series, extract the forward dependency of the sequence and capture the reverse information flow from the future to the present, extract the backward dependency of the sequence, and obtain the prediction result according to the forward dependency and the backward dependency;

[0028] The stacked layer is used to correct the prediction result;

[0029] The output layer is used to output the corrected prediction result.

[0030] Optionally, the three-dimensional spatiotemporal sequence feature extraction module includes: an input layer, a convolution layer, an activation layer, a pooling layer and an output layer;

[0031] The input layer is used to input the neighborhood spatiotemporal series data of missing points;

[0032] The convolution layer is used to extract the three-dimensional features of the spatiotemporal sequence data through a three-dimensional convolution operation;

[0033] The activation layer is used to perform nonlinear processing on the three-dimensional features using an activation function;

[0034] The pooling layer is used to reduce feature dimensions and extract representative three-dimensional features;

[0035] The output layer is used to output the three-dimensional feature information.

[0036] Optionally, fusing the one-dimensional feature information and the three-dimensional feature information includes:

[0037] Unifying the feature dimensions of the one-dimensional feature information and the three-dimensional feature information through a convolution operation to obtain dimensionally unified features;

[0038] The unified features of the dimensions are fused to obtain fused feature information.

[0039] Optionally, performing restoration processing on the fused one-dimensional feature information and three-dimensional feature information;

[0040] Performing a dimensionality reduction operation on the restored one-dimensional feature information and three-dimensional feature information to extract the reduced one-dimensional features and three-dimensional features;

[0041] Mapping the one-dimensional features and the three-dimensional features after dimensionality reduction to obtain one-dimensional mapping features and three-dimensional mapping features;

[0042] The one-dimensional mapping feature and the three-dimensional mapping feature are synthesized to obtain a complementation result.

[0043] Optionally, when training the spatiotemporal feature fusion network model, the Adam optimization algorithm is used to update the parameters of the spatiotemporal feature fusion network model.

[0044] Compared with the prior art, the present invention has the following advantages and technical effects:

[0045] (1) Improving the accuracy of the compensation: By using the 1DCNN-BiNbeats module to process one-dimensional time series data, the present invention can more effectively capture the nonlinear characteristics and long-term dependencies in the data. The design of this model enables it to perform well when processing long time series data, thereby significantly improving the accuracy of the compensation. In particular, when compensating MODIS surface temperature data, this model can better fit the actual temperature change trend, making the compensation result more reliable.

[0046] (2) Effective integration of spatiotemporal information: The model input used in the present invention is a one-dimensional time series and a three-dimensional spatiotemporal series, and has two input processing modules, namely the 1DCNN-BiNbeats module and the 3D-CNN module, which fully consider the spatiotemporal characteristics of the surface temperature data. The model not only considers the one-dimensional time series data of the target point, but also integrates the data in its surrounding neighborhood. In this way, the compensation result is more reasonable, can reflect the impact of the surrounding environment on the target point, and improves the accuracy and consistency of the compensation result.

[0047] (3) Feature fusion and restoration capability: The uniqueness of the present invention lies in the fact that it adopts a multi-dimensional feature fusion and restoration strategy in the value-adding process. In the value-adding process, the three-dimensional features extracted by the 3D-CNN module and the one-dimensional features extracted by the 1DCNN-BiNbeats module are fused and restored through the spatiotemporal interactive fusion module to more effectively extract important information in the time domain and space domain. A loss function is generated in this process. This feature fusion and restoration method enhances information exchange between modules and reduces information loss, ultimately improving the accuracy of the value-adding results. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 It is a flow chart of a MODIS surface temperature compensation method based on spatiotemporal feature fusion according to an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of a spatiotemporal feature fusion network according to an embodiment of the present invention;

[0051] Figure 3 is a structural diagram of a spatiotemporal interactive fusion module according to an embodiment of the present invention;

[0052] Figure 4 is a structural diagram of a multi-dimensional feature synthesis module according to an embodiment of the present invention;

[0053] Figure 5 This is a graph of surface temperature compensation results in Yutian area according to an embodiment of the present invention;

[0054] Figure 6 This is a graph of the surface temperature compensation results in the Wenchuan area according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0056] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0057] The professional terms involved in the embodiments are explained below:

[0058] MODIS data: MODIS (Moderate Resolution Imaging Spectroradiometer) is a sensor on an Earth observation satellite launched by NASA that can capture spectral images of the Earth's surface at medium resolution. MODIS provides multispectral data covering the atmosphere, land and ocean, and is widely used in environmental monitoring, climate research, vegetation analysis and disaster assessment. Its data can be used for spatiotemporal series analysis to help scientists and decision makers better understand the Earth system and its changes.

[0059] Deep learning: Deep learning is a branch of machine learning that uses multi-layer neural networks to automatically extract and learn features from data by simulating the working mode of human brain neurons. It can process large-scale data sets and is widely used in image recognition, natural language processing, and speech recognition. Deep learning optimizes model parameters through the back-propagation algorithm, making it perform well in complex tasks, thus surpassing traditional machine learning methods in many application scenarios.

[0060] CNN: Convolutional Neural Network (CNN) is a deep learning model that is particularly suitable for image processing and computer vision tasks. It extracts features through convolutional layers, uses local receptive fields and weight sharing to reduce the number of parameters and improve computational efficiency. CNN usually contains multiple convolutional layers, pooling layers, and fully connected layers, which can automatically learn and identify important patterns and features in images. It is widely used in image classification, object detection, and image generation.

[0061] N-BEATS: N-BEATS (Neural Basis Expansion Analysis for Time Series) is a time series forecasting model based on deep learning. It processes time series data through a multi-layer fully connected network and uses a block structure to capture different patterns. N-BEATS does not rely on prior knowledge and can automatically learn the trend and seasonality of time series. The model generates forecasts through a feedforward neural network and uses a regression loss function to optimize performance. It is suitable for various time series forecasting tasks, with excellent performance and high flexibility.

[0062] Existing LST reconstruction methods mainly include the following categories: reconstruction methods based on spatial information, reconstruction methods based on temporal information, and methods that integrate temporal and spatial information. Spatial information reconstruction is to interpolate based on the spatial correlation between missing pixels and their adjacent clear sky pixels, such as spline function, regression tree analysis, inverse distance weighting, and Kriging interpolation. Such methods do not require additional reference information and are easy to implement. However, such methods are greatly affected by changes in surface cover and surface elevation, and have problems such as unclear reconstructed images and poor accuracy. Temporal reconstruction is also called multi-temporal information reconstruction or time series information reconstruction. Its principle is to reconstruct missing pixels using surface temperatures of adjacent dates at the same location. It mainly includes: harmonic analysis method, singular spectrum analysis method, method based on daily temperature cycle model, method based on physical model, method based on multi-temporal information robust regression, method based on SG filtering, etc. Although the reconstruction method based on temporal filtering can smooth the daily variation of surface temperature, the reconstruction effect will deteriorate when there are more missing data in adjacent periods. The most basic strategy for the reconstruction method of integrated spatiotemporal information is to apply spatial and temporal reconstruction methods in sequence: generally, fill in the time domain and then fill in the spatial domain to achieve the reconstruction of surface temperature data. Although this type of method takes temporal and spatial information into consideration during the reconstruction process, it does not achieve true spatiotemporal joint modeling, which restricts the improvement of the accuracy of the reconstruction results.

[0063] In addition to the above methods, in recent years, reconstruction methods based on machine learning have attracted more and more attention and use from scholars due to their strong learning ability and high robustness, such as artificial neural networks, deep convolutional networks, and long short-term memory networks. Nowadays, in the study of time series prediction and complementation, the most commonly used are CNN models and LSTM models. CNN can automatically extract spatial features in data through convolutional layers, which is suitable for multidimensional data such as images and time series. Due to the local connection and shared weights of convolution operations, CNN is usually more efficient than fully connected layers, especially when processing large-scale data. However, when processing time series data, CNN has difficulty capturing the long-term dependency of data, especially when the sequence is long. The LSTM model is designed to capture long-term dependencies and can effectively process long-term relationships in time series data. LSTM performs well when processing fast-changing or irregular data and is suitable for dynamic and complex sequence data. However, the recursive structure of the LSTM model makes the training and inference process relatively slow, especially in the case of long sequence data, and the computing resources are consumed more. Due to its recursive nature, LSTM is difficult to perform parallel computing during training, which limits its scalability. LSTM usually has more parameters, which may lead to higher training time and larger model storage requirements. The N-Beats model proposed today is different from the recursive structure of LSTM. N-Beats adopts a fully connected feedforward neural network structure, which makes the model easier to parallelize and improves training speed and efficiency. In addition, N-Beats can handle longer time series and perform well in feature extraction, especially in the absence of time dependence, which can effectively avoid the gradient vanishing problem in long sequence learning. In addition, N-Beats learns the features of time series by stacking multiple blocks, without manual feature selection or additional pre-processing, which simplifies the modeling process. However, these models lack an efficient feature fusion mechanism during training, usually relying only on a single feature for complementation, and cannot effectively capture the complex relationship between data. This lack of comprehensive feature extraction and fusion makes the complementation results vulnerable to noise and outliers, reducing the robustness and generalization ability of the model.

[0064] In view of the above problems, the present invention proposes a MODIS surface temperature compensation method based on spatiotemporal feature fusion, such as Figure 1 As shown, the specific steps include:

[0065] Get the value data to be supplemented;

[0066] The data to be supplemented are input into the spatiotemporal feature fusion network model to obtain the supplementation results, wherein the spatiotemporal feature fusion network model is obtained through training with a training set, the training set is historical surface temperature data, and the spatiotemporal feature fusion network model is constructed using a neural network, which is used to perform spatiotemporal interactive fusion of the data to be supplemented and perform multi-dimensional feature synthesis to obtain the supplementation results.

[0067] Specifically, this method uses one-dimensional time series and three-dimensional spatiotemporal series as input features, fully integrates the feature information of the two, and extracts the feature information of surface temperature in time and space in a relatively comprehensive manner. The model consists of two parallel branch modules and two feature fusion modules. Each branch module is used to process different types of data. The 1D-CNN-BiNbeats module consists of a 1D-CNN model and a bidirectional N-Beats model, which is used to process one-dimensional time series and deeply extract the feature information of surface temperature in time series. The 3D-CNN module extracts the feature information of surface temperature in time and space through the 3D-CNN model. It should be noted that a multi-dimensional feature fusion module is introduced in the model to extract and fuse the features of each branch, thereby enhancing the information exchange between modules and reducing information loss. The model can simultaneously and efficiently realize the mining of one-dimensional time features and three-dimensional spatiotemporal features of surface temperature, and based on the multi-dimensional feature fusion module, it realizes the use of multi-dimensional features for surface temperature compensation, and obtains a higher-precision surface temperature compensation result.

[0068] Furthermore, obtaining a training set includes:

[0069] Get raw surface temperature data;

[0070] The original surface temperature data are spliced, cropped and reprojected to obtain the training set.

[0071] Specifically, the MODIS data product used in the present invention is in HDF format, and the original image needs to be spliced, cropped and reprojected with the help of the MRT (MODISProjection Tool) software provided by NASA, because the projection of MODIS data is generally Sinusoidal projection, which is not suitable for conventional data viewing methods, so the acquired data needs to be reprojected. The present invention uses MOD11A2 data from 2003 to 2015 with a size of 900×900×598 for a period of 8 days. MOD11A2 data comes from the Terra satellite, is less affected by factors such as cloud cover, and the data is more accurate.

[0072] Furthermore, the spatiotemporal feature fusion network model includes: 1D CNN-BiNbeats module, 3D-CNN module, spatiotemporal interaction fusion module and multi-dimensional feature synthesis module;

[0073] 1D CNN-BiNbeats module, used to extract one-dimensional feature information of surface temperature;

[0074] 3D-CNN module, used to extract three-dimensional feature information of surface temperature;

[0075] The spatiotemporal interactive fusion module is used to fuse one-dimensional feature information and three-dimensional feature information;

[0076] The multi-dimensional feature synthesis module is used to synthesize the fused one-dimensional feature information and three-dimensional feature information.

[0077] Specifically, Figure 2 As shown, the spatiotemporal feature fusion network consists of a 1D CNN-BiNbeats module, a 3D-CNN module and two feature fusion modules. Among them, the 1D CNN-BiNbeats module is good at processing one-dimensional sequence data, which is used to extract the characteristic information of surface temperature in the time series; the 3D-CNN module extracts the three-dimensional features of the surface temperature through the 3D-CNN model to better mine the spatiotemporal characteristic information of the surface temperature; in addition, the present invention designs a multi-dimensional feature fusion module and fusion loss, which can extract and fuse the features of each branch and strengthen the information exchange between different branches, among which, Fl: data 1 feature, F1': fused data 1, feature f1: branch 1 output feature, F2: data 2 feature, F2': fused data 2 feature, f2: branch 2 output feature, ConvlD: 1x1 convolution Pooling Layer: pooling layer, L rec : Fusion restoration loss, Bi-BB: Bidirectional basis block, Conv3D: 3x3 convolution L final : Make up for the loss of precision.

[0078] Furthermore, the one-dimensional characteristic information of the ground surface temperature is extracted including:

[0079] 1D-CNN unit is used to extract the data to be complemented for one-dimensional time series feature extraction;

[0080] Use bidirectional N-Beats units to capture the basic patterns of one-dimensional time series features and learn long-term dependencies;

[0081] According to the long-term dependency, one-dimensional feature information is obtained.

[0082] Specifically, 1D convolutional neural network (1D-CNN) is a deep learning architecture specifically designed to process one-dimensional data (such as time series or text data). This model can effectively extract the features of input data through local connections and weight sharing, and is suitable for tasks such as classification, regression, and sequence prediction. In 1D-CNN, the convolution operation extracts local features of the input data through the convolution kernel. Let the input vector be X and the convolution kernel be , then the convolution operation can be expressed as:

[0083]

[0084] in, is the output of the convolutional layer, is the size of the convolution kernel, Indicates the position of the current output element. Represents the position of the element within the convolution kernel or window. The output of the convolution layer is usually processed by an activation function to introduce nonlinear characteristics. The commonly used activation function is ReLU (rectifier linear unit), which is defined as:

[0085]

[0086] In order to reduce the feature dimension and reduce the computational complexity, 1D-CNN uses a pooling layer. The maximum pooling operation is defined as:

[0087]

[0088] The window size determines the scope of the pooling operation. Finally, the extracted features are passed to the fully connected layer for integration. Let the input be the feature vector , the weight matrix is , the output of the fully connected layer It can be expressed as:

[0089]

[0090] in, is the bias term. Through the above structure, 1D-CNN can effectively extract time series features from one-dimensional data, thereby improving the performance of the model in related tasks. This model shows significant advantages when processing input data with time series dependencies.

[0091] Further, the N-Beats unit includes: an input layer, a basic block, a stacking layer, and an output layer;

[0092] Input layer, used to input one-dimensional time series features;

[0093] Basic blocks are used to capture the information flow from the past to the present in the time series, extract the forward dependency of the sequence, and capture the reverse information flow from the future to the present, extract the backward dependency of the sequence, and obtain the prediction results based on the forward dependency and backward dependency;

[0094] Stacking layers to correct prediction results;

[0095] The output layer is used to output the corrected prediction results.

[0096] Specifically, the N-Beats (Neural Basis Expansion Analysis Time Series Forecasting) model is an innovative time series forecasting method, which is mainly composed of multiple stacked structural blocks to efficiently extract basic features such as trends and seasonality of the series. The key to the design of this model lies in the superposition of multiple basic blocks (BasicBlock), each of which is composed of a fully connected neural network, which improves the forecasting effect through recursion. The model structure is as follows:

[0097] Input layer: input time series data The model breaks it into appropriately sized segments for prediction.

[0098] Basic Block: Each basic block contains a multi-layer fully connected neural network, and the output is divided into two parts: "forward prediction" and "backward reconstruction", which are used to generate future values ​​and reconstruct the current input sequence respectively. : Used to predict future time points and output backward : Used to reconstruct the input, help the model learn the basic patterns of the time series, and form a low-dimensional representation of the data.

[0099] Stacked structure: The model recursively corrects the prediction error layer by layer by stacking multiple basic blocks, and accumulates the prediction results of each layer in sequence to improve the prediction accuracy.

[0100] Output layer: The final forward prediction result is obtained by adding the forward outputs of each basic block.

[0101] Assume that the entry time series is , the forward and backward outputs of each basic block are recorded as and The final prediction value of the model is and reconstruction value The definition is as follows:

[0102]

[0103]

[0104] in, is the number of basic blocks, Indicates The N-BEATS model weights and fuses the prediction results of all basic blocks to generate the final prediction result:

[0105]

[0106] in, and Represent the weighting coefficients of the previous and backward outputs respectively.

[0107] The present invention improves the N-Beats model and constructs a bidirectional N-Beats layer. The forward N-Beats layer captures the information flow from the past to the present in the time series and extracts the forward dependency of the sequence. The backward N-Beats layer captures the reverse information flow from the future to the present and extracts the backward dependency of the sequence. In other words, the basic block is improved into a bidirectional basic block, and finally connected by concat to output the corresponding one-dimensional features. The bidirectional structure can enhance the model's understanding of the overall time series information, thereby possibly improving the accuracy of the prediction.

[0108] In summary, the specific approach of the 1D CNN-BiNbeats module is as follows: first, the one-dimensional time series where the missing points are located is taken as input and input into the 1D-CNN model for feature extraction to reduce redundant data, so that the data volume becomes smaller and the data features remain unchanged. Then, the bidirectional N-beats model is used to learn the basic patterns, temporal features and long-term dependencies of the data, capture the information flow from the past to the present and the reverse information flow from the future to the present in the time series, extract the forward dependency and backward dependency of the sequence, and finally output the corresponding one-dimensional features through concat connection.

[0109] Furthermore, the 3D-CNN module includes: an input layer, a convolutional layer, an activation layer, a pooling layer, and an output layer;

[0110] The input layer is used to input the neighborhood spatiotemporal series data of missing points;

[0111] Convolutional layer, used to extract three-dimensional features of spatiotemporal sequence data through three-dimensional convolution operation;

[0112] The activation layer is used to perform nonlinear processing on three-dimensional features using activation functions;

[0113] Pooling layer, used to reduce feature dimensionality and extract representative three-dimensional features;

[0114] The output layer is used to output three-dimensional feature information.

[0115] Specifically, the 3D-CNN module receives the 3×3 neighborhood spatiotemporal sequence data of the missing point through the 3D-CNN model, that is, the dimension of the input data is 3×3×598. The module extracts spatiotemporal features by performing convolution operations on the neighborhood data. Specifically, the 3D-CNN module uses its three-dimensional convolution kernel to extract features in both time and space dimensions, thereby capturing the deep features of the data. This process helps to build a richer three-dimensional feature representation and provide a more accurate basis for subsequent value-adding tasks. Finally, the three-dimensional features output by the model will be used to merge with the results of other modules to further improve the accuracy and robustness of the value-adding.

[0116] 3D-CNN (3D Convolutional Neural Network) is a deep learning architecture designed for processing three-dimensional data and can effectively extract spatiotemporal features. In this invention, the 3D-CNN model is used to process the 3×3×598 neighborhood spatiotemporal sequence data of missing points to enhance the model's understanding of complex temporal patterns. Its model structure is:

[0117] Input layer: The input data is , which represents the spatiotemporal sequence of the 3×3 neighborhood of the missing point.

[0118] Convolutional layer: extracts features through three-dimensional convolution operation. The convolution operation can be expressed as:

[0119] (8)

[0120] in, is a three-bit convolution kernel, are the sizes of the convolution kernel in spatial and temporal dimensions, respectively. is the index of the output tensor, indicating the position in three-dimensional space, is the index of the convolution kernel, which is used to traverse each element of the convolution kernel.

[0121] Activation function: The output of the convolutional layer is usually processed by an activation function to introduce nonlinearity. The commonly used activation function is ReLU, which is defined as:

[0122]

[0123] in, is the output tensor of the activation layer.

[0124] Pooling layer: In order to reduce the feature dimension and extract more representative features, 3D-CNN uses the pooling layer. The formula for maximum pooling is:

[0125]

[0126] in: is the output tensor of the pooling layer, It is the size of the pooling window, which determines the scope of the pooling operation.

[0127] Output layer: Finally, after multiple layers of convolution and pooling, the output feature representation is , used for subsequent value-added tasks.

[0128] The 3D-CNN model can effectively capture the spatiotemporal features around missing points and improve the ability to understand data. This enables the model to more accurately predict missing data points during the filling process, thereby improving the overall prediction performance. By utilizing three-dimensional convolution operations, 3D-CNN not only focuses on the correlation of spatial neighborhoods, but also considers dynamic changes in the time dimension, providing a richer information basis for the filling task.

[0129] Furthermore, fusing the one-dimensional feature information and the three-dimensional feature information includes:

[0130] The feature dimensions of the one-dimensional feature information and the three-dimensional feature information are unified through the convolution operation to obtain dimensionally unified features;

[0131] The dimensional unified features are fused to obtain the fused feature information.

[0132] Specifically, the steps of the spatiotemporal interaction fusion module are as follows: embed this module in two modules (1D CNN-BiNbeats module and 3D-CNN module). This module focuses on processing features of different dimensions and unifies the feature dimensions of each branch through convolution operations to ensure that these features can be effectively fused in the same space. Subsequently, these unified dimension features are fused through feature addition (Add operation). Feature addition refers to adding two features element by element to increase the amount of information while keeping the dimension unchanged. The fused features are passed back to their respective branch networks through the convolution kernel for further processing. It should be noted that multi-dimensional feature fusion loss is introduced in this module. (See Figure 3 ).exist Figure 3 In the above, Fl: data 1 feature, F1': fused data 1 feature, F2: data 2 feature, F2': fused data 2 feature, Conv: convolution layer, Kernel: convolution kernel, ADD: addition, Concat: concatenation, L rec: Fusion restoration loss. The specific calculation method is: first, concatenate the two input features (F1, F2) and the two output features (F1', F2') respectively, and then calculate the loss in the fusion process through the loss function. The feature fusion loss function designed by the present invention aims to ensure that features of different dimensions can maintain the integrity and similarity of information during the fusion process, and enhance the synergy between the features of each branch, so that the fused features are more accurate. By introducing the spatiotemporal interactive fusion module, features of different dimensions or sources are effectively fused, the feature expression ability is improved, the robustness of the model is enhanced, and the information loss is effectively reduced. Each branch can not only process its own data independently, but also share and interact with other branches during the fusion process, and effectively absorb useful information from other branches. This design not only enhances the feature interaction between branches, but also reduces the loss of information, achieves more efficient feature fusion, and improves the accuracy and generalization ability of the final complementation result.

[0133] Furthermore, synthesizing the fused one-dimensional feature information and three-dimensional feature information includes:

[0134] Performing restoration processing on the fused one-dimensional feature information and three-dimensional feature information;

[0135] Performing dimensionality reduction operation on the restored one-dimensional feature information and three-dimensional feature information, and extracting the reduced one-dimensional features and three-dimensional features;

[0136] Mapping the one-dimensional features and the three-dimensional features after dimensionality reduction to obtain one-dimensional mapping features and three-dimensional mapping features;

[0137] The one-dimensional mapping features and the three-dimensional mapping features are synthesized to obtain the complement result.

[0138] Specifically, the specific steps of the multi-dimensional feature synthesis module are as follows: after each branch network completes feature extraction and fusion, the model uses the convolution layer to reduce the dimension of the output features (f1, f2) from the two branches and extract features, such as Figure 4 As shown, fl: branch 1 output feature, f2: branch 2 output feature, Conv: convolution layer, Add: addition, FC: fully connected layer, L final : The accuracy loss of the complementation. Then, the features processed by the convolutional layer are further mapped through the fully connected layer to improve the expressiveness of the features. Finally, the two feature elements output by the fully connected layer are added to obtain the final complementation result, and a loss is set to evaluate the complementation result. The present invention is in the process of training and The mean square error (MSE) is used as the loss function. It is a commonly used numerical standard and the calculation formula is as follows:

[0139]

[0140] represents the i-th true value, represents the i-th predicted value, is the total number of data points.

[0141] These two loss functions together constitute the loss function of the present invention, and the total loss It can be expressed as:

[0142]

[0143] This design not only strengthens the expression of features in each dimension, but also realizes the unified processing of multi-dimensional features at the overall level of the model, learns more spatiotemporal information and multi-dimensional features, thereby improving the performance of the model in complex tasks and enabling the model to more accurately capture the spatiotemporal variation characteristics of surface temperature.

[0144] Furthermore, when training the spatiotemporal feature fusion network model, the Adam optimization algorithm is used to update the parameters of the spatiotemporal feature fusion network model.

[0145] The present invention also includes: model training and model testing and result evaluation;

[0146] Model training:

[0147] The model data input is one-dimensional time series and three-dimensional spatiotemporal series data. The size of the MODIS data used is 900×900×598. 70% of the data set is used as the training set for model training. During the training process, the model hyperparameters are first set: learning rate: 0.001; epoch: 100; batch size: 64, optimizer: Adam; loss function: MSE; Dropout parameter: 0.5.

[0148] This model first obtains the prediction result by passing the data from the input layer to the output layer through layer-by-layer calculation between each network layer, and then adjusts the network parameters based on the gradient descent algorithm to reduce the error index between the prediction result and the reference value. This study uses the Adam optimization algorithm in the training process. The Adam optimizer can automatically adjust the learning rate and momentum values ​​according to the current training status, better cope with different data dimensions and model complexity, and thus significantly improve the training effect of the model. Compared with other optimization algorithms, the Adam optimizer obtains momentum by calculating the mean of the gradient. By introducing momentum, the Adam optimizer can consider the influence of the previous gradient when updating the parameters, so that the training process can be more stable. The adaptive learning rate dynamically adjusts the size of the learning rate and the update step size of the parameters by calculating the variance of the gradient to meet the update requirements of different parameters. The update principle of the Adam optimizer is:

[0149]

[0150] in, represents the loss function; Represents the current gradient; , and , Represent the mean and variance of the gradient at the current and previous moments respectively; and Indicates the revised results of the estimated value; Represents the learning rate, the default value is 0.001; and are the decay exponents of the mean and variance of the gradient respectively; represents the smoothing term, which is used to prevent the denominator from being 0; t is the number of iterations, Indicates the updated parameters.

[0151] Model testing and result evaluation:

[0152] 30% of the data set was selected as the test set and input into the trained model. The inversion accuracy of the spatiotemporal feature fusion network model was evaluated using two performance indicators: correlation coefficient (R) and root mean square error (RMSE). Root mean square error (RMSE) is a commonly used indicator that is often used to measure the size of the error between predicted data and reference data; while the correlation coefficient (R) is an indicator used to evaluate the degree of association between two variables, and its value range is between -1 and 1. The calculation methods of root mean square error (RMSE) and correlation coefficient (R) are:

[0153]

[0154]

[0155] in, is the actual observed value, is the model prediction value, Represents the total number of sample points, and are the average of the predicted values ​​and the average of the actual observed values, respectively.

[0156] The following takes Yutian area in Xinjiang and Wenchuan area in Sichuan as examples to describe this embodiment in detail:

[0157] In Yutian area of ​​Xinjiang (center location 79.9°E, 37.1°N) and Wenchuan area of ​​Sichuan (center location 103.4°E, 31.0°N), the method of this embodiment is compared with the following three methods: a surface temperature compensation method based on a CNN model (CNN method), a surface temperature compensation method based on an N-beats model (N-beats method), and a surface temperature compensation method based on a hybrid model (SCLSTM method) which currently has the best compensation effect.

[0158] In order to evaluate the reconstruction effect of the model, the reconstruction effects of the method of this embodiment and the comparative method were qualitatively and quantitatively analyzed according to the prior art using the strategy of “removal-reconstruction-comparison”. The “removal-reconstruction-comparison” strategy is to first remove some existing values ​​from the complete LST image and fill them with zeros, and then reconstruct the missing pixels using the method of this embodiment and the comparative method, and finally compare and analyze the reconstructed LST values ​​with the original data. Taking the image of the 30th issue of Yutian area in 2008 as an example, the following is a comparison image and accuracy comparison table before and after the supplementation according to the method of this embodiment. Figure 5 It can be seen that the texture of the image after complementation is delicate and natural, without obvious boundary effects.

[0159] It can be seen from Table 1 that the method in this embodiment has the best compensation result, with the highest correlation coefficient of 0.951 and the lowest RMSE of 0.903, both of which are better than the comparison method.

[0160] Table 1 Comparison results of the method in this embodiment with the CNN method, N-beats method, and SCLSTM method (Yutian area)

[0161]

[0162] The following is the supplementation result map and accuracy comparison table of the 15th issue of the Wenchuan area in 2007:

[0163] from Figure 6It can be seen that the texture of the image after the supplementation is delicate and natural, and there is no obvious boundary effect. As can be seen from Table 2, the supplementation result of the method in this embodiment is the best, with the highest correlation coefficient of 0.952 and the lowest RMSE of 0.709, which are still better than the comparison method, indicating that the method in this embodiment has universal applicability.

[0164] Table 2 Comparison results of the method in this embodiment with the CNN method, N-beats method, and SCLSTM method (Wenchuan area)

[0165]

[0166] Through the actual application of this technical solution, the following two conclusions can be drawn:

[0167] (1) The method of this embodiment can better explore the characteristics of the surface temperature in the spatiotemporal sequence, so that the reconstructed image has no obvious boundary effect, the texture information is more natural, and the reconstructed LST accuracy is more stable and accurate.

[0168] (2) The method of this embodiment has strong regional applicability. It can achieve good reconstruction results in areas with a small number of missing values, areas with a large number of missing values, areas with smooth terrain, areas with large terrain fluctuations, and areas with different climatic conditions, which proves the regional practicability of this embodiment.

[0169] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. The MODIS surface temperature compensation method based on spatiotemporal feature fusion is characterized by: include: Get the value data to be supplemented; The data to be supplemented is input into the spatiotemporal feature fusion network model to obtain the supplementation result, wherein the spatiotemporal feature fusion network model is obtained by training with a training set, the training set is historical surface temperature data, the spatiotemporal feature fusion network model is constructed using a neural network, and the neural network is used to extract one-dimensional time series features and three-dimensional spatiotemporal series features, and the one-dimensional time series features and the three-dimensional spatiotemporal series features are feature-fused to obtain the supplementation result.

2. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 1 is characterized in that: Acquiring the training set includes: Get raw surface temperature data; The original surface temperature data is spliced, cropped and reprojected to obtain the training set.

3. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 1 is characterized in that: The spatiotemporal feature fusion network model includes: a one-dimensional time series feature extraction module, a three-dimensional spatiotemporal sequence feature extraction module, a spatiotemporal interactive fusion module and a multi-dimensional feature synthesis module; The one-dimensional time series feature extraction module is used to extract one-dimensional feature information of the surface temperature; The three-dimensional spatiotemporal sequence feature extraction module is used to extract three-dimensional feature information of surface temperature; The spatiotemporal interactive fusion module is used to fuse the one-dimensional feature information and the three-dimensional feature information; The multi-dimensional feature synthesis module is used to synthesize the fused one-dimensional feature information and three-dimensional feature information.

4. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 1 is characterized in that: Extracting one-dimensional characteristic information of surface temperature includes: Extracting the to-be-complemented value data using a 1D-CNN unit to perform one-dimensional time series feature extraction; Using bidirectional N-Beats units to capture the basic patterns of the one-dimensional time series features and learn long-term dependencies; The one-dimensional feature information is obtained according to the long-term dependency relationship.

5. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 4 is characterized in that: The N-Beats unit includes: an input layer, a basic block, a stacking layer and an output layer; The input layer is used to input the one-dimensional time series feature; The basic block is used to capture the information flow from the past to the present in the time series, extract the forward dependency of the sequence and capture the reverse information flow from the future to the present, extract the backward dependency of the sequence, and obtain the prediction result according to the forward dependency and the backward dependency; The stacked layer is used to correct the prediction result; The output layer is used to output the corrected prediction result.

6. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 3 is characterized in that: The three-dimensional spatiotemporal sequence feature extraction module includes: an input layer, a convolution layer, an activation layer, a pooling layer and an output layer; The input layer is used to input the neighborhood spatiotemporal series data of missing points; The convolution layer is used to extract the three-dimensional features of the spatiotemporal sequence data through a three-dimensional convolution operation; The activation layer is used to perform nonlinear processing on the three-dimensional features using an activation function; The pooling layer is used to reduce feature dimensions and extract representative three-dimensional features; The output layer is used to output the three-dimensional feature information.

7. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 3 is characterized in that: Fusion of the one-dimensional feature information and the three-dimensional feature information includes: Unifying the feature dimensions of the one-dimensional feature information and the three-dimensional feature information through a convolution operation to obtain dimensionally unified features; The unified features of the dimensions are fused to obtain fused feature information.

8. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 3 is characterized in that: The step of synthesizing the fused one-dimensional feature information and three-dimensional feature information includes: Performing restoration processing on the fused one-dimensional feature information and three-dimensional feature information; Performing a dimensionality reduction operation on the restored one-dimensional feature information and three-dimensional feature information to extract the reduced one-dimensional features and three-dimensional features; Mapping the one-dimensional features and the three-dimensional features after dimensionality reduction to obtain one-dimensional mapping features and three-dimensional mapping features; The one-dimensional mapping feature and the three-dimensional mapping feature are synthesized to obtain a complementation result.

9. The MODIS surface temperature compensation method based on spatiotemporal feature fusion according to claim 1 is characterized in that: When training the spatiotemporal feature fusion network model, the Adam optimization algorithm is used to update the parameters of the spatiotemporal feature fusion network model.

Images