Soil humidity data missing filling method and device, electronic equipment and medium
By constructing a transfer learning model with a pre-trained set and a fine-tuned training set using multi-source data, and combining a temporal convolutional network and a self-attention mechanism, the problem of missing remote sensing soil moisture data was solved, improving the accuracy of data filling and the adaptability of the model.
Patent Information
- Application Number
- CN202511518461.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-23
- Publication Date
- 2026-01-16
AI Technical Summary
Existing remote sensing soil moisture datasets often suffer from data gaps due to various factors, resulting in low incompleteness accuracy. This is especially true in complex terrain and variable climate conditions, where traditional interpolation methods are insufficient in accuracy. Meanwhile, machine learning-based methods ignore terrain features and meteorological factors, making it impossible to accurately fill in missing data.
A soil moisture data missing incomplete model is constructed using multi-source soil moisture data to build a pre-training set and a fine-tuning training set for transfer learning. The model combines a temporal convolutional network, a bidirectional long short-term memory network, and a self-attention mechanism module to extract the spatiotemporal features of the soil moisture data and output the incomplete results.
It improves the accuracy of filling in data-scarce areas and the model's generalization ability, achieving high-precision filling of soil moisture data.
Smart Images
Figure CN121350430A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing data processing technology, and in particular to a method, apparatus, electronic device, and medium for filling in missing soil moisture data. Background Technology
[0002] Soil moisture, also known as soil water content, is a physical quantity that indicates the degree of dryness or wetness of soil, and is a relative variable of soil water content. The spatiotemporal dynamics of soil moisture have significant application value in agricultural management (such as irrigation and crop yield estimation), flood and drought prediction, water quality management, and nature conservation, especially remote sensing soil moisture datasets with high temporal resolution. However, due to various factors such as precipitation, evaporation, and seasonal variations, remote sensing soil moisture datasets often experience data gaps, which affects the completeness and accuracy of soil moisture data, thus limiting their effectiveness in practical applications.
[0003] In existing technologies, missing soil moisture data can be filled in using traditional interpolation algorithms and modern machine learning-based methods. However, traditional interpolation methods such as linear interpolation and Kriging interpolation, while computationally simple and efficient, suffer from low accuracy when dealing with complex terrain and variable climate conditions. Modern machine learning-based methods, especially deep learning techniques, can learn the spatiotemporal characteristics of soil moisture through large amounts of training data, thus significantly improving the accuracy of data filling. However, most existing deep learning methods rely on a single data source, ignoring the significant influence of terrain features and meteorological factors on soil moisture distribution, and therefore cannot accurately fill in missing soil moisture data. Summary of the Invention
[0004] Based on this, it is necessary to provide a method, device, electronic device, and medium for filling in missing soil moisture data to address the above-mentioned technical problems. This method can effectively solve the problem of insufficient training sample data in data-scarce regions, improve the generalization ability and filling accuracy of the model, and fully integrate the parallel computing capabilities of the temporal convolutional network module, the advantages of the bidirectional long short-term memory network module in temporal modeling, and the feature selection capabilities of the self-attention mechanism module. This enables the effective extraction of spatiotemporal features of soil moisture data and improves the prediction accuracy of the filled data results.
[0005] In a first aspect, embodiments of the present invention provide a method for filling in missing soil moisture data, comprising: Obtain the soil moisture data sequence corresponding to the soil moisture data to be filled in the ESA CCI soil moisture data. The soil moisture data sequence is composed of multi-dimensional feature data corresponding to multi-source soil moisture related data. The multi-source soil moisture related data includes: surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data. The soil moisture data sequence is input into a trained soil moisture data missing imputation model, and the soil moisture data missing imputation model outputs the imputation data corresponding to the soil moisture data to be imputed in ESA CCI. The soil moisture data missing imputation model is trained by transfer learning based on a pre-training set and a fine-tuned training set. The soil moisture data missing imputation model includes: a temporal convolutional network module, a bidirectional long short-term memory network module, a self-attention mechanism module, and an output module.
[0006] In one embodiment, the soil moisture data missing imputation model is trained by transfer learning based on a pre-training set and a fine-tuned training set, including: The pre-training set is constructed based on the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data. The fine-tuning training set is constructed based on ESA CCI soil moisture data, surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data. The pre-training set is input into the initial soil moisture data missing imputation model for training. The weight parameters of the initial soil moisture data missing imputation model are adjusted according to the preset loss function until the model converges, and the intermediate soil moisture data missing imputation model after training is obtained. The fine-tuned training set is input into the intermediate soil moisture data missing imputation model for training. The weight parameters of the intermediate soil moisture data missing imputation model are adjusted according to the preset loss function until the model converges, and the soil moisture data missing imputation model after training is obtained.
[0007] In one embodiment, the pre-training set includes: multiple pre-training sample data and first label sample data corresponding to each pre-training sample data. The step of constructing the pre-training set based on the surface temperature data, normalized difference vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data includes: The surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, and ground verification data at each preset time are determined as pre-training sample data, and the soil moisture reanalysis data at the same preset time are determined as the first label sample data corresponding to the pre-training sample data. The pre-training set is constructed based on multiple pre-training sample data and the first label sample data corresponding to each pre-training sample data; The fine-tuning training set includes: multiple fine-tuning training sample data and second-label sample data corresponding to each fine-tuning training sample data. The fine-tuning training set is constructed based on ESA CCI soil moisture data, the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data, including: The surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data at each preset time are determined as fine-tuning training sample data, and the ESA CCI soil moisture data at the same preset time are determined as the second label sample data corresponding to the fine-tuning training sample data. The fine-tuning training set is constructed based on multiple fine-tuning training sample data and the second label sample data corresponding to each fine-tuning training sample data.
[0008] In one embodiment, obtaining the soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled includes: Obtain multi-source soil moisture-related data corresponding to the ESA CCI soil moisture data to be filled; The multi-source soil moisture correlation data were subjected to coordinate alignment processing; Multidimensional feature extraction is performed on the processed multi-source soil moisture correlation data to obtain the soil moisture data sequence.
[0009] In one embodiment, the step of inputting the soil moisture data sequence into a trained soil moisture data missing imputation model, and outputting the imputation data result corresponding to the ESA CCI soil moisture data to be imputed through the soil moisture data missing imputation model, includes: The soil moisture data sequence is input into a temporal convolutional network module, and the initial time series features are extracted through the temporal convolutional network module. The initial time series features are input into the bidirectional long short-term memory network module, and the target time series features are extracted through the bidirectional long short-term memory network module. The target time series features are input into the self-attention mechanism module, and attention features are extracted through the self-attention mechanism module. The attention features are input to the output module, and the data filling results are obtained through the output module.
[0010] In one embodiment, the bidirectional long short-term memory (LSTM) network module includes: a first LTM network unit and a second LTM network unit. The initial time series features are input into the bidirectional LTM network module, and the target time series features are extracted through the bidirectional LTM network module, including: The initial time series features are input into the first long short-term memory network unit according to the forward time and the reverse time to obtain the forward time series features and the reverse time series features. The forward time series features and the reverse time series features are concatenated and input into the second long short-term memory network unit to obtain the target time series features.
[0011] In one embodiment, the temporal convolutional network module includes: a three-layer stacked temporal convolutional layer; inputting the fine-tuned training set into the intermediate soil moisture data missing imputation model for training; adjusting the weight parameters of the intermediate soil moisture data missing imputation model according to a preset loss function until the model converges; and obtaining the soil moisture data missing imputation model after training, including: The fine-tuned training set is input into the intermediate soil moisture data missing imputation model. The weight parameters of the third temporal convolution stacked layer, the second long short-term memory network unit, and the self-attention mechanism module are adjusted according to the preset loss function until the model converges, and the soil moisture data missing imputation model after training is obtained.
[0012] Secondly, embodiments of the present invention provide a soil moisture data missing filling device, comprising: The soil moisture data sequence acquisition module is used to acquire the soil moisture data sequence corresponding to the soil moisture data to be filled in the ESA CCI. The soil moisture data sequence is composed of multi-dimensional feature data corresponding to multi-source soil moisture related data, including: surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground verification data. The data imputation result acquisition module is used to input the soil moisture data sequence into the trained soil moisture data missing imputation model, and output the imputation data result corresponding to the soil moisture data to be imputed through the soil moisture data missing imputation model. The soil moisture data missing imputation model is trained by transfer learning based on the pre-training set and the fine-tuning training set. The soil moisture data missing imputation model includes: a temporal convolutional network module, a bidirectional long short-term memory network module, a self-attention mechanism module, and an output module.
[0013] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the soil moisture data missing filling method described in the first aspect.
[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the soil moisture data missing filling method described in the first aspect.
[0015] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: This invention provides a method for imputing missing soil moisture data. It acquires a soil moisture data sequence corresponding to the ESACCI soil moisture data to be imputed. This sequence comprises multi-dimensional feature data from multiple sources of soil moisture-related data, including: surface temperature data, normalized difference vegetation index (NDI), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data. The soil moisture data sequence is input into a trained soil moisture data imputation model, which outputs the imputed data corresponding to the ESACCI soil moisture data. Since the soil moisture data imputation model is trained using transfer learning on a pre-training set and a fine-tuned training set, it effectively solves the problem of insufficient training sample data in data-scarce regions, improving the model's generalization ability and imputation accuracy. The soil moisture data imputation model includes a temporal convolutional network module, a bidirectional long short-term memory (LSTM) network module, a self-attention mechanism module, and an output module. This model fully integrates the parallel computing capabilities of the temporal convolutional network module, the advantages of the bidirectional long short-term memory network module in temporal modeling, and the feature selection capabilities of the self-attention mechanism module, thereby achieving effective extraction of spatiotemporal features of soil moisture data and improving the prediction accuracy of the data filling results. Attached Figure Description
[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0018] Figure 1 A schematic flowchart illustrating a method for filling missing soil moisture data according to an embodiment of the present invention; Figure 2 A schematic diagram of a soil moisture data missing imputation model provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a soil moisture data missing filling device provided in an embodiment of the present invention. Detailed Implementation
[0019] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.
[0020] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.
[0021] In one embodiment, such as Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for filling missing soil moisture data according to an embodiment of the present invention. The method specifically includes the following steps: S10: Obtain the soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled.
[0022] The ESA CCI soil moisture data to be filled refers to the soil moisture data obtained through satellite data monitoring by the European Space Agency Climate Change Initiative (ESA CCI), a climate research project initiated by the European Space Agency. For example, the spatial resolution of this soil moisture data is 0.25° and the temporal resolution is day, used to represent the volumetric soil moisture at a soil depth of 0-5 cm, but it is not limited thereto. This invention does not specifically limit it, and those skilled in the art can set it according to the actual situation.
[0023] The soil moisture data sequence consists of multidimensional feature data corresponding to multi-source soil moisture related data. These multidimensional feature data include: statistical feature data, wavelet feature data, Fourier feature data, temporal variation feature data, and physical constraint feature data.
[0024] The aforementioned multi-source soil moisture-related data include: surface temperature data, normalized difference vegetation index (NDV), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data. Surface temperature data can be determined using MOD11A1 and MOD11A2 products; the NPV can be determined using MODIS MOD13C1 and MYD13C1 products; precipitation data can be from the publicly available CHM_PRE dataset; soil texture data can be determined using the SoilGrid250m product; and DEM data can be determined using the SRTM product. Reanalysis data includes ERA5, GLDAS, and SMAP data. Ground validation data uses observation station data from the ISMN network, but is not limited to this; this invention is not specifically limited, and those skilled in the art can set it according to actual conditions.
[0025] Optionally, based on the above embodiments, in some embodiments of the present invention, one implementation of S10 may be: S101: Obtain multi-source soil moisture-related data corresponding to the ESA CCI soil moisture data to be filled.
[0026] Specifically, multi-source soil moisture-related data corresponding to the soil moisture data to be filled in the ESA CCI will be obtained. These multi-source soil moisture-related data include: surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data.
[0027] It should be noted that, due to the absence of surface temperature data and normalized vegetation index (NDI), in order to improve the accuracy of filling missing data in the ESA CCI soil moisture data, when it is determined that surface temperature data and NDI are missing, the missing values of surface temperature data and NDI can be reconstructed using time series harmonic analysis algorithms or based on Savitzky-Golay filters.
[0028] S102: Perform coordinate alignment processing on multi-source soil moisture related data.
[0029] Specifically, a preset projection coordinate system is determined for the soil moisture data of the ESA CCI to be filled, such as a 0.25° geographic projection coordinate system. The surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground verification data are projected onto the preset projection coordinate system to ensure that the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground verification data are in the same preset projection coordinate system as the soil moisture data of the ESA CCI to be filled, thereby achieving coordinate alignment of multi-source soil moisture related data.
[0030] S103: Perform multidimensional feature extraction on the processed multi-source soil moisture correlation data to obtain soil moisture data sequences.
[0031] Specifically, after coordinate alignment processing of multi-source soil moisture-related data, multi-dimensional feature extraction processing is performed on the processed multi-source soil moisture-related data to obtain soil moisture data sequences.
[0032] Optionally, based on the above embodiments, in some embodiments of the present invention, the multidimensional feature extraction processing includes: statistical feature extraction processing, wavelet feature extraction processing, Fourier feature extraction processing, temporal variation feature extraction processing, and physical constraint feature extraction processing.
[0033] For example, statistical feature extraction can involve calculating the mean, variance, skewness, and kurtosis of a preset time series to obtain the overall distribution characteristics of multi-source soil moisture-related data over time. Wavelet feature extraction can involve using Daubechies wavelets for three-level wavelet decomposition to extract approximation and detail coefficients of multi-source soil moisture-related data at different scales, thereby capturing the local variation characteristics of the multi-source soil moisture-related data. Fourier feature extraction can involve calculating the power spectral density of multi-source soil moisture-related data over time using fast Fourier transform to extract the main periodicity and frequency characteristics. Temporal variation feature extraction can involve calculating the first and second differences of multi-source soil moisture-related data. Physical constraint feature extraction can involve setting a preset range of physically reasonable values for multi-source soil moisture-related data based on different soil types and climate regions, identifying and constraining outliers exceeding the preset range. However, this invention is not limited to these methods; those skilled in the art can set the appropriate values according to the actual situation.
[0034] S11: Input the soil moisture data sequence into the trained soil moisture data missing imputation model, and output the imputation data result corresponding to the soil moisture data to be imputed in ESA CCI through the soil moisture data missing imputation model.
[0035] The soil moisture data missing completion model is trained using transfer learning on a pre-training set and a fine-tuned training set. Transfer learning refers to first learning and adjusting the model's weight parameters using the pre-training set, and then further fine-tuning the model's weight parameters based on the weight parameters obtained during pre-training using the fine-tuned training set. This allows the feature information learned during training through the fine-tuned training set to more closely resemble the feature information trained on the pre-training set. Therefore, one possible implementation of the soil moisture data missing completion model using transfer learning on a pre-training set and a fine-tuned training set is as follows: S20: Construct a pre-training set based on surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data.
[0036] The pre-training set is used to pre-train the soil moisture data missing data imputation model.
[0037] Specifically, after obtaining surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data, a pre-training set is constructed based on the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data to pre-train the soil moisture data missing data imputation model.
[0038] Optionally, based on the above embodiments, in some embodiments of the present invention, the pre-training set includes: multiple pre-training sample data and first label sample data corresponding to each pre-training sample data. Based on this, one implementation of S20 may be: S201: Determine the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, and ground validation data at each preset time as pre-training sample data, and determine the soil moisture reanalysis data at the same preset time as the first label sample data corresponding to the pre-training sample data.
[0039] The preset time is used to align surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, and ground validation data with the pre-training sample data in terms of time.
[0040] S202: Construct a pre-training set based on multiple pre-training sample data and the first label sample data corresponding to each pre-training sample data.
[0041] Specifically, for surface temperature data, normalized vegetation index (NVI), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data, surface temperature data, NVI, precipitation data, DEM data, soil texture data, and ground validation data at each preset time are determined as pre-training sample data. Soil moisture reanalysis data at the same preset time are determined as the first label sample data of the pre-training sample data. Based on multiple pre-training sample data and the first label sample data corresponding to each of the multiple pre-training sample data, a pre-training set is constructed for pre-training the soil moisture data missing data imputation model.
[0042] S21: Construct a fine-tuning training set based on ESA CCI soil moisture data, surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data.
[0043] The fine-tuning training set is used to fine-tune the weight parameters of the soil moisture data missing imputation model.
[0044] Specifically, after obtaining ESA CCI soil moisture data, surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data, a fine-tuning training set is constructed based on the ESA CCI soil moisture data, surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data to fine-tune the soil moisture data missing data imputation model.
[0045] Optionally, based on the above embodiments, in some embodiments of the present invention, the fine-tuning training set includes: multiple fine-tuning training sample data and second label sample data corresponding to each fine-tuning training sample data. Based on this, one implementation of S21 may be: S211: Determine the surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data for each preset time as fine-tuning training sample data, and determine the ESA CCI soil moisture data at the same preset time as the second label sample data corresponding to the fine-tuning training sample data.
[0046] S212: Construct a fine-tuning training set based on multiple fine-tuning training sample data and the second label sample data corresponding to each fine-tuning training sample data.
[0047] Specifically, for surface temperature data, normalized vegetation index (NVI), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data, the surface temperature data, NVI, precipitation data, DEM data, soil texture data, soil moisture reanalysis data, and ground validation data at each preset time point are determined as fine-tuning training sample data. The ESA CCI soil moisture data at the same preset time point are determined as the second label sample data corresponding to the fine-tuning training sample data. Based on multiple fine-tuning training sample data and the second label sample data corresponding to each fine-tuning training sample data, a fine-tuning training set is constructed for fine-tuning the soil moisture data missing data imputation model.
[0048] S22: Input the pre-training set into the initial soil moisture data missing imputation model for training. Adjust the weight parameters of the initial soil moisture data missing imputation model according to the preset loss function until the model converges, and obtain the intermediate soil moisture data missing imputation model after training is completed.
[0049] The preset loss function can be, for example, the mean squared error function, or it can be determined by summing the mean squared error function and the regularization term, but it is not limited to this. This invention does not impose specific limitations, and those skilled in the art can set it according to the actual situation.
[0050] Specifically, after obtaining a pre-training set constructed from multiple pre-training sample data and the second label sample data corresponding to each pre-training sample data, the pre-training set is input into the initial soil moisture data missing imputation model. The model is pre-trained on the initial soil moisture data missing imputation model based on a preset loss function. The weight parameters of the initial soil moisture data missing imputation model are adjusted until the model converges, and the intermediate soil moisture data missing imputation model after training is obtained.
[0051] It should be noted that before pre-training, the model training parameters are initialized, for example, by setting the initial learning rate to 0.001, the batch size to 64, and training with the Adam optimizer. However, this invention is not limited to these parameters and those skilled in the art can set them according to the actual situation.
[0052] S23: Input the fine-tuned training set into the intermediate soil moisture data missing imputation model for training. Adjust the weight parameters of the intermediate soil moisture data missing imputation model according to the preset loss function until the model converges and obtain the soil moisture data missing imputation model after training is completed.
[0053] Specifically, after training the initial soil moisture data missing imputation model using a pre-training set to obtain an intermediate soil moisture data missing imputation model, the fine-tuning training set is then input into the intermediate soil moisture data missing imputation model. Based on a preset loss function, the intermediate soil moisture data missing imputation model is fine-tuned and trained, adjusting the weight parameters of the intermediate soil moisture data missing imputation model until the model converges, thus obtaining the soil moisture data missing imputation model after training.
[0054] Optionally, based on the above embodiments, refer to Figure 2 As shown, the soil moisture data missing completion model includes: a temporal convolutional network module 10, a bidirectional long short-term memory network module 11, a self-attention mechanism module 12, and an output module 13. The temporal convolutional network module includes three stacked temporal convolutional layers, and the bidirectional long short-term memory network module includes a first long short-term memory network unit (BiLSTM) and a second long short-term memory network unit (BiLSTM). Based on this, in some embodiments of the present invention, one implementation of S23 can be: The fine-tuned training set is input into the intermediate soil moisture data missing imputation model. The weight parameters of the third temporal convolution stacked layer, the second long short-term memory network unit, and the self-attention mechanism module are adjusted according to the preset loss function until the model converges, and the soil moisture data missing imputation model after training is obtained.
[0055] Specifically, after training the initial soil moisture data missing imputation model using a pre-training set to obtain an intermediate soil moisture data missing imputation model, the fine-tuning training set is further input into the intermediate soil moisture data missing imputation model. The weight parameters of the first two temporal convolutional stacked layers, the first long short-term memory network unit, and the output module are frozen. The weight parameters of the third temporal convolutional stacked layer, the second long short-term memory network unit, and the self-attention mechanism module are fine-tuned according to the preset loss function until the model converges, and the soil moisture data missing imputation model after training is obtained.
[0056] Optionally, based on the above embodiments, in some embodiments of the present invention, another implementation of S23 may be: The fine-tuned training set is input into the intermediate soil moisture data missing imputation model. According to the preset loss function, the weight parameters of the output module are adjusted in the first preset number of iterations. In the second preset number of iterations, the second long short-term memory network unit is frozen and other weight parameters of the model are adjusted. In the third preset number of iterations, the weight parameters of the entire model are adjusted until the model converges, and the soil moisture data missing imputation model after training is obtained.
[0057] In this way, this embodiment can first learn and adjust the weight parameters of the model using the pre-training set. Then, based on the weight parameters obtained through pre-training, the model's weight parameters can be further fine-tuned using the fine-tuning training set. This makes the feature information learned during training through the fine-tuning training set closer to the feature information pre-trained based on the pre-training set. This effectively solves the problem of insufficient training sample data in data-scarce areas, thereby improving the generalization ability and filling accuracy of the soil moisture data missing model.
[0058] Optionally, based on the above embodiments, in some embodiments of the present invention, reference continues to be made to... Figure 2 As shown, one possible implementation of S11 is: S110: Input the soil moisture data sequence into the temporal convolutional network module, and extract the initial time series features through the temporal convolutional network module.
[0059] The temporal convolutional network module comprises three stacked temporal convolutional layers, each consisting of a causal dilated convolution and a residual block. The kernel size for the causal dilated convolution is uniformly set to 3, with dilation rates of 1, 2, and 4 respectively. Using causal dilated convolution for feature learning ensures that feature learning is time-based, avoiding feature information leakage. Furthermore, by increasing the receptive field of the convolution kernel, causal dilated convolution effectively captures long-term temporal dependencies, enabling more efficient acquisition of temporal feature information.
[0060] Specifically, the soil moisture data sequence is input into the temporal convolutional network module, which extracts features from the soil moisture data sequence to obtain initial time series features.
[0061] S111: Input the initial time series features into the bidirectional long short-term memory network module, and extract the target time series features through the bidirectional long short-term memory network module.
[0062] Optionally, based on the above embodiments, the bidirectional long short-term memory network module includes: a first long short-term memory network unit and a second long short-term memory network unit. Therefore, in some embodiments of the present invention, one implementation of S111 can be: S1110: Input the initial time series features into the first long short-term memory network unit according to the forward and reverse time directions to obtain the forward and reverse time series features.
[0063] In this context, forward time refers to the normal time sequence, while reverse time refers to the reverse time sequence.
[0064] S1111: The forward time series features and the reverse time series features are concatenated and input into the second long short-term memory network unit to obtain the target time series features.
[0065] Specifically, the initial time series features are input into the first long short-term memory network unit according to forward and reverse time. The first long short-term memory network unit obtains the forward time series features corresponding to the forward time input and the reverse time series features corresponding to the reverse time input. After obtaining the forward and reverse time series features, the forward and reverse time series features are concatenated. The concatenated features are then input into the second long short-term memory network unit. The target time series features are obtained through the second long short-term memory network unit.
[0066] S112: Input the target time series features into the self-attention mechanism module, and extract attention features through the self-attention mechanism module.
[0067] The self-attention mechanism module is used to calculate the correlation of feature information over time, and learn the more important features based on the correlation, thereby improving the model's sensitivity to key features and making more accurate predictions.
[0068] S113: Input the attention features into the output module and obtain the data filling results through the output module.
[0069] Specifically, the soil moisture data sequence is input into a temporal convolutional network module, which performs feature extraction to obtain initial time-series features. These initial time-series features are then input into a bidirectional long short-term memory (LSTM) network module for feature extraction to obtain target time-series features. These target time-series features are then input into a self-attention mechanism module for feature extraction to obtain attention features. Finally, the attention features are input into the output module, which then provides the data imputation results.
[0070] This embodiment provides a method for imputing missing soil moisture data. It acquires a soil moisture data sequence corresponding to the ESA CCI soil moisture data to be imputed. This sequence comprises multi-dimensional feature data from multiple sources of soil moisture-related data, including: surface temperature data, normalized difference vegetation index (NDVI), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data. The soil moisture data sequence is input into a trained soil moisture data imputation model, which outputs the imputed data corresponding to the ESA CCI soil moisture data. Since the soil moisture data imputation model is trained using transfer learning on a pre-training set and a fine-tuned training set, it effectively solves the problem of insufficient training sample data in data-scarce regions, improving the model's generalization ability and imputation accuracy. The soil moisture data imputation model includes a temporal convolutional network module, a bidirectional long short-term memory (LSTM) network module, a self-attention mechanism module, and an output module. This model fully integrates the parallel computing capabilities of the temporal convolutional network module, the advantages of the bidirectional long short-term memory network module in temporal modeling, and the feature selection capabilities of the self-attention mechanism module, thereby achieving effective extraction of spatiotemporal features of soil moisture data and improving the prediction accuracy of the data filling results.
[0071] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0072] In one embodiment, such as Figure 3 As shown, a soil moisture data missing filling device is provided, including: a soil moisture data sequence acquisition module 10 and a filling data result acquisition module 11.
[0073] The soil moisture data sequence acquisition module is used to acquire the soil moisture data sequence corresponding to the soil moisture data to be filled in the ESA CCI. The soil moisture data sequence consists of multi-dimensional feature data corresponding to multi-source soil moisture related data, including: surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground validation data.
[0074] The data imputation result acquisition module is used to input the soil moisture data sequence into the trained soil moisture data missing imputation model, and output the imputation data result corresponding to the soil moisture data to be imputed by the soil moisture data missing imputation model. The soil moisture data missing imputation model is trained by transfer learning based on the pre-training set and the fine-tuning training set. The soil moisture data missing imputation model includes: a temporal convolutional network module, a bidirectional long short-term memory network module, a self-attention mechanism module, and an output module.
[0075] In the above embodiments, the soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled is obtained through the soil moisture data sequence acquisition module. This soil moisture data sequence consists of multi-dimensional feature data corresponding to multi-source soil moisture-related data, including: surface temperature data, normalized difference vegetation index (NDVI), precipitation data, digital elevation model (DEM) data, soil texture data, soil moisture reanalysis data, and ground validation data. The soil moisture data sequence is input into a trained soil moisture data missing completion model through the data completion result acquisition module. The model outputs the completed data result corresponding to the ESA CCI soil moisture data to be filled. Since the soil moisture data missing completion model is trained using transfer learning on a pre-training set and a fine-tuned training set, it effectively solves the problem of insufficient training sample data in data-scarce regions, improving the model's generalization ability and completion accuracy. Furthermore, the soil moisture data missing completion model includes: a temporal convolutional network module, a bidirectional long short-term memory (LSTM) network module, a self-attention mechanism module, and an output module. This model fully integrates the parallel computing capabilities of the temporal convolutional network module, the advantages of the bidirectional long short-term memory network module in temporal modeling, and the feature selection capabilities of the self-attention mechanism module, thereby achieving effective extraction of spatiotemporal features of soil moisture data and improving the prediction accuracy of the data filling results.
[0076] Specific limitations regarding the soil moisture data missing completion device can be found in the limitations of the soil moisture data missing completion method described above, and will not be repeated here. Each module in the aforementioned server can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0077] This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement a soil moisture data missing filling method provided in this invention. For example, when the processor executes the computer program, it can implement... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.
[0078] This invention provides a computer-readable storage medium storing at least one program, which is executed by a processor to implement a soil moisture data missing filling method provided in this invention. The implementation principle and technical effect are similar and will not be described again here.
[0079] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static random access memory (SRAM) and dynamic random access memory (DRAM), etc.
[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0081] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A soil moisture data missing imputation method, characterized in that, The method comprises the following steps: obtaining a soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled, wherein the soil moisture data sequence is composed of multi-dimensional feature data corresponding to multi-source soil moisture related data, and the multi-source soil moisture related data includes surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data, and ground verification data; inputting the soil moisture data sequence into a trained soil moisture data missing filling model, and outputting a filling data result corresponding to the ESA CCI soil moisture data to be filled through the soil moisture data missing filling model, wherein the soil moisture data missing filling model is obtained by transfer learning training according to a pre-training set and a fine-tuning training set, and the soil moisture data missing filling model includes a time convolution network module, a bidirectional long short-term memory network module, a self-attention mechanism module, and an output module.
2. The method of claim 1, wherein, The soil moisture data missing filling model is obtained by transfer learning training according to a pre-training set and a fine-tuning training set, and comprises: constructing the pre-training set according to the surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, the soil moisture reanalysis data, and the ground verification data; constructing the fine-tuning training set according to the ESA CCI soil moisture data, the surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, the soil moisture reanalysis data, and the ground verification data; inputting the pre-training set into an initial soil moisture data missing filling model for training, adjusting the weight parameters of the initial soil moisture data missing filling model according to a preset loss function, until the model converges, and obtaining an intermediate soil moisture data missing filling model at the end of training; inputting the fine-tuning training set into the intermediate soil moisture data missing filling model for training, adjusting the weight parameters of the intermediate soil moisture data missing filling model according to a preset loss function, until the model converges, and obtaining a soil moisture data missing filling model at the end of training.
3. The method of claim 2, wherein, The pre-training set comprises a plurality of pre-training sample data and first label sample data corresponding to each pre-training sample data, and the pre-training set is constructed according to the surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, the soil moisture reanalysis data, and the ground verification data, which comprises: determining the surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, and the ground verification data at each preset time as pre-training sample data, and determining the soil moisture reanalysis data at the same preset time as the first label sample data corresponding to the pre-training sample data; constructing the pre-training set according to a plurality of pre-training sample data and first label sample data corresponding to each pre-training sample data; The fine-tuning training set comprises: a plurality of fine-tuning training sample data, and second label sample data corresponding to each fine-tuning training sample data, and the fine-tuning training set is constructed according to ESA CCI soil moisture data, the land surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, the soil moisture reanalysis data and the ground verification data, comprising: determining the land surface temperature data, the normalized vegetation index, the precipitation data, the digital elevation model data, the soil texture data, the soil moisture reanalysis data and the ground verification data at each preset time as fine-tuning training sample data, and determining the ESA CCI soil moisture data at the same preset time as second label sample data corresponding to the fine-tuning training sample data; constructing the fine-tuning training set according to a plurality of fine-tuning training sample data and second label sample data corresponding to each fine-tuning training sample data.
4. The method of claim 3, wherein, The soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled in comprises: obtaining multi-source soil moisture related data corresponding to the ESA CCI soil moisture data to be filled in; performing coordinate alignment processing on the multi-source soil moisture related data; performing multi-dimensional feature extraction on the processed multi-source soil moisture related data to obtain the soil moisture data sequence.
5. The method of claim 4, wherein, The soil moisture data sequence is input into the trained soil moisture data missing filling model, and the filling data result corresponding to the ESA CCI soil moisture data to be filled in is output by the soil moisture data missing filling model, comprising: inputting the soil moisture data sequence into a time convolution network module to extract initial time sequence features by the time convolution network module; inputting the initial time sequence features into the bidirectional long short-term memory network module to extract target time sequence features by the bidirectional long short-term memory network module; inputting the target time sequence features into the self-attention mechanism module to extract attention features by the self-attention mechanism module; inputting the attention features into an output module to obtain the filling data result by the output module.
6. The method of claim 5, wherein, The bidirectional long short-term memory network module comprises: a first long short-term memory network unit and a second long short-term memory network unit, and the initial time sequence features are input into the bidirectional long short-term memory network module to extract target time sequence features by the bidirectional long short-term memory network module, comprising: inputting the initial time sequence features into the first long short-term memory network unit according to forward time and reverse time to obtain forward time sequence features and reverse time sequence features; performing splicing processing on the forward time sequence features and the reverse time sequence features, and inputting the spliced features into the second long short-term memory network unit to obtain target time sequence features.
7. The method of claim 6, wherein, The time convolution network module comprises: a three-layer time convolution stacking layer; the fine-tuning training set is input into the intermediate soil moisture data missing filling model for training; the weight parameters of the intermediate soil moisture data missing filling model are adjusted according to a preset loss function until the model converges, and a trained soil moisture data missing filling model is obtained, comprising: The fine-tuning training set is input into the intermediate soil moisture data missing filling model, the weight parameters of the third-layer time convolution stacking layer, the second long short-term memory network unit and the self-attention mechanism module are adjusted according to a preset loss function until the model converges, and a trained soil moisture data missing filling model is obtained.
8. A soil moisture data missing data imputation apparatus, characterized by, Comprising: The soil moisture data sequence acquisition module is configured to acquire a soil moisture data sequence corresponding to the ESA CCI soil moisture data to be filled, wherein the soil moisture data sequence is composed of multi-dimensional feature data corresponding to multi-source soil moisture related data, and the multi-source soil moisture related data includes surface temperature data, normalized vegetation index, precipitation data, digital elevation model data, soil texture data, soil moisture reanalysis data and ground verification data. The filling data result acquisition module is configured to input the soil moisture data sequence into the trained soil moisture data missing filling model, and output a filling data result corresponding to the ESA CCI soil moisture data to be filled through the soil moisture data missing filling model, wherein the soil moisture data missing filling model is obtained by transfer learning training according to a pre-training set and a fine-tuning training set, and the soil moisture data missing filling model comprises a time convolution network module, a bidirectional long short-term memory network module, a self-attention mechanism module and an output module. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to realize the steps of the soil moisture data missing filling method in any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the soil moisture data missing filling method in any one of claims 1 to 7.