Soil moisture reconstruction method and device mixing partial convolution and attention mechanism

Abstract

The application provides a soil moisture reconstruction method and device combining a partial convolution and an attention mechanism, innovatively combines a Transformer encoder and a partial convolution neural network architecture, introduces context attention in front of the Transformer encoder to better utilize adjacent spatial information, and continues to make many optimization settings in details, so that the reconstruction of satellite soil moisture observation data can be stably and high-precisely realized, high-quality satellite soil moisture observation requirements are met, and the application prospect is better.

Description

Technical Field

[0001] This application relates to the field of soil moisture detection, specifically to a method and apparatus for soil moisture reconstruction based on mixing and attention mechanisms. Background Technology

[0002] Soil moisture, as an important parameter (indicator) describing the Earth's environmental system, has wide applications in many fields, such as agricultural drought and flood prediction, wildfire prediction, and geological disaster prediction. In agriculture, soil moisture is a key element for crop growth.

[0003] The development of global Earth observation technology has driven the emergence of multi-source soil moisture products. Although the global satellite observation system is becoming increasingly sophisticated, existing soil moisture products generally suffer from spatiotemporal discontinuity due to the inherent revisit period of polar-orbiting satellites. This technical bottleneck is particularly prominent in high-latitude regions such as the Arctic, where the daily data loss rate can reach 20%-80% due to the geometric limitations of orbital inclination and sun-synchronous orbits.

[0004] To address this problem, researchers have been continuously exploring solutions, and spatiotemporal data reconstruction has made groundbreaking progress in recent years. Currently, among numerous deep learning spatiotemporal reconstruction schemes, spatial modeling techniques represented by Convolutional Neural Networks (CNNs) and time-series prediction frameworks built using Long Short-Term Memory (LSTM) networks are becoming the most popular techniques in the field of data imputation. CNNs can accurately capture and learn the spatial characteristics of soil moisture, while LSTM networks excel at using past time information to construct long-term sequence dependencies. Therefore, many scholars have proposed solutions based on these two algorithms.

[0005] Although CNNs and LSTMs have shown remarkable advantages in capturing the spatial heterogeneity and temporal homogeneity of soil moisture, they still face several challenges: First, the traditional fully convolutional deep network structure results in high computational complexity; second, the spatial mixing of valid and missing values ​​in the input data of fully convolutional models can cause feature propagation bias; and third, LSTM-based solutions still struggle to handle ultra-long-term sequence information and have limited ability to model global dependencies.

[0006] In other words, the inventors of this application have discovered that existing satellite soil moisture observation data reconstruction technologies still have limitations in terms of stable reconstruction accuracy, making it difficult to meet the needs of high-quality satellite soil moisture observation. Summary of the Invention

[0007] This application provides a method and apparatus for soil moisture reconstruction using a hybrid attention mechanism. It innovatively combines a Transformer encoder with a partial convolutional neural network architecture, and introduces contextual attention before the Transformer encoder to better utilize adjacent spatial information. Furthermore, it makes many optimizations in detail, thereby enabling stable and high-precision reconstruction of satellite soil moisture observation data, meeting the requirements for high-quality satellite soil moisture observation, and showing good application prospects.

[0008] Firstly, this application provides a soil moisture reconstruction method that combines partial convolution and attention mechanisms, the method comprising: Acquire the first sample of satellite soil moisture observation data, wherein the first sample of satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products; For the first sample satellite soil moisture observation data, a corresponding mask is generated by marking invalid pixels. Then, some data is randomly removed from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set. Based on the first sample satellite soil moisture observation data and sample set, a soil moisture reconstruction model is trained. The soil moisture reconstruction model is used to reconstruct the satellite soil moisture observation data input to the model in a spatiotemporal manner. The model architecture of the soil moisture reconstruction model includes, in order of input and output, a convolutional neural network structure on the input side, a convolutional neural network structure in the middle part, a context attention module, a Transformer encoder, and a convolutional neural network structure on the output side. The context attention module takes the query vector Q, the key vector K, and the value vector V as input under the standard attention quantization method, and enhances the model's repair ability by explicitly modeling the relationship between the missing region and the known region.

[0009] Secondly, this application provides a soil moisture reconstruction device based on a hybrid and attention mechanism, the device comprising: The acquisition unit is used to acquire the first sample satellite soil moisture observation data, wherein the first sample satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products; The processing unit is used to generate a corresponding mask from the first sample satellite soil moisture observation data by marking invalid pixels, and to randomly remove some data from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set. The training unit is used to train a soil moisture reconstruction model based on the first sample satellite soil moisture observation data and sample set. The soil moisture reconstruction model is used to reconstruct the satellite soil moisture observation data input to the model in a spatiotemporal manner. The model architecture of the soil moisture reconstruction model includes, in order of input and output directions, a convolutional neural network structure on the input side, a convolutional neural network structure in the middle part, a context attention module, a Transformer encoder, and a convolutional neural network structure on the output side. The context attention module takes the query vector Q, the key vector K, and the value vector V as input under the standard attention quantization method, and enhances the model's repair ability by explicitly modeling the relationship between the missing region and the known region.

[0010] From the above, it can be concluded that this application has the following beneficial effects: To address the goal of reconstructing satellite soil moisture observation data, this application innovatively combines a Transformer encoder with a partial convolutional neural network architecture. Contextual attention is introduced before the Transformer encoder to better utilize adjacent spatial information, and numerous optimizations are made in the details. This enables stable and high-precision reconstruction of satellite soil moisture observation data, meeting the requirements for high-quality satellite soil moisture observation and demonstrating promising application prospects. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a soil moisture reconstruction method that combines partial convolution and attention mechanisms as described in this application. Figure 2 This is a schematic diagram of a network architecture for the soil moisture reconstruction model of this application; Figure 3 This is a schematic diagram of another network architecture for the soil moisture reconstruction model in this application; Figure 4 This is a schematic diagram of a soil moisture reconstruction device that combines partial convolution and attention mechanisms according to this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps appearing in this application does not imply that the steps in the method flow must be performed in the chronological / logical order indicated by the naming or numbering. The execution order of named or numbered process steps can be changed according to the desired technical purpose, as long as the same or similar technical effect is achieved.

[0015] The module division described in this application is a logical division. In practical applications, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between modules shown or discussed may be through some interfaces, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules. Some or all of the modules may be selected to achieve the purpose of the solution in this application according to actual needs.

[0016] First, refer to Figure 1 , Figure 1 This paper illustrates a flowchart of a soil moisture reconstruction method that combines partial convolution and attention mechanisms, as described in this application. The soil moisture reconstruction method that combines partial convolution and attention mechanisms provided in this application may specifically include the following steps S101 to S103: Step S101: Obtain the first sample satellite soil moisture observation data, wherein the first sample satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products; It is understandable that, in order to meet the training requirements of the soil moisture reconstruction model, corresponding training samples need to be configured.

[0017] In this regard, the present application can obtain the initial sample satellite soil moisture observation data. For ease of distinction, the sample satellite soil moisture observation data obtained here is referred to as the first sample satellite soil moisture observation data. The same applies to the subsequent second sample satellite soil moisture observation data and target satellite soil moisture observation data.

[0018] It should be noted that this application specifically targets the type of soil moisture observation data obtained from satellite observation products. However, due to the fact that the global satellite observation system is becoming increasingly sophisticated, satellite soil moisture products generally suffer from spatiotemporal discontinuity due to the inherent revisit cycle limitation of polar-orbiting satellites. Therefore, there is a need for high-quality satellite soil moisture reconstruction, which this application aims to address.

[0019] As an example, the first sample of satellite soil moisture observation data obtained here is a product of a specific satellite soil moisture observation data, which covers ascending orbit (daytime) data from 2010 to 2019, using the EASE-Grid2 projected coordinate system, with a resolution of 36km, and the data unit is meters. 3 / m 3 Due to the inherent limitations of polar-orbiting satellite revisit cycles and soil moisture retrieval algorithms, the daily soil moisture data included in this product cannot fully cover the entire globe.

[0020] Of course, it is understandable that, in practice, this application does not specifically limit the type of satellite soil moisture observation data product used to obtain the first sample satellite soil moisture observation data, and adjustments can be made according to the actual situation.

[0021] Step S102: For the first sample satellite soil moisture observation data, generate a corresponding mask by marking invalid pixels, and continue to randomly remove some data from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set; Understandably, the first sample of satellite soil moisture observation data obtained earlier is raw data. Corresponding to the soil moisture reconstruction work of the model to be trained, it needs to be further processed so that there are corresponding missing areas in the sample, allowing the model to continuously try to make predictions and achieve the training goal.

[0022] This application relates to masking and random erasure operations. The masking function constrains and limits the data processing range within the corresponding data processing task, while random erasure distinguishes the data from the first sample satellite soil moisture observation data. This allows the model to evaluate its prediction accuracy by calculating the differences, thereby continuously optimizing the model and improving its performance. The two operations are independent of each other. Of course, random erasure can also be performed within the data processing range (or image range) constrained by the mask.

[0023] Furthermore, it is understandable that in practice, training samples may also involve relevant preprocessing to improve data quality and thus improve the quality of subsequent model training. However, considering that preprocessing can directly adopt existing technologies and is not the focus of this application, it will not be elaborated on further.

[0024] In this way, the configuration of training samples is completed, and the resulting sample set can be used in the subsequent specific model training work.

[0025] Step S103: Based on the first sample satellite soil moisture observation data and sample set, a soil moisture reconstruction model is trained. The soil moisture reconstruction model is used to reconstruct the satellite soil moisture observation data input to the model in a spatiotemporal manner. The model architecture of the soil moisture reconstruction model includes, in order of input and output directions, an input-side convolutional neural network structure, a middle-side convolutional neural network structure, a context attention module, a Transformer encoder, and an output-side convolutional neural network structure. Under the standard attention quantization method, the context attention module takes the query vector Q, the key vector K, and the value vector V as inputs, and enhances the model's repair capability by explicitly modeling the relationship between the missing region and the known region.

[0026] Understandably, the soil moisture reconstruction model to be configured and trained in this application adopts a hybrid design specifically configured in this application in terms of its specific model structure. That is, it combines some convolution and attention mechanisms (or some convolutional neural networks and attention mechanisms) on the basis of the Transformer encoder (a deep learning model, which is generally referred to by its English name).

[0027] Specifically, the Transformer model has demonstrated outstanding performance in fields such as natural language processing and image inpainting in recent years. With its powerful global capture capability and ultra-long contextual information utilization capability, it has made up for the shortcomings of LSTM and has become a research hotspot in academia.

[0028] Attention mechanism, as the core of the Transformer model, can accurately model global contextual information by dynamically adjusting the importance weights of different positions in the input sequence. This mechanism significantly improves the model's performance in long sequence tasks, enabling the Transformer model to exhibit outstanding efficiency when processing large-scale data.

[0029] A standard attention mechanism takes three inputs: a query vector (Q), a key vector (K), and a value vector (V), and is defined as follows: , in, To quantify the results of attention, Let K be the dimension of the key vector, and its specific value is determined by the embedding dimension and the number of attention heads in the model. This is a common normalized exponential function used in neural networks.

[0030] However, it should be understood that while the Transformer model excels at utilizing long-term temporal information correlations, it is also weak in capturing adjacent spatial features. This problem is particularly evident in the soil moisture reconstruction objective that this application focuses on.

[0031] Therefore, this application designs a hybrid model architecture based on the Transformer model, which is highly sensitive to the correlation of long-term information and also highly sensitive to the characteristics of adjacent spaces, thus achieving high-quality soil moisture reconstruction results.

[0032] Regarding the specific model architecture, refer to Figure 2 The diagram shows a network architecture of the soil moisture reconstruction model of this application. In the model architecture of the soil moisture reconstruction model, the input of the input-side convolutional neural network structure (or input convolutional layer) is the model input. The output of the input-side convolutional neural network structure is connected to the input of the intermediate convolutional neural network structure. The output of the intermediate convolutional neural network structure is connected to the input of the context attention module. The output of the context attention module is connected to the input of the Transformer encoder. The output of the Transformer encoder is connected to the input of the output-side convolutional neural network structure (or output convolutional layer). The output of the output-side convolutional neural network structure is the model output.

[0033] In more concise terms, regarding data flow, the soil moisture reconstruction model architecture, in order of input and output direction, includes an input-side convolutional neural network structure, a middle-side convolutional neural network structure, a context attention module, a Transformer encoder, and an output-side convolutional neural network structure.

[0034] The input-side convolutional neural network structure serves as the input layer, used to map single channels to multiple channels. The corresponding input-side convolutional neural network structure, which serves as the output layer, is used to map multiple channels back to single channels for output. The intermediate convolutional neural network structure following the input-side convolutional neural network structure is responsible for feature extraction, and multiple layers can be configured for processing depending on the desired effect.

[0035] Continuing to focus on the context attention module, as an exemplary embodiment, the main working process of the context attention module can be specifically configured as follows: the input feature map (the output form of the input part of the convolutional neural network structure) is divided into blocks, and attention scores are calculated for weighted aggregate value features. The mask is used to control the reconstruction region, prompting the model to apply the attention results only to the missing regions.

[0036] The standard attention quantification method involved is similar to the traditional attention mechanism, and can be specifically expressed by the following formula: , in, The result of attention quantification, also known as a weighted attention score, Let K be the dimension of the key vector. The specific value is determined by the model embedding dimension and the number of attention heads. This is a common normalized exponential function used in neural networks.

[0037] In addition, this application also takes into account the computing power limitations that may exist in actual situations. Downsampling operations can be performed on the input side of the context attention module, and upsampling operations can be performed on the output side of the context attention module.

[0038] In summary, the combination of downsampling and segmentation strategies with the local feature extraction capabilities of partial convolution allows the model to efficiently borrow spatial patterns from known regions of soil moisture images, generating smooth and semantically consistent restoration results.

[0039] Meanwhile, since scientific data is typically presented in a single-channel (1-dimensional) structure, this application can be specifically applied to the model output. Partial convolution is used to map the multidimensional features output by the Transformer encoder to a 1-dimensional feature space, providing a suitable structure for subsequent scientific data applications.

[0040] Furthermore, all convolutional layers in the soil moisture reconstruction model can be configured as follows: The kernel size is adjusted to ensure that local correlations of the data are preserved to the maximum extent while extracting features.

[0041] Meanwhile, for convolutional layers, it's understandable that batch normalization (BN) layers and rectified linear unit (ReLU) activation function layers can typically be configured sequentially after the convolutional layers. BN is a normalization technique used in neural networks to address internal covariate shift by standardizing input features, accelerating training and improving generalization ability. The main function of the ReLU function is to set the negative parts of the input signal to 0, while keeping the positive parts unchanged. This approach makes the ReLU function computationally simple and efficient during forward propagation, and also alleviates the gradient vanishing problem to some extent. In this way, the nonlinear expressive power and feature extraction capabilities of the neural network are enhanced, enabling the model to more accurately capture and represent complex soil moisture variation characteristics.

[0042] As for the Transformer encoder, it can be understood that it specifically includes multiple Transformer encoder layers (the number of layers can be denoted as M). Each Transformer encoder layer contains two core modules: a multi-head self-attention mechanism module and a feedforward neural network module. To alleviate the gradient vanishing problem during deep network training and improve the model's generalization ability, both the multi-head self-attention mechanism module and the feedforward neural network module introduce residual connections and layer normalization operations. Before the input data enters the encoder, it is first mapped to a high-dimensional feature space through input embedding and combined with position encoding to retain temporal or spatial order information. The feature sequences after embedding and encoding are sequentially fed into each encoder layer for feature extraction and information fusion.

[0043] Meanwhile, reference Figure 3The diagram shown illustrates another network architecture of the soil moisture reconstruction model of this application. It can be clearly seen that the input-side convolutional neural network structure mentioned above can specifically include two parts: a first input-side convolutional neural network structure and a second input-side convolutional neural network structure. The output features of the first and second input-side convolutional neural network structures are concatenated and then input into the intermediate convolutional neural network structure. The features processed by the intermediate convolutional neural network structure are input into the context attention module. The output features of the input-side convolutional neural network structure are also input together with the output features of the Transformer encoder into the output-side convolutional neural network structure (residual connection) to achieve shallow and deep feature fusion, enhancing the model's feature representation capability. The output-side convolutional neural network structure outputs the model processing results.

[0044] The specific inputs to the first input-side convolutional neural network structure are the second sample satellite soil moisture observation data on day T and the corresponding mask data of the second sample satellite soil moisture observation data on day T. The specific inputs to the second input side part of the convolutional neural network structure are the second sample satellite soil moisture observation data for 9 consecutive days centered on day T, and the mask data corresponding to the second sample satellite soil moisture observation data for 9 consecutive days centered on day T.

[0045] It is understandable that the output of the input-side convolutional neural network structure (the first input-side convolutional neural network structure and the second input-side convolutional neural network structure) involves not only the intermediate convolutional neural network structure but also the output-side convolutional neural network structure. This is to enhance the model's ability to integrate features at different levels. Thus, for the overall output stage of the encoder, a residual connection mechanism is introduced, similar to the Transformer encoder layer mentioned earlier, to enable effective interaction between shallow and deep features, thereby optimizing the completeness of feature representation and achieving better soil moisture reconstruction results.

[0046] Furthermore, it can be seen that this application designs a dual-channel input architecture and establishes a spatiotemporal feature collaborative extraction mechanism. The first input-side convolutional neural network structure (which can also be referred to as input part convolutional layer 1) only receives the daily soil moisture and its mask matrix on day T, ensuring accurate capture of the soil moisture information features of the current date. The second input-side convolutional neural network structure (which can also be referred to as input part convolutional layer 2) receives the long-term soil moisture and its mask matrix for 9 consecutive days, so as to realize the analysis and processing of long-term soil moisture information. In this way, the original data is mapped to a higher-dimensional feature space through partial convolution operation, and two sets of data are output. These two sets of data are effectively integrated after feature concatenation and passed to the middle part convolutional neural network structure.

[0047] In addition, the training process may involve many other configuration tasks.

[0048] For example, in practical applications, it is inevitable that some of the input data will exceed the normal range (0-0.6m). 3 / m 3 Outliers (such as those in the model) need to be excluded during training to avoid interfering with model parameters and causing output errors, ensuring the reliability of the model output. To determine the optimal learning rate strategy, this application can also use the Adam optimizer to balance the need for rapid convergence in the early stages of training with the need for dynamic adjustment of the learning rate during training, preventing training stagnation due to a sharp drop in the learning rate. Specifically, considering the characteristics of this model, the default learning rate of the Adam optimizer can be set to 0.0002, and the learning rate will be dynamically adjusted according to a preset decay strategy as the number of iterations increases.

[0049] As for the loss function during the training process, the specific loss function is calculated by combining the soil moisture reconstruction results output by the model for the sample set and the original first sample satellite soil moisture observation data (that is, the original first sample satellite soil moisture observation data is used as a ground truth or a label for the sample set).

[0050] Meanwhile, compared to existing loss functions that can be directly adopted, this application believes that the loss function, as the core of model optimization, is directly related to prediction accuracy and spatial smoothness. Therefore, this application also provides a matching loss function scheme that is adapted to the application scenario of this application and has better performance.

[0051] Specifically, the loss function used in this application during training can be: , , , , in, For the total loss, For effective area loss, For the loss of voids, To smooth out the loss, , and For different balance factors, N is the total number of pixels in the image, and i is the pixel index. Let be the hole mask tensor of the i-th pixel, where mask=1 for the valid region and mask=0 for the invalid region (corresponding to the hole). Let be the predicted soil moisture data tensor (or predicted soil moisture value) for the i-th pixel. Let be the tensor of the actual soil moisture data of the i-th pixel (or the original value of soil moisture). The values ​​of i, j, and k are the predicted values ​​obtained after each training round of the original soil moisture tensor. The corresponding index.

[0052] For one of them This is to reduce the spatial boundary effect of reconstructed soil moisture, aiming to maximize the smoothness of soil moisture in adjacent spaces, through calculation. The smoothness of an image is measured by the sum of the differences between adjacent time steps.

[0053] In terms of model validation, practical applications have shown that the soil moisture reconstruction model configured in this application has been evaluated for accuracy and reliability through three methods: site validation, time series validation, and simulation of missing regions. The correlation coefficient can reach up to 0.952. The validation results reflect that the generated long-term daily-scale global seamless soil moisture product has high accuracy and reliability, and has good application value and application prospects.

[0054] Among them, the soil moisture observation data used for verification can also be ground soil moisture observation data. Ground soil moisture observation data is easy to understand as it is measured by ground observation points. In this regard, this application can also involve obtaining sample ground soil moisture observation data, which can serve to verify the accuracy of the reconstructed satellite soil moisture observation data after the model has been trained.

[0055] As an example, this application directly uses readily available ground soil moisture observation data products, which involve 199 in-situ soil moisture stations worldwide. The time span needs to be consistent with the first sample satellite soil moisture observation data, for example, both covering 2010-2019.

[0056] After the model has been trained, it is easy to understand that it can be put into practical applications, that is, to carry out the reconstruction of specific satellite soil moisture observation data.

[0057] In this regard, after training the soil moisture reconstruction model based on the first sample satellite soil moisture observation data and sample set, the method of this application may further include: Acquire soil moisture observation data from the target satellite; Input the target satellite soil moisture observation data into the soil moisture reconstruction model; The reconstruction results of satellite soil moisture observation data are extracted from the soil moisture reconstruction model.

[0058] It is understandable that the target satellite soil moisture observation data here can be soil moisture observation data obtained from the same satellite observation product as the first sample satellite soil moisture observation data mentioned above, or it can be soil moisture observation data obtained from other satellite observation products.

[0059] Thus, after obtaining the target satellite soil moisture observation data that needs to be reconstructed to improve spatiotemporal continuity, the soil moisture reconstruction model configured in this application can be used to perform reconstruction processing on it. After the soil moisture reconstruction model completes the processing, it can output the corresponding soil moisture reconstruction results. From this, the reconstruction results of the satellite soil moisture observation data can be extracted.

[0060] At this point, the reconstruction results of the satellite soil moisture observation data can be stored locally, stored off-site, forwarded, output as a prompt or display of the completed reconstruction process, or further analyzed. Understandably, the specific data application content that may be involved later can be adjusted according to the real-time configuration or pre-configured data application strategies / rules, and this application does not impose any specific limitations.

[0061] In conclusion, regarding the above solutions, this application innovatively combines a Transformer encoder with a partial convolutional neural network architecture to achieve the goal of reconstructing satellite soil moisture observation data. Contextual attention is introduced before the Transformer encoder to better utilize adjacent spatial information, and numerous optimizations have been made in the details. As a result, the reconstruction of satellite soil moisture observation data can be achieved stably and with high accuracy, meeting the requirements for high-quality satellite soil moisture observation and showing promising application prospects.

[0062] The above is an introduction to the soil moisture reconstruction method with hybrid partial convolution and attention mechanisms provided in this application. In order to facilitate better implementation of the soil moisture reconstruction method with hybrid partial convolution and attention mechanisms provided in this application, this application also provides a soil moisture reconstruction device with hybrid partial convolution and attention mechanisms from the perspective of functional modules.

[0063] See Figure 4 , Figure 4 This is a schematic diagram of a soil moisture reconstruction device that combines partial convolution and attention mechanisms according to this application. In this application, the soil moisture reconstruction device 400 that combines partial convolution and attention mechanisms may specifically include the following structure: The acquisition unit 401 is used to acquire the first sample satellite soil moisture observation data, wherein the first sample satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products. The processing unit 402 is used to generate a corresponding mask from the first sample satellite soil moisture observation data by marking invalid pixels, and to continue to randomly erase part of the data from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set. Training unit 403 is used to train a soil moisture reconstruction model based on the first sample satellite soil moisture observation data and sample set. The soil moisture reconstruction model is used to reconstruct the satellite soil moisture observation data input to the model in a spatiotemporal manner. The model architecture of the soil moisture reconstruction model includes, in order of input and output directions, an input-side convolutional neural network structure, a middle-side convolutional neural network structure, a context attention module, a Transformer encoder, and an output-side convolutional neural network structure. The context attention module, under the standard attention quantization method, takes the query vector Q, the key vector K, and the value vector V as inputs, and enhances the model's repair capability by explicitly modeling the relationship between the missing region and the known region.

[0064] In one exemplary embodiment, during the main operation of the context attention module, the input feature map is divided into blocks, and attention scores are calculated for weighted aggregated features. A mask is used to control the reconstructed region, causing the model to apply the attention results only to the missing regions.

[0065] In yet another exemplary embodiment, the standard attention quantization method is specifically represented by the following formula: , in, To quantify the results of attention, Let K be the dimension of the key vector. The specific value is determined by the model embedding dimension and the number of attention heads. It is a normalized exponential function.

[0066] In yet another exemplary embodiment, a downsampling operation is performed on the input side of the context attention module, and an upsampling operation is performed on the output side of the context attention module.

[0067] In yet another exemplary embodiment, all convolutional layers included in the soil moisture reconstruction model are configured as follows: The size of the convolution kernel.

[0068] In another exemplary embodiment, the Transformer encoder includes multiple Transformer encoder layers. Each Transformer encoder layer contains a multi-head self-attention mechanism module and a feedforward neural network module. Both the multi-head self-attention mechanism module and the feedforward neural network module introduce residual connections and layer normalization operations. Before the input data enters the encoder, it is first mapped to a high-dimensional feature space through input embedding and combined with position encoding to retain temporal or spatial order information. The feature sequence after embedding and encoding is sequentially sent to each encoder layer for feature extraction and information fusion.

[0069] In another exemplary embodiment, the input-side convolutional neural network structure includes a first input-side convolutional neural network structure and a second input-side convolutional neural network structure. The output features of the first input-side convolutional neural network structure and the second input-side convolutional neural network structure are concatenated and then input into the intermediate convolutional neural network structure. The features processed by the intermediate convolutional neural network structure are input into the context attention module. The output features of the input-side convolutional neural network structure are also input into the output-side convolutional neural network structure together with the output features of the Transformer encoder to achieve shallow and deep feature fusion and enhance the model's feature representation capability. The output-side convolutional neural network structure outputs the model processing results. The specific inputs to the first input-side convolutional neural network structure are the second sample satellite soil moisture observation data on day T and the corresponding mask data of the second sample satellite soil moisture observation data on day T. The specific inputs to the second input side part of the convolutional neural network structure are the second sample satellite soil moisture observation data for 9 consecutive days centered on day T, and the mask data corresponding to the second sample satellite soil moisture observation data for 9 consecutive days centered on day T.

[0070] In yet another exemplary embodiment, the loss function used during training is specifically: , , , , in, For the total loss, For effective area loss, For the loss of voids, To smooth out the loss, , and For different balance factors, N is the total number of pixels in the image, and i is the pixel index. Let be the hole mask tensor for the i-th pixel, where mask=1 for valid regions and mask=0 for invalid regions. Let be the predicted soil moisture tensor for the i-th pixel. Let be the tensor of the actual soil moisture data for the i-th pixel. The values ​​of i, j, and k are the predicted values ​​obtained after each training round of the original soil moisture tensor. The corresponding index.

[0071] In yet another exemplary embodiment, the apparatus further includes an application unit 404 for: Acquire soil moisture observation data from the target satellite; Input the target satellite soil moisture observation data into the soil moisture reconstruction model; The reconstruction results of satellite soil moisture observation data are extracted from the soil moisture reconstruction model.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the soil moisture reconstruction device and its corresponding units based on the hybrid partial convolution and attention mechanisms described above can be found in, for example... Figure 1 The description of the soil moisture reconstruction method that combines partial convolution and attention mechanisms in the corresponding embodiments will not be repeated here.

[0073] The above provides a detailed description of the soil moisture reconstruction method and apparatus based on the hybrid partial convolution and attention mechanism provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for reconstructing soil moisture using a hybrid partial convolution and attention mechanism, characterized in that, The method includes: Acquire first sample satellite soil moisture observation data, wherein the first sample satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products; For the first sample satellite soil moisture observation data, a corresponding mask is generated by marking invalid pixels, and then some data is randomly erased from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set; Based on the first sample satellite soil moisture observation data and the sample set, a soil moisture reconstruction model is trained. The soil moisture reconstruction model is used to perform spatiotemporal reconstruction of the satellite soil moisture observation data input to the model. The model architecture of the soil moisture reconstruction model includes, in order of input and output directions, an input-side convolutional neural network structure, a middle convolutional neural network structure, a context attention module, a Transformer encoder, and an output-side convolutional neural network structure. The context attention module, under the standard attention quantization method, takes the query vector Q, the key vector K, and the value vector V as inputs, and enhances the model's repair capability by explicitly modeling the relationship between the missing region and the known region.

2. The method according to claim 1, characterized in that, In the main working process of the context attention module, the input feature map is divided into blocks, and attention scores are calculated for weighted aggregation value features. The mask is used to control the reconstruction region, prompting the model to apply the attention results only to the missing region.

3. The method according to claim 1, characterized in that, The standard attention quantification method is specifically expressed by the following formula: ， in, To quantify the results of attention, Let K be the dimension of the key vector. The specific value is determined by the model embedding dimension and the number of attention heads. It is a normalized exponential function.

4. The method according to claim 1, characterized in that, Downsampling is performed on the input side of the context attention module, and upsampling is performed on the output side of the context attention module.

5. The method according to claim 1, characterized in that, All convolutional layers in the soil moisture reconstruction model are configured as follows: The size of the convolution kernel.

6. The method according to claim 1, characterized in that, The Transformer encoder includes multiple Transformer encoder layers. Each Transformer encoder layer contains a multi-head self-attention mechanism module and a feedforward neural network module. Both the multi-head self-attention mechanism module and the feedforward neural network module introduce residual connections and layer normalization operations. Before the input data enters the encoder, it is first mapped to a high-dimensional feature space through input embedding and combined with position encoding to retain temporal or spatial order information. The feature sequences after embedding and encoding are sequentially fed into each encoder layer for feature extraction and information fusion.

7. The method according to claim 1, characterized in that, The input-side partial convolutional neural network structure includes a first input-side partial convolutional neural network structure and a second input-side partial convolutional neural network structure. The output features of the first input-side partial convolutional neural network structure and the second input-side partial convolutional neural network structure are concatenated and then input into the intermediate partial convolutional neural network structure. The features processed by the intermediate partial convolutional neural network structure are input into the context attention module. The output features of the input-side partial convolutional neural network structure are also input into the output-side partial convolutional neural network structure together with the output features of the Transformer encoder to achieve shallow and deep feature fusion and enhance the model's feature representation capability. The output-side partial convolutional neural network structure outputs the model processing results. The specific inputs to the first input-side convolutional neural network structure are the second sample satellite soil moisture observation data on day T and the corresponding mask data of the second sample satellite soil moisture observation data on day T. The specific inputs to the second input-side convolutional neural network structure are the second sample satellite soil moisture observation data for 9 consecutive days centered on day T, and the corresponding mask data of the second sample satellite soil moisture observation data for 9 consecutive days centered on day T.

8. The method according to claim 1, characterized in that, The loss function used during training is as follows: ， ， ， ， in, For the total loss, For effective area loss, For the loss of voids, To smooth out the loss, , and For different balance factors, N is the total number of pixels in the image, and i is the pixel index. Let be the hole mask tensor for the i-th pixel, where mask=1 for valid regions and mask=0 for invalid regions. Let be the predicted soil moisture data tensor for the i-th pixel. Let i be the tensor of the actual soil moisture data of the i-th pixel. The predicted values ​​are obtained from each round of training of the original soil moisture tensor. j and k are The corresponding index, The smoothness of an image is measured by calculating the sum of the differences between adjacent time steps.

9. The method according to claim 1, characterized in that, After training the soil moisture reconstruction model based on the first sample satellite soil moisture observation data and the sample set, the method further includes: Acquire soil moisture observation data from the target satellite; Input the target satellite soil moisture observation data into the soil moisture reconstruction model; Extract the reconstruction results from the satellite soil moisture observation data output by the soil moisture reconstruction model.

10. A soil moisture reconstruction device that combines partial convolution and attention mechanisms, characterized in that, The device includes: The acquisition unit is used to acquire first sample satellite soil moisture observation data, wherein the first sample satellite soil moisture observation data is soil moisture observation data obtained from satellite observation products; The processing unit is used to generate a corresponding mask from the first sample satellite soil moisture observation data by marking invalid pixels, and to continue to randomly erase part of the data from the first sample satellite soil moisture observation data and the mask data to obtain the corresponding second sample satellite soil moisture observation data and the mask data corresponding to the second sample satellite soil moisture observation data, forming a sample set. The training unit is used to train a soil moisture reconstruction model based on the first sample satellite soil moisture observation data and the sample set. The soil moisture reconstruction model is used to perform spatiotemporal reconstruction of the satellite soil moisture observation data input to the model. The model architecture of the soil moisture reconstruction model includes, in order of input and output directions, an input-side convolutional neural network structure, a middle convolutional neural network structure, a context attention module, a Transformer encoder, and an output-side convolutional neural network structure. The context attention module, under the standard attention quantization method, takes the query vector Q, the key vector K, and the value vector V as inputs, and enhances the model's repair capability by explicitly modeling the relationship between the missing region and the known region.

Images