DS-xNet-based multi-source remote sensing time series data cultivated land utilization current situation extraction method

Abstract

The invention belongs to the technical field of remote sensing image processing agricultural information, and particularly relates to a DS-xNet-based multi-source remote sensing time sequence data cultivated land utilization current situation extraction method. According to the method, the cultivated land and the water body can still be reliably recognized under the cloud or fog condition by combining Sentinel-1 and Sentinel-2, missing detection and misjudgment caused by cloud shielding are reduced, sNET captures short-term dynamic conditions, and mNET matrix memory enhances long-term dependence modeling, so that the recognition precision of seasonal and interannual changes is improved, supervision is applied to a branch and fusion layer, and the recognition accuracy of the cultivated land and the water body is improved. According to the method, each branch can learn independent discrimination capability and can cooperatively improve fusion output, over-fitting is reduced, generalization capability is improved, training convergence is accelerated, channel splicing retains discrimination information of each modal in a fusion stage, modal information conflict or loss caused by direct early fusion is avoided, classification accuracy is improved, and classification efficiency is improved. Various optical and radar indexes are combined and used to enhance the distinguishing capability of paddy fields, irrigation areas, different crops and non-cultivated areas.

Description

Technical Field

[0001] This invention belongs to the field of agricultural information technology of remote sensing image processing, and specifically relates to a method for extracting the current status of cultivated land use based on multi-source remote sensing time-series data using DS-xNet. Background Technology

[0002] Remote sensing technology, as an important means of agricultural resource monitoring, enables the acquisition of large-scale, periodic farmland use information, playing a crucial role in crop classification and planting structure identification. In particular, with the European Space Agency's open-source data policy for its Sentinel series satellites, Sentinel-1 synthetic aperture radar (SAR) data and Sentinel-2 multispectral optical imagery provide agricultural remote sensing with high spatiotemporal resolution and multimodal observation capabilities. SAR data has the advantage of being unaffected by cloud and rain conditions, sensitively reflecting surface structure and moisture information; optical data provides rich spectral features, effectively characterizing vegetation growth. Combining the two theoretically creates complementarity in temporal, spatial, and feature dimensions, improving the robustness and accuracy of crop classification.

[0003] Currently, deep learning-based temporal remote sensing image classification methods have become a research hotspot. Among them, recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and their variants (such as GRUs) are widely used for modeling crop phenological time-series characteristics. However, these traditional time-series models suffer from insufficient long-term dependency modeling capabilities and are prone to gradient vanishing or exploding when dealing with growth cycle sequences lasting from several months to a year, making it difficult to effectively capture the complete growth rhythm and key phenological characteristics of crops. Furthermore, some studies have attempted to fuse SAR and optical features, but these often employ simple early stitching or late-stage decision-level fusion strategies, failing to achieve deep synergy and interaction between the two heterogeneous data at the temporal feature level, and thus failing to fully explore their inherent complementarity.

[0004] The aforementioned problems are particularly pronounced in crop rotation areas with complex planting structures (such as wheat-rice and rapeseed-rice rotation areas). These areas feature diverse crop varieties, overlapping growing seasons, and dramatic changes in remote sensing reflectance characteristics over time, placing extremely high demands on the model's temporal modeling capabilities and feature discrimination. Existing methods generally suffer from poor feature separation, insensitivity to phenological changes, and insufficient class discrimination when handling these scenarios, resulting in classification accuracy that fails to meet practical application requirements and limiting their widespread adoption in precision agriculture monitoring. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting the current status of cultivated land use based on multi-source remote sensing time-series data using DS-xNet, which has several advantages.

[0006] The specific technical solution adopted by this invention is as follows:

[0007] A method for extracting farmland use status from multi-source remote sensing time-series data based on DS-xNet includes:

[0008] Multi-source remote sensing time-series data of the study area were acquired, including Sentinel-1 radar images and Sentinel-2 optical images. VV polarization, VH polarization, dual polarization water index SDWI and SAR vegetation index DPSVIm were extracted from Sentinel-1 radar images, and NDVI, EVI, GNDVI, NDWI, REI and SAVI optical vegetation indices were extracted from Sentinel-2 optical images.

[0009] Representative sample points were selected based on multi-source remote sensing data and field survey data. These representative sample points included crop types and non-arable land types. ArcGIS was used to extract the temporal feature vectors of the representative sample points in the multi-source remote sensing time-series data to construct a multi-source time-series feature sample dataset.

[0010] A dual-stream temporal modeling network DS-xNet is constructed, which includes a temporal feature coding branch of Sentinel-1 and a temporal feature coding branch of Sentinel-2. The dual-stream temporal modeling network DS-xNet adopts the xNET structure for modeling.

[0011] The high-dimensional feature vectors output by the dual-stream temporal modeling network DS-xNet are channel-separated and fused, and the classification results of the current status of cultivated land use are obtained through linear mapping and softmax operation.

[0012] During model training, a multi-supervised loss function is introduced into the output and fused output of the dual-stream temporal modeling network DS-xNet to obtain the trained DS-xNet model. The loss terms of the multi-supervised loss function include Loss1, Loss2 and Loss3, and all loss terms adopt the cross-entropy loss function.

[0013] The remote sensing image samples to be classified are input into the trained DS-xNet model to obtain the spatial location and classification results of the current farmland use type.

[0014] In a preferred embodiment, the Sentinel-1 radar imagery acquisition of the study area takes place at a time interval of 15 days, while the Sentinel-2 optical imagery acquisition of the study area takes place at a time interval of 30 days.

[0015] In a preferred embodiment, the training set and validation set are divided into multiple time-series feature sample data in a 7:3 ratio in the multi-source time-series feature sample dataset.

[0016] In a preferred embodiment, xNET consists of alternating stacked sNET and mNET units, where sNET is used for short-term dynamic feature extraction and mNET is used to enhance the modeling ability for long-term dependencies by introducing a matrix memory structure.

[0017] In a preferred embodiment, xNET consists of alternating stacked sNET and mNET units, and the calculation process includes the following formula: Cellular infusion: ; Input Gate: ; Forgotten Gate: ; Output gate: ; Stabilization factor: ; Stabilize the input gate: ; Stabilization forgetting gate: ; Cell status update: ; Normalized state update: ; Hidden state output: ; In the formula, This represents the input feature vector at time step t. This represents the hidden state of the previous time step. It is the cell input at the current time step, generated by the transformation between the current input and the previous hidden state. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This indicates the current state of the cell. This indicates the cell state at the previous moment. For normalized state factors, This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. Represents the hyperbolic tangent function. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , , These represent the input transformation weights that control the candidate state, input gate, forget gate, and output gate, respectively. , , , These represent the recurrent weights for the linear transformation of the four gates from their previous hidden state. These represent the bias terms for these four gates, respectively.

[0018] In a preferred embodiment, the mNET unit enhances its ability to model long-term dependencies by introducing a matrix memory structure. The calculation process includes the following formula: Query vector: , Key vector: , Value vector: , Input Gate: , Forgotten Gate: , Output gate: , Stabilization factor: , Normalized input gate: , Normalized Forgetting Gate: , Cell status update: , Normalized state update: , Hidden state output: , In the formula, , , These represent the query vector, key vector, and value vector generated in the feature space from the current input, respectively. for transpose, This serves as a matrix-style memory unit for the current moment, recording higher-order feature relationships across time steps. This indicates the cell state at the previous moment. This is a normalized state vector used to normalize the attention weights. The scaling factor ensures the stability of the dot product; Ensure the denominator is not less than 1 to avoid numerical instability; This represents the input feature vector at time step t. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , These represent the weight matrices input to the query, key, and value, respectively. The system learns how to generate attention vectors. , , These represent the weight matrices input to the gate, for transpose, for transpose, , , , , , These represent the corresponding bias vectors.

[0019] In a preferred embodiment, the high-dimensional feature vectors output by the dual-stream temporal modeling network DS-xNet are channel-wise concatenated and fused, and the classification results of farmland use status are obtained through linear mapping and softmax operation, including:

[0020] The high-dimensional feature vectors output by the temporal feature coding branches of Sentinel-1 and Sentinel-2 after modeling are fused through channel concatenation.

[0021] The fused high-dimensional feature vector is input into the decoder for classification prediction;

[0022] The decoder outputs the unified mapped high-dimensional feature vector to the category dimension, and obtains the multi-category classification results of the current status of cultivated land use through the softmax classifier.

[0023] In a preferred embodiment, model training employs a multi-supervision loss function, defined as follows: In the formula, This represents the total loss value. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-1 is used. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-2 is used. To fuse the cross-entropy loss of the output and the label; , , These are the weighting coefficients for each loss term.

[0024] In a preferred embodiment, all loss terms employ the cross-entropy loss function, defined as follows: In the formula, The first one representing the true label The one-hot encoded value of the class, Let be the predicted probability value of the model for the i-th class, C be the total number of classes, and log represent the natural logarithm function used to calculate the information loss.

[0025] And, a terminal for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data, comprising:

[0026] One or more processors;

[0027] A storage device on which one or more programs are stored;

[0028] When one or more programs are executed by one or more processors, the one or more processors implement a method for extracting farmland use status from multi-source remote sensing time-series data based on DS-xNet.

[0029] The technical effects achieved by this invention are as follows:

[0030] This invention combines Sentinel-1 and Sentinel-2 to reliably identify farmland and water bodies even under cloudy or foggy conditions, reducing missed detections and misjudgments caused by cloud cover. sNET captures short-term dynamics, while mNET's matrix memory enhances long-term dependency modeling, thereby improving the accuracy of identifying seasonal and interannual variations. Supervision is applied to the branch and fusion layers, enabling each branch to learn independent discrimination capabilities while collaboratively improving the fusion output, reducing overfitting, improving generalization ability, and accelerating training convergence. Channel splicing retains the discrimination information of each modality during the fusion stage, avoiding modal information conflicts or loss caused by direct early fusion, thus improving classification accuracy. The combined use of multiple optical and radar indices enhances the ability to distinguish paddy fields, irrigated areas, different crops, and non-farmland. Attached Figure Description

[0031] Figure 1 This is a flowchart of the method provided by the present invention; Figure 2 This is a network architecture diagram of the present invention; Figure 3 This is a schematic diagram of the specific structure of sNET and mNET of the present invention. Detailed Implementation

[0032] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0033] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0034] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0035] Furthermore, the present invention will be described in detail with reference to the schematic diagrams. When describing the embodiments of the present invention in detail, the schematic diagrams are merely examples for ease of explanation and should not limit the scope of protection of the present invention.

[0036] Please see the appendix Figure 1 , Figure 2 and Figure 3 As shown, a method for extracting farmland use status from multi-source remote sensing time-series data based on DS-xNet is provided, including: S1. Acquire multi-source remote sensing time-series data of the study area, including Sentinel-1 radar images and Sentinel-2 optical images. Extract VV polarization, VH polarization, dual polarization water index SDWI and SAR vegetation index DPSVIm from Sentinel-1 radar images. Extract NDVI, EVI, GNDVI, NDWI, REI and SAVI optical vegetation indices from Sentinel-2 optical images. S2. Select representative sample points based on multi-source remote sensing data and field survey data. The representative sample points include crop types and non-arable land types. Use ArcGIS to extract the temporal feature vectors of the representative sample points in the multi-source remote sensing time series data to construct a multi-source time series feature sample dataset. S3. Construct a two-stream temporal modeling network DS-xNet, which includes a temporal feature coding branch of Sentinel-1 and a temporal feature coding branch of Sentinel-2. The two-stream temporal modeling network DS-xNet adopts the xNET structure for modeling. S4. Channel splicing and fusion of the high-dimensional feature vectors output by the dual-stream time-series modeling network DS-xNet, and obtaining the classification results of the current status of cultivated land use through linear mapping and softmax operation; S5. During the model training process, a multi-supervised loss function is introduced into the output and fused output of the dual-stream temporal modeling network DS-xNet to obtain the trained DS-xNet model. The loss terms of the multi-supervised loss function include Loss1, Loss2 and Loss3, and all loss terms adopt the cross-entropy loss function. S6. Input the remote sensing image samples to be classified into the trained DS-xNet model to obtain the spatial location and classification results of the current farmland use type.

[0037] As described in steps S1 to S6 above, two types of complementary remote sensing time-series data are acquired: radar mode (Sentinel-1) and optical mode (Sentinel-2). Multiple images are collected at set time intervals to form a time-series sequence. Polarization channels (VV, VH) and radar-based indices (Dual Polarimetric Water Index SDWI, SAR Vegetation Index DPSVIm) are extracted from Sentinel-1. These indices are sensitive to water bodies, humidity, and physical structure, and are not affected by cloud cover. Commonly used optical vegetation / water indices (NDVI, EVI, GNDVI, NDWI, REI, SAVI) are extracted from Sentinel-2. These indices can reflect vegetation spectral characteristics, growth status, and moisture information. The original features are registered, missing values ​​are imputed, and normalized to allow the temporal features of both modalities to be directly input into the model on both temporal and numerical scales. Representative sample points (including various crop types and non-arable land types) are selected based on field surveys and expert annotations. ArcGIS is used to map the sample point locations onto multi-source time-series images and extract the corresponding temporal feature vectors, constructing a multi-source time-series feature sample dataset. The dataset is then divided into training and validation sets according to an optimal ratio to provide high-quality labeled data for supervised learning. DS-xNet consists of two parallel temporal coding branches: one branch specifically models Sentinel-1 temporal features, and the other branch specifically models Sentinel-2 temporal features. Each branch can target its respective modality... Physical meaning and noise characteristics are optimized through modeling. Each branch uses xNET as the backbone. xNET captures both short-term changes and long-term dependencies simultaneously by alternating stacked sNETs (short-term dynamic units) and mNETs (matrix memory enhancement units). sNETs excel at extracting short-term, irregular, or sudden dynamic features (e.g., rapid changes in greenness during the growth period), controlling information inflow, retention, and output through gating mechanisms (input gate, forget gate, output gate). mNETs introduce matrix-style memory (higher-order temporal relationships or interactive memory between multiple variables) within the unit, enhancing long-term dependency modeling capabilities and improving the numerical stability of attention computation through query, key-value mechanisms, and normalization and stabilization factors. xNET outputs more at each time step. Expressive hidden states and memory states can effectively capture complex temporal patterns caused by optical and radar variations due to seasons, irrigation, and harvesting. High-dimensional temporal feature vectors from the two branches are concatenated by channel and fused, preserving complementary information from different modalities. The fused features are projected onto the category dimension through a linear mapping (decoder), and a softmax classifier outputs the predicted probabilities for each category, thus obtaining multi-category classification results of farmland use status (crop type, non-farmland category, etc.) and their spatial distribution. During training, supervision (Loss1, Loss2, Loss3) is applied simultaneously to the intermediate outputs of the two branches and the final fused output. Each loss uses cross-entropy loss and can be assigned different weights to balance branch learning and fusion effects.Multiple supervision enables each branch to independently learn modality discrimination capabilities while collaboratively optimizing the final fusion prediction. This avoids over-reliance on the final result or information loss by any single branch, improving convergence speed and robustness. After training, the multi-source remote sensing samples to be classified are input into the trained DS-xNet to obtain pixel-by-pixel or sample point category predictions, corresponding probabilities, and spatial location information, forming a farmland use status layer and statistical results. Combined with Sentinel-1 and Sentinel-2, it can reliably identify farmland and water bodies even under cloud or fog conditions, reducing missed detections and false positives caused by cloud obscuration. sNET captures short-term dynamics (such as sowing / harvesting), while mNET... Matrix memory enhancement relies on long-term modeling (such as crop maturity cycles and crop rotation patterns) to improve the accuracy of identifying seasonal and interannual variations. Supervision is applied to the branching and fusion layers, enabling each branch to learn independent discriminative abilities while collaboratively improving the fusion output. This reduces overfitting, improves generalization ability, and accelerates training convergence. Channel splicing retains the discriminative information of each modality during the fusion stage, avoiding modal information conflicts or loss caused by direct early fusion, thereby improving classification accuracy. A combination of various optical and radar indices (NDVI, EVI, SDWI, DPSVIm, etc.) enhances the ability to distinguish between paddy fields, irrigated areas, different crops, and non-arable land (buildings, bare land, forest land, etc.).

[0038] In a preferred embodiment, the Sentinel-1 radar imagery acquisition of the study area takes place at a time interval of 15 days, while the Sentinel-2 optical imagery acquisition of the study area takes place at a time interval of 30 days.

[0039] As mentioned above, in the acquisition of multi-source remote sensing time-series data, Sentinel-1 radar imagery, due to its all-weather and all-time imaging capabilities, can acquire image sequences of the study area at 15-day intervals, thereby forming a high-density time-series sequence within a crop growth cycle. This effectively compensates for the loss of optical imagery due to cloud cover or poor weather, ensuring the continuity of the overall time-series data. While Sentinel-2 optical imagery provides rich spectral information, it is greatly affected by weather and cloud cover. Therefore, a 30-day interval is used for acquisition to ensure a moderate amount of data and maintain the continuity of the time series. This not only reflects the spectral variation characteristics of crops during key growth periods but also avoids redundancy and storage and computational pressure caused by excessively high-frequency acquisition. By setting two different acquisition intervals, they can complement each other in the time dimension. Radar data provides high-frequency structural and scattering characteristics, while optical data provides relatively low-frequency but richer spectral information. The differentiated setting of the time interval ensures modal complementarity, ultimately improving the accuracy and stability of farmland use status classification.

[0040] In a preferred implementation, the training set and validation set are divided into multiple time series feature sample data in the multi-source time series feature sample dataset according to a 7:3 ratio.

[0041] As described above, after constructing the multi-source time-series feature sample dataset, all sample data are divided into training and validation sets in a 7:3 ratio. The training set is used for parameter learning and model fitting of the DS-xNet network, while the validation set is used to evaluate the model's generalization ability and classification performance during training, avoiding model overfitting. This fixed ratio division can maintain a reasonable balance between model training and performance evaluation, ensuring that the model can still maintain good performance on unseen data, and improving the reliability and stability of classification results.

[0042] In a preferred implementation, xNET consists of alternating stacked sNET and mNET units, where sNET is used for short-term dynamic feature extraction and mNET is used to enhance the modeling ability for long-term dependencies by introducing a matrix memory structure.

[0043] As described above, the xNET network is composed of alternating stacked sNET and mNET units. The sNET unit mainly performs short-term dynamic modeling on the input temporal features, and can capture rapid growth changes or mutations of crops on a short time scale (such as feature fluctuations in key stages such as sowing, greening, and harvesting). The mNET unit, based on the traditional recurrent neural structure, introduces a matrix memory structure, and records long-range dependency features across time steps by constructing high-order relationships between query vectors, key vectors, and value vectors, thereby enhancing the ability to model the changing trends of crops throughout their entire growth cycle. The alternating stacking method allows the model to focus on both short-term dynamics and long-term trends during the layer-by-layer feature extraction process, forming a full-scale model of remote sensing temporal features. sNET captures short-term dynamic changes, while mNET strengthens long-term dependencies. The alternating stacking of the two improves the model's ability to characterize the temporal features of the entire crop growth cycle. By simultaneously utilizing short-term fluctuations and long-term trend information, it can more accurately distinguish between different crop types and cultivated and non-cultivated land, avoiding classification bias caused by a single time scale.

[0044] In a preferred embodiment, xNET consists of alternately stacked sNET and mNET units, and the calculation process includes the following formula: Cellular infusion: ; Input Gate: ; Forgotten Gate: ; Output gate: ; Stabilization factor: ; Stabilize the input gate: ; Stabilization forgetting gate: ; Cell status update: ; Normalized state update: ; Hidden state output: ; In the formula, This represents the input feature vector at time step t. This represents the hidden state of the previous time step. It is the cell input at the current time step, generated by the transformation between the current input and the previous hidden state. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This indicates the current state of the cell. This indicates the cell state at the previous moment. For normalized state factors, This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. Represents the hyperbolic tangent function. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , , These represent the input transformation weights that control the candidate state, input gate, forget gate, and output gate, respectively. , , , These represent the recurrent weights for the linear transformation of the four gates from their previous hidden state. These represent the bias terms for these four gates, respectively.

[0045] As described above, xNET models temporal features using alternating stacked sNET and mNET units. Its basic computation process follows a gated loop structure, incorporating stabilization factors and normalization mechanisms. Cell input is generated from the input feature vector of the current time step and the hidden state of the previous time step, serving as the basic input for temporal modeling. The input gate, forget gate, and output gate control the writing, forgetting, and output of information, respectively. The sigmoid activation function restricts the information flow to the [0,1] interval, enabling dynamic control of information flow. An exponential stabilization factor is introduced in the calculation of the input and forget gates to adjust the gating range and prevent gradient explosion or vanishing in long-term sequence computation. Normalization operations are added to the traditional recurrent cell state update to ensure the distribution of state values ​​across time steps. The model is stable, enhancing the stability of training. The hidden state output is determined by the normalized cell state and the output gate, providing the temporal feature representation for that time step. Through the above calculation mechanism, xNET can dynamically control the retention and forgetting of information in the stepwise recursion of the time series, maintaining a balance between long-term dependencies and short-term dynamic features. The introduction of stabilization factors effectively prevents the gradient explosion or gradient vanishing problems common in long-term series modeling, improving model convergence and robustness. The gating mechanism ensures that the model can flexibly choose to retain key information or forget invalid features at different stages, making the dynamic changes of crop growth cycle more accurate. Combined with the dynamic regulation mechanism of input, forgetting and output gates, it can simultaneously take into account short-term mutations and long-term trend information, providing a more accurate temporal feature representation for the extraction of farmland use status.

[0046] In a preferred implementation, the mNET unit enhances its ability to model long-term dependencies by introducing a matrix memory structure. The calculation process includes the following formula: Query vector: , Key vector: , Value vector: , Input Gate: , Forgotten Gate: , Output gate: , Stabilization factor: , Normalized input gate: , Normalized Forgetting Gate: , Cell status update: , Normalized state update: , Hidden state output: , In the formula, , , These represent the query vector, key vector, and value vector generated in the feature space from the current input, respectively. for transpose, This serves as a matrix-style memory unit for the current moment, recording higher-order feature relationships across time steps. This indicates the cell state at the previous moment. This is a normalized state vector used to normalize the attention weights. The scaling factor ensures the stability of the dot product; Ensure the denominator is not less than 1 to avoid numerical instability; This represents the input feature vector at time step t. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , These represent the weight matrices input to the query, key, and value, respectively. The system learns how to generate attention vectors. , , These represent the weight matrices input to the gate, for transpose, for transpose, , , , , , These represent the corresponding bias vectors.

[0047] As described above, the mNET unit enhances its ability to model long-term dependencies through a matrix memory structure. The current input generates query vectors, key vectors, and value vectors in the feature space, used to calculate higher-order correlations between the input and historical information. The matrix memory unit records feature relationships across time steps, forming a high-dimensional representation of long-term dependencies. The input gate, forget gate, and output gate control the writing, forgetting, and output intensity of information, respectively. A sigmoid activation function is used to limit the gating to the [0,1] interval for dynamic adjustment. A stabilization factor is used to adjust the numerical range of the exponential gating, preventing gradient explosion or vanishing, while ensuring stable dot product calculations (denominator not less than 1). The gating output is normalized, and the attention weights are standardized, making the information fusion across time steps more balanced and stable. The input and historical matrix memory are weighted and updated to form new cell states. These states are then normalized to ensure stable state value distribution. The hidden state output is generated jointly by the normalized cell state and the output gate, used for the next time step or the final model output, carrying long-term dependency information. Through this calculation process, the mNET unit can dynamically learn and store long-term dependency features in the time series, while also considering the updating and output of short-term information. This enhances DS-xNet's modeling capability for multi-source remote sensing time series data. The matrix memory structure can record high-order feature relationships across time steps, enabling the model to more accurately depict the trend changes of crops throughout their entire growth cycle. It can handle multimodal and multi-timescale remote sensing data, improving the ability to extract complex farmland use status quo information.

[0048] In a preferred implementation, the high-dimensional feature vectors output by the dual-stream temporal modeling network DS-xNet are channel-wise concatenated and fused, and the classification results of farmland use status are obtained through linear mapping and softmax operation, including: The high-dimensional feature vectors output by the temporal feature coding branches of S401, Sentinel-1 and Sentinel-2 after modeling are fused through channel concatenation. S402. The fused high-dimensional feature vector is input into the decoder for classification prediction. S403, the decoder outputs the unified mapped high-dimensional feature vector to the category dimension, and obtains the multi-category classification results of the current status of cultivated land use through the softmax classifier.

[0049] As described in steps S401 to S403 above, the dual-stream temporal modeling network DS-xNet models Sentinel-1 radar images and Sentinel-2 optical images through two separate temporal feature encoding branches, resulting in two classes of high-dimensional feature vectors. The high-dimensional feature vectors output from the Sentinel-1 and Sentinel-2 branches are concatenated along the channel dimension to form a unified multi-source temporal feature representation. This operation enables spatial alignment and information integration of data features from different modalities, allowing the model to simultaneously utilize the structural and scattering characteristics of radar and the spectral characteristics of optics. The concatenated high-dimensional feature vector is input into the decoder for linear mapping, mapping the feature vector to the category dimension. The decoder converts the fused features into a vector representation suitable for classifier processing. The softmax function is used to convert the mapped feature vector into probability distributions for each category, obtaining multi-category classification results of farmland use status. Each pixel or sample point ultimately obtains its corresponding category and probability. Channel splicing and fusion integrates radar and optical features, enabling the model to fully utilize the complementary information provided by different sensors and improve the farmland type recognition capability. The fused high-dimensional features contain short-term dynamic features, long-term dependency features, and multimodal information. The classifier can more accurately distinguish between crop types and non-farmland types. The final prediction is completed through linear mapping and softmax classification, avoiding complex nonlinear post-processing and improving computational efficiency.

[0050] In a preferred implementation, model training employs a multi-supervision loss function, defined as follows: In the formula, This represents the total loss value. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-1 is used. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-2 is used. To fuse the cross-entropy loss of the output and the label; , , These are the weighting coefficients for each loss term.

[0051] As described above, during the training of the DS-xNet model, a multi-supervised loss function is introduced to supervise each branch and the fused output separately. The cross-entropy of the Sentinel-1 temporal feature encoding branch output and the true label is calculated to obtain the branch loss. This is used to guide the radar branch in learning short-term dynamic and structural features. The Sentinel-2 temporal feature encoding branch output is cross-entropy calculated with the true label to obtain the branch loss. This is used to guide the optical branch in learning spectral information and long-term trends. The high-dimensional features after channel fusion are output by the decoder and cross-entropy is calculated with the real labels to obtain the fusion loss. This is used to guide the model in learning multimodal integrated features, improving the final classification accuracy, and weighting each loss term according to a weighted coefficient. , , The total loss is calculated using the following formula: By optimizing network parameters through backpropagation, each branch and the fusion output are effectively supervised simultaneously, enabling end-to-end training across multiple scales and modalities. The multiple supervision mechanisms not only supervise the fusion output but also provide constraints on individual modal branches, allowing each branch to effectively learn features and avoiding gradient sparsity or loss of branch information caused by single supervision. By optimizing the fusion output and each branch simultaneously, the model can better integrate short-term dynamics, long-term dependencies, and multimodal information, resulting in more accurate classification results.

[0052] In a preferred implementation, all loss terms employ the cross-entropy loss function, defined as follows: In the formula, The first one representing the true label The one-hot encoded value of the class, Let C be the predicted probability of the model for class i, C be the total number of classes, and log represent the natural logarithm function used to calculate the information loss.

[0053] As described above, in the training of the DS-xNet model, all loss terms (including each branch and the fused output) use the cross-entropy loss function. Cross-entropy measures the difference between the model's predicted probability distribution and the true label distribution. The true label is represented by one-hot encoding, with a value of 1 for the i-th class and 0 for the other classes. For each sample, the difference between the model's predicted probability value for the i-th class and the true label is calculated, and the information loss is calculated using the natural logarithm function. Cross-entropy loss is used for backpropagation to optimize model parameters. It directly measures the degree of matching of probability distributions, provides a clear gradient signal, guides the model to maximize the prediction probability of the correct class, and suppresses the probability of the wrong class.

[0054] And, a terminal for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data, comprising: One or more processors; A storage device on which one or more programs are stored; When one or more programs are executed by one or more processors, the one or more processors implement a method for extracting farmland use status from multi-source remote sensing time-series data based on DS-xNet.

[0055] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A method for extracting farmland use status from multi-source remote sensing time-series data based on DS-xNet, characterized in that, include: Multi-source remote sensing time-series data of the study area were acquired, including Sentinel-1 radar images and Sentinel-2 optical images. VV polarization, VH polarization, dual polarization water index SDWI and SAR vegetation index DPSVIm were extracted from Sentinel-1 radar images, and NDVI, EVI, GNDVI, NDWI, REI and SAVI optical vegetation indices were extracted from Sentinel-2 optical images. Representative sample points were selected based on multi-source remote sensing data and field survey data. These representative sample points included crop types and non-arable land types. ArcGIS was used to extract the temporal feature vectors of the representative sample points in the multi-source remote sensing time-series data to construct a multi-source time-series feature sample dataset. A dual-stream temporal modeling network DS-xNet is constructed, which includes a temporal feature coding branch of Sentinel-1 and a temporal feature coding branch of Sentinel-2. The dual-stream temporal modeling network DS-xNet adopts the xNET structure for modeling. The high-dimensional feature vectors output by the dual-stream temporal modeling network DS-xNet are channel-separated and fused, and the classification results of the current status of cultivated land use are obtained through linear mapping and softmax operation. During model training, a multi-supervised loss function is introduced into the output and fused output of the dual-stream temporal modeling network DS-xNet to obtain the trained DS-xNet model. The loss terms of the multi-supervised loss function include Loss1, Loss2 and Loss3, and all loss terms adopt the cross-entropy loss function. The remote sensing image samples to be classified are input into the trained DS-xNet model to obtain the spatial location and classification results of the current farmland use type.

2. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, The Sentinel-1 radar imagery acquisition of the study area was conducted at a time interval of 15 days, while the Sentinel-2 optical imagery acquisition of the study area was conducted at a time interval of 30 days.

3. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, The training set and validation set are divided into multiple time series feature sample data in the multi-source time series feature sample dataset according to a 7:3 ratio.

4. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, xNET consists of alternating stacked sNET and mNET units, where sNET is used for short-term dynamic feature extraction and mNET is used to enhance the modeling ability for long-term dependencies by introducing a matrix memory structure.

5. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 4, characterized in that, xNET consists of alternating stacked sNET and mNET units, and the calculation process includes the following formula: Cellular infusion: ; Input Gate: ; Forgotten Gate: ; Output gate: ; Stabilization factor: ; Stabilize the input gate: ; Stabilization forgetting gate: ; Cell status update: ; Normalized state update: ; Hidden state output: ; In the formula, This represents the input feature vector at time step t. This represents the hidden state of the previous time step. It is the cell input at the current time step, generated by the transformation between the current input and the previous hidden state. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This indicates the current state of the cell. This indicates the cell state at the previous moment. For normalized state factors, This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. Represents the hyperbolic tangent function. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , , These represent the input transformation weights that control the candidate state, input gate, forget gate, and output gate, respectively. , , , These represent the recurrent weights for the linear transformation of the four gates from their previous hidden state. These represent the bias terms for these four gates, respectively.

6. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 4, characterized in that, The mNET unit enhances its ability to model long-term dependencies by introducing a matrix memory structure. The calculation process includes the following formula: Query vector: , Key vector: , Value vector: , Input Gate: , Forgotten Gate: , Output gate: , Stabilization factor: , Normalized input gate: , Normalized Forgetting Gate: , Cell status update: , Normalized state update: , Hidden state output: , In the formula, , , These represent the query vector, key vector, and value vector generated in the feature space from the current input, respectively. for transpose, This serves as a matrix-style memory unit for the current moment, recording higher-order feature relationships across time steps. This indicates the cell state at the previous moment. This is a normalized state vector used to normalize the attention weights. The scaling factor ensures the stability of the dot product; Ensure the denominator is not less than 1 to avoid numerical instability; This represents the input feature vector at time step t. , and These represent the activation values ​​of the input gate, forget gate, and output gate, respectively, used to control the intensity of information input, forgetting, and output. and These represent the outputs of the improved forget gate and input gate, respectively. This is a stabilization factor used to adjust the numerical range of the exponential gating to prevent gradient explosion. and Let m and m represent the values ​​of m and m respectively at the time point preceding time t. This represents the sigmoid activation function. Represents the natural logarithm function. Represents an exponential function. This represents element-wise multiplication. , , These represent the weight matrices input to the query, key, and value, respectively. The system learns how to generate attention vectors. , , These represent the weight matrices input to the gate, for transpose, for transpose, , , , , , These represent the corresponding bias vectors.

7. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, The high-dimensional feature vectors output by the dual-stream temporal modeling network DS-xNet are channel-wise concatenated and fused, and the classification results of farmland use status are obtained through linear mapping and softmax operation, including: The high-dimensional feature vectors output by the temporal feature coding branches of Sentinel-1 and Sentinel-2 after modeling are fused through channel concatenation. The fused high-dimensional feature vector is input into the decoder for classification prediction; The decoder outputs the unified mapped high-dimensional feature vector to the category dimension, and obtains the multi-category classification results of the current status of cultivated land use through the softmax classifier.

8. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, Model training employs a multi-supervised loss function, defined as follows: In the formula, This represents the total loss value. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-1 is used. The cross-entropy loss between the temporal feature encoding branch and the label in Sentinel-2 is used. To incorporate the cross-entropy loss between the output and the label; , , These are the weighting coefficients for each loss term.

9. The method for extracting farmland use status based on DS-xNet multi-source remote sensing time-series data according to claim 1, characterized in that, All loss terms use the cross-entropy loss function, defined as follows: In the formula, The first one representing the true label The one-hot encoded value of the class, Let be the predicted probability value of the model for the i-th class, C be the total number of classes, and log represent the natural logarithm function used to calculate the information loss.

10. A terminal for extracting farmland use status based on multi-source remote sensing time-series data using DS-xNet, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When one or more programs are executed by one or more processors, the one or more processors implement the method for extracting farmland use status based on multi-source remote sensing time-series data according to any one of claims 1 to 9.

Images