Remote sensing image-oriented multi-scale time sequence characteristic collaborative rice extraction method

Abstract

The invention discloses a remote sensing image-oriented multi-scale time sequence feature collaborative rice extraction method, which comprises the following steps of: constructing a CDCPoter-Unet multi-scale feature extraction network, and inputting input time sequence data containing VV and VH wavebands into the multi-scale feature extraction network; multi-scale spatial feature extraction is carried out through an encoder and a multi-layer CPAT module integrated with the encoder, and deep features are obtained; inputting the deep features into a ConvLSTM unit to capture and model a time sequence dependency relationship; the method solves the problems of boundary ambiguity and category confusion caused by weak global and local feature combination capability and insufficient shallow and deep information fusion of an existing rice mapping method, and can be used for performing up-sampling reconstruction on the fused spatial and temporal features under the condition that computing resources are limited. The time sequence context and the multi-scale spatial features are effectively coordinated, long-range dependence modeling and local detail keeping are considered, and therefore high-precision boundary recognition and category distinguishing are achieved in a complex planting scene.

Description

Technical Field

[0001] This invention belongs to the field of agricultural remote sensing image processing technology, and in particular relates to a method for collaborative extraction of multi-scale temporal features of rice from remote sensing images. Background Technology

[0002] Deep learning models are essential for agricultural remote sensing. In terms of modeling methods, deep learning models, especially Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs), are widely used due to their ability to effectively capture spatiotemporal dependencies in Synthetic Aperture Radar (SAR) images. Recent research shows that spatiotemporal attention mechanisms are receiving increasing attention, aiming to collaboratively model dynamic crop growth processes with static spatial structures. While self-attention-based methods such as Transformers have shown potential in extracting long-range dependencies, they typically require significant computational resources and exhibit poor generalization ability in applications with limited samples of remote sensing time-series data. Furthermore, these methods often neglect crucial fusion between spatiotemporal dimensions or are prone to overfitting due to overly complex structures.

[0003] In spatial modeling research, U-Net and its variants remain the dominant architecture in SAR rice mapping due to their encoder-decoder structure, multi-scale feature representation capabilities, and good adaptability to temporal fusion. However, existing methods still face ongoing challenges in accurately extracting paddy field boundaries and distinguishing rice from other vegetation types with similar spectral characteristics, especially in areas with irregular shapes and mixed cropping.

[0004] Therefore, it is necessary to design a multi-scale temporal feature collaborative rice extraction method for remote sensing images to solve the above problems. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a multi-scale temporal feature collaborative rice extraction method for remote sensing images. It aims to solve the problems of boundary blurring and category confusion caused by the weak ability to combine global and local features and the insufficient fusion of shallow and deep information in existing rice mapping methods. Under the condition of limited computing resources, it can effectively coordinate temporal context and multi-scale spatial features, and take into account both long-range dependency modeling and local detail preservation, so as to achieve high-precision boundary recognition and category differentiation in complex planting scenarios.

[0006] To achieve the above-mentioned technical effects, the technical solution adopted by the present invention is as follows: A method for collaborative extraction of multi-scale temporal features of rice from remote sensing images includes the following steps: S1, construct the CDCPotter-Unet multi-scale feature extraction network, which includes an encoder, a ConvLSTM spatiotemporal unit and a decoder in sequence; the encoder integrates a multi-level central difference adaptive attention module CPAT. S2, inputs the input time series data containing VV and VH bands into the multi-scale feature extraction network; S3 extracts deep features by multi-scale spatial feature extraction through the encoder and its integrated multi-layer CPAT module. S4, input deep features into the ConvLSTM unit to capture and model temporal dependencies; S5 uses a decoder to upsample and reconstruct the fused spatiotemporal features, and finally outputs a spatial distribution map of rice.

[0007] Preferably, the multi-level cascaded and skip connection relationship between the encoder and the CDCPotter module is as follows: a. The first layer of the encoder performs convolution processing on the input data, and its output is simultaneously fed into the second layer downsampling operation of the encoder and the first layer CPAT module, and forms a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder. b. The output of the second-layer downsampling of the encoder is added to the output of the second-layer CPAT module, and the sum is used as the input of the next-layer convolution operation, and forms a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder; c. The output of the third layer convolution of the encoder is added to the output of the third layer CPAT module. One path of the sum is used as the input of the next layer convolution operation, and the other path is used as the input of the first layer ConvLSTM unit. The output of the first layer ConvLSTM unit forms a skip connection with the above sample operation of the corresponding layer of the decoder. d. The output of the second-to-last downsampled layer of the encoder is added to the output of the CPAT module at the same level, and the sum is fed into the last downsampled layer operation and the second-layer ConvLSTM unit; the output of the second-layer ConvLSTM unit forms a skip connection with the above-mentioned sampling operation at the corresponding level of the decoder. e. The output of the last layer downsampled by the encoder is added to the output of the last layer CPAT module, and the sum is used as the input of the end ConvLSTM unit.

[0008] Preferably, the encoder's downsampling module operates according to the following procedure: The input features are first transformed by a 3x3 convolutional layer; then non-linear mapping is performed by the ReLU activation function; finally, a 2x2 max pooling layer is used to reduce the spatial size of the feature map and complete the downsampling operation.

[0009] Preferably, the operation flow of the decoder's upsampling module is as follows: The spatial size of the feature map is increased by upsampling the input features; The upsampled features are then skipped and connected to the features at the corresponding level in the encoder path. The fused features are input into a 3x3 convolutional layer for smoothing and feature integration. Nonlinear transformation is achieved using the ReLU activation function.

[0010] Preferably, the CDCPotter module implements feature transfer and fusion through a main-sub-branch architecture. The feature transfer and fusion relationship between the main branch and the sub-branch is as follows: a. The input features first undergo a Patch Embedding operation, and its output is fed into the first-level CPAT module of the main branch. The output of the first-level CPAT module of the main branch is fed into the second-level CPAT module of the main branch, and then fed into the first-level CPAT module of the sub-branch after a Patch Split operation. b. The output of the second-level CPAT module of the main branch is input to the third-level CPAT module of the main branch after the second-level Patch Merging operation. The other output is added to the output of the first-level CPAT module of the sub-branch after the Patch Split operation, and then input to the second-level CPAT module of the sub-branch. c. The output of the third-level CPAT module of the main branch is input to the subsequent CPAT module of the main branch after the third-level Patch Merging operation. The other output is added to the output of the second-level CPAT module of the sub-branch after the Patch Split operation, and then input to the third-level CPAT module of the sub-branch. d. Similarly, the output of the second-to-last CPAT module of the main branch is input to the last CPAT module of the main branch after the corresponding level of Patch Merging operation. The other output is added to the output of the second-to-last CPAT module of the sub-branch after the Patch Split operation, and then input to the last CPAT module of the sub-branch. e. The output of the last CPAT module of the sub-branch is used as the header output of the sub-branch; f. Furthermore, the output of each CPAT module in the main branch is combined with the downsampling portion of the encoder.

[0011] Preferably, the Patch Embedding and Patch Merging are convolution-based block embedding methods, and their process includes the following steps: Input a dimension as tensor; The input tensor is projected using a Conv3×3 layer, with the kernel size and stride of the convolutional layer set to the same target value, thereby reducing the spatial size of the input tensor by the stride and transforming its number of channels to a preset embedding dimension. Output a dimension of The tensor.

[0012] Preferably, the CPAT module processes the input features in the following manner: The input features are passed to the BatchNorm layer for processing; The features processed by BatchNorm are input into the CDCPooling module, and the features processed by BatchNorm are added to and fused with the output of the CDCPooling module. The result of the above addition and fusion is then input back into the BatchNorm layer; The features processed by BatchNorm are input into the MLP part for transformation, and the output of the MLP part is weighted and combined with the features after addition and fusion to form the final output of the CPAT module.

[0013] Preferably, the CPAT module further includes an adaptive attention mechanism utilizing global pooling difference, which is implemented through a parallel processing dual-branch structure: The ordinary convolution branch first performs global pooling to compress the input features, then extracts spatial features through depthwise convolution (DWConv 3×3), and finally generates channel attention weight vectors through the ReLU activation function and the fully connected layer. Meanwhile, the central difference convolution branch uses central difference convolution (CDCConv 3×3) to extract the gradient structure information of the input features, and generates another channel attention weight vector through ReLU and the fully connected layer; The weight vectors generated by the two branches are multiplied by the original input features channel by channel, and the two weighted features are added and fused to form the final output of the adaptive attention module. The center difference convolution (CDCConv) extracts edge features by calculating the difference between the center pixel and the surrounding pixels. Its formula is: ; In the formula, For the input feature map, The spatial coordinates of the center pixel. Here, M represents the channel index of the input feature map, and M is the total number of channels. For the convolution kernel relative to the center pixel The offset, r is the radius that defines the neighborhood of the convolution kernel. For the offset Convolution kernel weights at position m and channel m; The depthwise convolution DWConv processes each input channel through independent spatial filtering, and its calculation process is described as follows: ; In the formula, Using the channel indices of the input feature map, the depthwise convolution processes each channel independently. The size of the depthwise convolution kernel; For corresponding channels ,Location The convolution kernel weights; This is to output the spatial coordinates of the current point on the feature map.

[0014] Preferably, the ConvLSTM unit is composed of three stacked ConvLSTM layers, wherein the calculation of each layer at time t follows the following formula: ; Among them, i t It is an input gate, f t It's the Gate of Oblivion, g t It is a memory unit, o t It's an output gate, C t It is in a stacked state, H t It refers to the update method, superscript. l =1,2,3 represent the number of stacked layers, represent the convolution operation, and ⊙ represents element-wise multiplication. σ The activation function is Sigmoid; for the first layer ( l =1), its input X t (1) For external input to the module; for higher levels ( l ≥2), its input X t (l) For the previous layer at time t Output H t (l−1) The input to the deep ConvLSTM comes from the state output of the hidden state of the previous layer, forming a stacked structure.

[0015] Preferably, the network output is supervised using a composite loss function, which is calculated as follows: ; Loss C Cross-entropy Loss calculated for the feature map output from the last layer of the decoder; Loss DThe Dice Loss is calculated for the feature map output from the head of the sub-branch of the CDCPotter module.

[0016] The beneficial effects of this invention are as follows: 1. This invention constructs a CDCPotter-Unet multi-scale feature extraction network and integrates a multi-layer CPAT module into the encoder to achieve collaborative extraction and enhancement of multi-scale spatial features. This design effectively solves the problems of weak combination of global and local features and insufficient fusion of shallow and deep information in existing methods, significantly improving the model's ability to identify rice boundaries and the accuracy of category distinction, especially in areas with irregular shapes and mixed vegetation, thus achieving more refined spatial structure reconstruction in complex planting scenarios.

[0017] 2. By introducing ConvLSTM spatiotemporal units to perform temporal modeling of deep features, this invention can effectively capture the dynamic changes in rice growth, overcoming the shortcomings of traditional methods in modeling temporal dependencies. This mechanism coordinates multi-scale spatial features with temporal context information, enhancing the model's ability to perceive the rice growth cycle, avoiding class confusion caused by insufficient utilization of temporal information, and improving the model's robustness and generalization ability in variable environments.

[0018] 3. This invention employs a dual-branch CDCPotter module structure, achieving multi-level feature transfer and fusion through operations such as Patch Embedding, Split, and Merging. Combined with the adaptive attention mechanism in the CPAT module, it further enhances the model's ability to extract edge and detail features. This design maintains high feature representation efficiency even under limited computational resources, effectively addressing the problems of overfitting and poor generalization ability in existing complex models, achieving a balance between high accuracy and high efficiency. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall CDCPotter-Unet network structure in an embodiment of the present invention; Figure 2 This is a schematic diagram of the main and secondary dual-branch architecture of the CDCPotter module in an embodiment of the present invention. Detailed Implementation

[0020] Example 1: A method for collaborative extraction of multi-scale temporal features of rice from remote sensing images includes the following steps: S1. Construct a CDCPotter-Unet multi-scale feature extraction network, which includes an encoder, a ConvLSTM spatiotemporal unit, and a decoder in sequence; the encoder integrates a multi-level central difference adaptive attention module CPAT. S2, inputs the input time series data containing VV and VH bands into the multi-scale feature extraction network; S3 extracts deep features by multi-scale spatial feature extraction through the encoder and its integrated multi-layer CPAT module. S4, input deep features into the ConvLSTM unit to capture and model temporal dependencies; S5 uses a decoder to upsample and reconstruct the fused spatiotemporal features, and finally outputs a spatial distribution map of rice.

[0021] like Figure 1 As shown, preferably, the multi-layer cascading and skip connection relationship between the encoder and the CDCPotter module is as follows: a. The first layer of the encoder performs convolution processing on the input data, and its output is simultaneously fed into the second layer downsampling operation of the encoder and the first layer CPAT module, and forms a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder. b. The output of the encoder's second-layer downsampling is added to the output of the second-layer CPAT module, and the sum is used as the input to the next-layer convolution operation, forming a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder; c. The output of the third layer convolution of the encoder is added to the output of the third layer CPAT module. One path of the sum is used as the input of the next layer convolution operation, and the other path is used as the input of the first layer ConvLSTM unit. The output of the first layer ConvLSTM unit forms a skip connection with the above sample operation of the corresponding layer of the decoder. d. The output of the second-to-last downsampling layer of the encoder is added to the output of the CPAT module at the same level, and the sum is fed into the last downsampling operation and the second-level ConvLSTM unit; the output of the second-level ConvLSTM unit forms a skip connection with the above-mentioned sampling operation at the corresponding level of the decoder. e. The output of the last layer downsampled by the encoder is added to the output of the last layer CPAT module, and the sum is used as the input of the end ConvLSTM unit.

[0022] Preferably, the encoder's downsampling module operates according to the following procedure: The input features are first transformed by a 3x3 convolutional layer; then non-linear mapping is performed by the ReLU activation function; finally, a 2x2 max pooling layer is used to reduce the spatial size of the feature map and complete the downsampling operation.

[0023] Preferably, the operation flow of the decoder's upsampling module is as follows: The spatial size of the feature map is increased by upsampling the input features; The upsampled features are then skipped and connected to the features at the corresponding level in the encoder path. The fused features are input into a 3x3 convolutional layer for smoothing and feature integration. Nonlinear transformation is achieved using the ReLU activation function.

[0024] like Figure 2 As shown, preferably, the CDCPotter module implements feature transfer and fusion through a main-sub-branch architecture. The feature transfer and fusion relationship between the main branch and the sub-branch is as follows: a. The input features first undergo a Patch Embedding operation, and its output is fed into the first-level CPAT module of the main branch. The output of the first-level CPAT module of the main branch is fed into the second-level CPAT module of the main branch, and then fed into the first-level CPAT module of the sub-branch after a Patch Split operation. b. The output of the second-level CPAT module of the main branch is input to the third-level CPAT module of the main branch after the second-level Patch Merging operation. The other output is added to the output of the first-level CPAT module of the sub-branch after the Patch Split operation, and then input to the second-level CPAT module of the sub-branch. c. The output of the third-level CPAT module of the main branch is input to the subsequent CPAT module of the main branch after the third-level Patch Merging operation. The other output is added to the output of the second-level CPAT module of the sub-branch after the Patch Split operation, and then input to the third-level CPAT module of the sub-branch. d. Similarly, the output of the second-to-last CPAT module of the main branch is input to the last CPAT module of the main branch after the corresponding level of Patch Merging operation. The other output is added to the output of the second-to-last CPAT module of the sub-branch after the Patch Split operation, and then input to the last CPAT module of the sub-branch. e. The output of the last CPAT module of the sub-branch is used as the header output of the sub-branch; f. Furthermore, the output of each CPAT module in the main branch is combined with the downsampling portion of the encoder.

[0025] Preferably, the Patch Embedding and Patch Merging are convolution-based block embedding methods, and their process includes the following steps: Input a dimension as tensor; The input tensor is projected using a Conv3×3 layer, with the kernel size and stride of the convolutional layer set to the same target value, thereby reducing the spatial size of the input tensor by the stride and transforming its number of channels to a preset embedding dimension. Output a dimension of The tensor.

[0026] Preferably, the CPAT module processes the input features in the following manner: The input features are passed to the BatchNorm layer for processing; The features processed by BatchNorm are input into the CDCPooling module, and the features processed by BatchNorm are added to and fused with the output of the CDCPooling module. The result of the above addition and fusion is then input back into the BatchNorm layer; The features processed by BatchNorm are input into the MLP part for transformation, and the output of the MLP part is weighted and combined with the features after addition and fusion to form the final output of the CPAT module.

[0027] Preferably, the CPAT module further includes an adaptive attention mechanism utilizing global pooling difference, which is implemented through a parallel processing dual-branch structure: The ordinary convolution branch first performs global pooling to compress the input features, then extracts spatial features through depthwise convolution (DWConv 3×3), and finally generates channel attention weight vectors through the ReLU activation function and the fully connected layer. Meanwhile, the central difference convolution branch uses central difference convolution (CDCConv 3×3) to extract the gradient structure information of the input features, and generates another channel attention weight vector through ReLU and the fully connected layer; The weight vectors generated by the two branches are multiplied by the original input features channel by channel, and the two weighted features are added and fused to form the final output of the adaptive attention module. The center difference convolution (CDCConv) extracts edge features by calculating the difference between the center pixel and the surrounding pixels. Its formula is: ; In the formula, For the input feature map, The spatial coordinates of the center pixel. Here, M represents the channel index of the input feature map, and M is the total number of channels. For the convolution kernel relative to the center pixel The offset, r is the radius that defines the neighborhood of the convolution kernel. For the offset Convolution kernel weights at position m and channel m; The depthwise convolution DWConv processes each input channel through independent spatial filtering, and its calculation process is described as follows: ; In the formula, Using the channel indices of the input feature map, the depthwise convolution processes each channel independently. The size of the depthwise convolution kernel; For corresponding channels ,Location The convolution kernel weights; This is to output the spatial coordinates of the current point on the feature map.

[0028] Preferably, the ConvLSTM unit is composed of three stacked ConvLSTM layers, wherein the calculation of each layer at time t follows the following formula: ; Among them, i t It is an input gate, f t It's the Gate of Oblivion, g t It is a memory unit, o t It's an output gate, C t It is in a stacked state, H t It refers to the update method, superscript. l =1,2,3 represent the number of stacked layers, represent the convolution operation, and ⊙ represents element-wise multiplication. σ The activation function is Sigmoid; for the first layer ( l =1), its input X t (1) For external input to the module; for higher levels ( l ≥2), its input X t (l) For the previous layer at time t Output H t (l−1) The input to the deep ConvLSTM comes from the state output of the hidden state of the previous layer, forming a stacked structure.

[0029] Preferably, the network output is supervised using a composite loss function, which is calculated as follows: ; Loss CCross-entropy Loss calculated for the feature map output from the last layer of the decoder; Loss D The Dice Loss is calculated for the feature map output from the head of the sub-branch of the CDCPotter module.

[0030] Example 2: The input data consists of pre-processed VV and VH band SAR time-series data, which are fed into the CDCPotter-UNet network for feature extraction and reconstruction. The network adopts an encoder-decoder architecture and, through multi-level feature processing and fusion, finally outputs a spatial distribution map of rice.

[0031] In the encoder stage, the input data is used to construct a multi-scale feature pyramid through convolution and downsampling operations. The output of each layer of the encoder is injected into the corresponding CPAT module for feature enhancement. Shallow features retain spatial details and are directly passed to the decoder through skip connections. Mid-layer features are partially input into ConvLSTM units for temporal modeling after CPAT enhancement. Deep features are finally input into the terminal ConvLSTM unit after multiple downsampling and CPAT processing. The encoder's downsampling module reduces the feature size while maintaining feature expressiveness through 3×3 convolution, ReLU activation, and 2×2 max pooling.

[0032] The CDCPotter module employs a main-sub-branch architecture for feature propagation and fusion. The main branch projects features using PatchEmbedding and transforms the input tensor from [B,C,H,W] to [B,embed_dim,H / stride,W / stride] using Conv3×3 convolutional kernels. The CPAT outputs from each layer of the main branch feed features to the sub-branch via Patch Split and are simultaneously reassembled using Patch Merging. The sub-branch receives features from each layer of the main branch and fuses them step-by-step to achieve multi-scale feature aggregation. The outputs from each layer of the main branch are also combined with downsampled features from the encoder, forming a spatial-channel synergistic enhancement. Finally, the output of the last layer of the sub-branch's CPAT module serves as the head output for loss calculation.

[0033] Each CPAT module enhances features through a refined processing flow. First, features are standardized using a BatchNorm layer and then input into the CDCPooling module. CDCPooling employs an adaptive attention mechanism with global pooling difference, using a two-branch parallel processing: a regular convolutional branch performs global pooling compression and extracts spatial features using DWConv 3×3; a central difference convolutional branch extracts gradient structure information using CDCPoov 3×3. Each branch generates a channel attention weight vector, which is then multiplied channel-by-channel with the original input features. Finally, the two weighted features are added and fused to form the enhanced feature output.

[0034] The network uses ConvLSTM units to model temporal dependencies. Features from the encoder's middle and deep layers are input into two ConvLSTM units respectively. The ConvLSTM employs a three-layer stacked structure, using input gates, forget gates, output gates, and memory units to pass temporal information. The outputs of each ConvLSTM layer are connected to the corresponding layers of the decoder via skip connections, ensuring the effective use of temporal information during the decoding process.

[0035] In the decoder stage, spatial resolution is gradually restored through upsampling operations. After each upsampling, the features are fused with the corresponding layer features from the encoder via skip connections, followed by 3×3 convolution and ReLU activation to complete feature reconstruction. This process fully utilizes the spatial detail information preserved by the encoder and the temporal features extracted by ConvLSTM to achieve accurate feature reconstruction.

[0036] Network training employs a composite loss function for collaborative supervision. The output of the last layer of the decoder calculates the cross-entropy loss to ensure pixel-level classification accuracy; the head output of the CDCPotter sub-branch calculates the Dice loss to optimize region segmentation. By weighted fusion of these two losses, end-to-end collaborative optimization is achieved, improving the model's generalization ability in complex scenarios.

[0037] Example 3: This embodiment provides an experimental process for applying a multi-scale temporal feature collaborative rice extraction method for remote sensing images. The process is as follows: 1. Experiment setup and parameter configuration: All experiments were performed on NVIDIA A100 GPUs using the PyTorch 2.0 framework. The AdamW optimizer was used for network training, with an initial learning rate of 0.0001 and a batch size of 4. Cosine annealing was used for learning rate scheduling during training, with a maximum of 40 iterations. Experimental data from the Arkansas River Basin in the United States, collected in 2017 and 2018, were divided into training and validation sets in a 6:4 ratio. Data from 2019 was used as an independent test set to evaluate spatiotemporal generalization ability. Comparison models included current mainstream rice mapping models such as U-Net, TFBS, BiLSTM, ConvLSTM, and STMA.

[0038] 2. Analysis of experimental results: In the Arkansas River Basin in 2017 and 2018, CDCPotter-UNet performed best, achieving an mIoU of 84.2%, significantly outperforming other models. Compared to the baseline Unet, CDCPotter-UNet improved mIoU by 2.3%. This indicates that the model can accurately predict rice planting areas. In particular, by utilizing the proposed CDCPotter, which considers both global and local information, the model can capture spatial features at different scales, improving its ability to recover details. The results are shown in Table 1 below. Table 1: Results of different models in the Arkansas River Basin, 2017-2018;

[0039] As shown in Table 2, in the 2019 Arkansas River Basin rice mapping, CDCPotter-UNet also performed exceptionally well, achieving an mIoU of 81.7%, significantly higher than other models, especially in OA-90.5% and F1-90.0%. Compared to the baseline model U-Net with an mIoU of 78.9%, CDCPotter-UNet demonstrated significantly stronger resolution generalization ability, maintaining high prediction accuracy across different spatial scales. Although the TFBS model also showed some advantages in this timeframe, CDCPotter-UNet still maintained a clear advantage in accuracy.

[0040] Table 2: Results of different models in the Arkansas River Basin in 2019;

[0041] To investigate the effectiveness of the two modules and evaluate their impact on model performance, the details are shown in Table 3 below: Table 3: Ablation Experiment Results;

[0042] As shown in Table 3 of the ablation experiment results, Main-Branch is more important than Sub-branch in improving model performance; when both modules are used simultaneously, CDCPotter-UNet achieves an mIoU of 84.2% on the test set, demonstrating the best performance. Compared to the baseline model U-Net's mIoU of 81.9%, the model performance is significantly improved. This indicates that the Main-Branch module effectively enhances feature extraction capabilities through spatial attention mechanisms, while the Sub-branch module provides shallow information to prevent data loss.

Claims

1. A method for collaborative extraction of multi-scale temporal features of rice from remote sensing images, characterized in that, Includes the following steps: S1, construct the CDCPotter-Unet multi-scale feature extraction network, which includes an encoder, a ConvLSTM spatiotemporal unit and a decoder in sequence; the encoder integrates a multi-level central difference adaptive attention module CPAT. S2, inputs the input time series data containing VV and VH bands into the multi-scale feature extraction network; S3 extracts deep features by multi-scale spatial feature extraction through the encoder and its integrated multi-layer CPAT module. S4, input deep features into the ConvLSTM unit to capture and model temporal dependencies; S5 uses a decoder to upsample and reconstruct the fused spatiotemporal features, and finally outputs a spatial distribution map of rice.

2. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 1, characterized in that, The multi-level cascading and skip connection relationships between the encoder and the CDCPotter module are as follows: a. The first layer of the encoder performs convolution processing on the input data, and its output is simultaneously fed into the second layer downsampling operation of the encoder and the first layer CPAT module, and forms a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder. b. The output of the second-layer downsampling of the encoder is added to the output of the second-layer CPAT module, and the sum is used as the input of the next-layer convolution operation, and forms a skip connection with the above-mentioned sampling operation of the corresponding layer of the decoder; c. The output of the third layer convolution of the encoder is added to the output of the third layer CPAT module. One path of the sum is used as the input of the next layer convolution operation, and the other path is used as the input of the first layer ConvLSTM unit. The output of the first layer ConvLSTM unit forms a skip connection with the above sample operation of the corresponding layer of the decoder. d. The output of the second-to-last downsampled layer of the encoder is added to the output of the CPAT module at the same level, and the sum is fed into the last downsampled layer operation and the second-layer ConvLSTM unit; the output of the second-layer ConvLSTM unit forms a skip connection with the above-mentioned sampling operation at the corresponding level of the decoder. e. The output of the last layer downsampled by the encoder is added to the output of the last layer CPAT module, and the sum is used as the input of the end ConvLSTM unit.

3. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 2, characterized in that, The encoder's downsampling module operates according to the following procedure: The input features are first transformed by a 3x3 convolutional layer; then non-linear mapping is performed by the ReLU activation function; finally, a 2x2 max pooling layer is used to reduce the spatial size of the feature map and complete the downsampling operation.

4. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 2, characterized in that, The operation flow of the decoder's upsampling module is as follows: The spatial size of the feature map is increased by upsampling the input features; The upsampled features are then skipped and connected to the features at the corresponding level in the encoder path. The fused features are input into a 3x3 convolutional layer for smoothing and feature integration. Nonlinear transformation is achieved using the ReLU activation function.

5. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 2, characterized in that, The CDCPotter module implements feature transfer and fusion through a main-sub-branch architecture. The feature transfer and fusion relationship between the main branch and the sub-branch is as follows: a. The input features first undergo a Patch Embedding operation, and its output is fed into the first-level CPAT module of the main branch. The output of the first-level CPAT module of the main branch is fed into the second-level CPAT module of the main branch, and then fed into the first-level CPAT module of the sub-branch after a Patch Split operation. b. The output of the second-level CPAT module of the main branch is input to the third-level CPAT module of the main branch after the second-level Patch Merging operation. The other output is added to the output of the first-level CPAT module of the sub-branch after the Patch Split operation, and then input to the second-level CPAT module of the sub-branch. c. The output of the third-level CPAT module of the main branch is input to the subsequent CPAT module of the main branch after the third-level Patch Merging operation. The other output is added to the output of the second-level CPAT module of the sub-branch after the Patch Split operation, and then input to the third-level CPAT module of the sub-branch. d. Similarly, the output of the second-to-last CPAT module of the main branch is input to the last CPAT module of the main branch after the corresponding level of Patch Merging operation. The other output is added to the output of the second-to-last CPAT module of the sub-branch after the Patch Split operation, and then input to the last CPAT module of the sub-branch. e. The output of the last CPAT module of the sub-branch is used as the header output of the sub-branch; f. Furthermore, the output of each CPAT module in the main branch is combined with the downsampling portion of the encoder.

6. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 5, characterized in that, The Patch Embing and Patch Merging are convolution-based block embedding methods, and their process includes the following steps: Input a dimension as tensor; The input tensor is projected using a Conv3×3 layer, with the kernel size and stride of the convolutional layer set to the same target value, thereby reducing the spatial size of the input tensor by the stride and transforming its number of channels to a preset embedding dimension. Output a dimension of The tensor.

7. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 6, characterized in that, The CPAT module processes input features in the following manner: The input features are passed to the BatchNorm layer for processing; The features processed by BatchNorm are input into the CDCPooling module, and the features processed by BatchNorm are added to and fused with the output of the CDCPooling module. The result of the above addition and fusion is then input back into the BatchNorm layer; The features processed by BatchNorm are input into the MLP part for transformation, and the output of the MLP part is weighted and combined with the features after addition and fusion to form the final output of the CPAT module.

8. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 7, characterized in that, The CPAT module also includes an adaptive attention mechanism that utilizes global pooling difference, which is implemented through a parallel processing dual-branch structure: The ordinary convolutional branch first performs global pooling to compress the input features, then extracts spatial features through depthwise convolution (DWConv), and finally generates channel attention weight vectors through the ReLU activation function and the fully connected layer. Meanwhile, the central difference convolution branch uses the central difference convolution CDConv to extract the gradient structure information of the input features, and generates another channel attention weight vector through ReLU and the fully connected layer; The weight vectors generated by the two branches are multiplied by the original input features channel by channel, and the two weighted features are added and fused to form the final output of the adaptive attention module. The center difference convolution (CDCConv) extracts edge features by calculating the difference between the center pixel and the surrounding pixels. Its formula is: ； In the formula, For the input feature map, The spatial coordinates of the center pixel. Here, M represents the channel index of the input feature map, and M is the total number of channels. For the convolution kernel relative to the center pixel The offset, r is the radius that defines the neighborhood of the convolution kernel. For the offset Convolution kernel weights at position m and channel m; The depthwise convolution DWConv processes each input channel through independent spatial filtering, and its calculation process is described as follows: ； In the formula, Using the channel indices of the input feature map, the depthwise convolution processes each channel independently. The size of the depthwise convolution kernel; For corresponding channels ,Location The convolution kernel weights; This is to output the spatial coordinates of the current point on the feature map.

9. The method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 1, characterized in that, The ConvLSTM unit is composed of three stacked ConvLSTM layers, and the calculation of each layer at time t follows the following formula: ； Among them, i t It's an input gate, f t It's the Gate of Oblivion, g t It is a memory unit, o t It's an output gate, C t It is in a stacked state, H t It refers to the update method, superscript. l =1,2,3 represent the number of stacked layers, represent the convolution operation, and ⊙ represents element-wise multiplication. σ The activation function is Sigmoid; for the first layer ( l =1), its input X t (1) For external input to the module; for higher levels ( l ≥2), its input X t (l) For the previous layer at time t Output H t (l−1) The input to the deep ConvLSTM comes from the state output of the hidden state of the previous layer, forming a stacked structure.

10. A method for collaborative extraction of multi-scale temporal features of rice from remote sensing images according to claim 4 or 5, characterized in that, The network output is supervised using a composite loss function, which is calculated as follows: ； Loss C Cross-entropy Loss calculated for the feature map output from the last layer of the decoder; Loss D The Dice Loss is calculated for the feature map output from the head of the sub-branch of the CDCPotter module.

Images