A rice field planting area extraction method based on LSTM-attention-UNet

Abstract

This invention relates to the field of remote sensing image processing technology and discloses a method for extracting paddy field planting area based on LSTM-Attention-UNet. The method includes: acquiring multi-temporal remote sensing images of the irrigation, tillering, grain-filling, and maturity stages and constructing a time-series image sequence; preprocessing the images and generating paddy field raster label data; constructing a time-series remote sensing dataset; inputting the time-series image sequence into ConvLSTM for recursive time-series feature modeling to generate a time-series feature map; weighting and fusing the features of each temporal stage through a time attention mechanism to obtain a time-series aggregated feature; and finally inputting the time-series aggregated feature into UNet for pixel-level segmentation to output the paddy field segmentation result. This invention effectively improves the extraction accuracy and boundary integrity of paddy field planting area through spatiotemporal feature collaborative modeling and attention mechanisms, and is suitable for monitoring rice planting in multi-source, multi-temporal remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, specifically to a method for extracting the planting area of ​​rice paddies based on LSTM-Attention-UNet. Background Technology

[0002] Rice is a major food crop in my country, and timely and accurate information on rice planting area and spatial distribution is of great practical significance for guiding grain production and food security. For a long time, grain planting area has been determined through sampling surveys and hierarchical reporting, but remote sensing technology, with its advantages of speed, efficiency, and objectivity, can become the mainstream method for monitoring rice paddies.

[0003] Current paddy field monitoring technologies mainly rely on ground statistical surveys and traditional remote sensing classification. Ground statistical surveys depend on grassroots reporting, on-site measurement, and ledger registration to obtain paddy field area and distribution, which is inefficient, costly, and time-consuming. It is also susceptible to human factors that can lead to inaccurate data and cannot quickly reflect dynamic changes in planting structures such as "dryland to paddy field conversion" or "paddy field to dryland conversion." Traditional remote sensing methods use single-temporal or limited-temporal remote sensing images to extract paddy fields through visual interpretation, supervised classification, unsupervised classification, and index calculation. However, these methods are easily affected by phenological periods and soil background, and the spectra of paddy fields are severely confused with those of wetlands, ponds, and ditches. They also have low automation and unstable accuracy, and cannot adapt to complex planting structures and fragmented field scenarios.

[0004] Semantic segmentation models such as CNN and UNet can automatically extract the planting area of ​​paddy fields, but they rely only on single-phase images and do not utilize the phenological rhythms of paddy fields. They are prone to misclassifying dry land, fallow land, and swamps as paddy fields, and are not sensitive to temporal changes such as paddy field flooding, growth, and harvest. They cannot distinguish between seasonal fallow land, dry land, and paddy fields, resulting in a large number of pseudo-patterns in the extraction results, and the boundary accuracy and completeness are difficult to meet the requirements. Therefore, this invention proposes a paddy field planting area extraction method based on LSTM-Attention-UNet. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting paddy field planting area based on LSTM-Attention-UNet, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for extracting the planting area of ​​rice paddies based on LSTM-Attention-UNet, comprising the following steps:

[0007] Step S1: Acquire multi-temporal remote sensing images covering key growth stages of rice, including the first remote sensing image of the irrigation stage, the second remote sensing image of the tillering stage, the third remote sensing image of the grain-filling stage, and the fourth remote sensing image of the maturity stage, and construct a time-series image sequence.

[0008] Step S2: Preprocess the first, second, third and fourth remote sensing images to obtain preprocessed first, second, third and fourth remote sensing images, and convert the paddy field vector data into raster label data that is spatially aligned with the remote sensing images;

[0009] Step S3: Construct a time-series remote sensing dataset from the preprocessed first, second, third, and fourth remote sensing images and paddy field label data, which will be used as input for model training.

[0010] Step S4: Input the time-series image sequences in the time-series remote sensing dataset into a convolutional long short-term memory network (ConvLSTM) for recursive time-series feature modeling. Update the spatial features of each time-phase image through a gating structure to generate a time-series feature map containing historical state information.

[0011] Step S5: Input the temporal feature map into the temporal attention mechanism module, assign weight coefficients to each temporal feature frame in the time dimension, generate each temporal weight through the normalization function, and perform weighted fusion of multi-temporal features to obtain temporal aggregated features.

[0012] Step S6: Input the temporal aggregated features into a U-shaped convolutional neural network (UNet) for pixel-level segmentation and output the paddy field segmentation result.

[0013] Preferably, the multi-temporal remote sensing images in step S1 include Sentinel-2 and Landsat series images, with a unified spatial resolution of 10m and a projection coordinate system of EPSG:4490.

[0014] Preferably, the preprocessing in step S2 includes radiometric calibration, band selection, reflectance normalization correction, multi-source sensor spectral consistency matching, cloud and snow shadow mask removal, and image median synthesis.

[0015] Preferably, the raster label data in step S2 is generated by converting surface to raster, with paddy fields marked as 1 and non-paddy fields marked as 0.

[0016] Preferably, the temporal feature map output by ConvLSTM in step S4 is used as the sole input to the temporal attention mechanism module in step S5.

[0017] Preferably, the temporal attention mechanism in step S5 is based on global average pooling and 1×1 convolution, and learns to generate normalized weight coefficients for each temporal feature frame.

[0018] Preferably, the UNet in step S6 includes an encoder and a decoder structure. The encoder extracts multi-scale features by downsampling layer by layer, and the decoder recovers spatial resolution by upsampling layer by layer and combining skip connections.

[0019] Preferably, the UNet employs a learnable hybrid activation function, dynamically balancing the combination of ReLU and Tanh-Softplus through learnable weight parameters.

[0020] Compared with the prior art, the beneficial effects of this invention are as follows:

[0021] 1. Spatiotemporal Feature Collaborative Modeling Improves Recognition Accuracy: By using ConvLSTM to perform temporal modeling on multi-temporal remote sensing images, the shortcomings of traditional semantic segmentation models in modeling temporal information are effectively compensated. Compared to segmentation methods that rely solely on single-temporal spatial features, ConvLSTM can capture the dynamic changes of ground features in the time series while preserving spatial structure information, completing high-level semantic encoding of the temporal dimension before entering the spatial segmentation network. This part enables the representation of the dynamic changes in the rice growth period, reduces phenological confusion between different ground features, and the introduction of ConvLSTM also optimizes the overall computational power allocation of the model, improves training efficiency, and enhances the overall classification accuracy.

[0022] 2. Attention mechanism highlights key temporal features: The temporal attention mechanism is used to perform weighted fusion of features of each time phase. Based on global average pooling and 1×1 convolution, normalized weight coefficients are learned for each temporal feature frame. Based on the weights, the features of multiple time phases are weighted and summed to highlight key growth period information, suppress redundant interference, and improve the extraction accuracy of paddy fields in areas with complex planting structures.

[0023] 3. Progressive Hybrid Activation Function Optimization: By using a learnable hybrid activation function to optimize the UNet structure, the combination of ReLU and Tanh-Softplus is dynamically balanced by learnable weight parameters, which effectively enhances the expressive ability of semantic features of paddy fields at different levels, while taking into account the extraction of shallow details such as paddy field boundaries and textures and deep semantic features such as plot categories.

[0024] 4. The method is highly versatile and easy to extend: For remote sensing data of different resolutions and different bands, it can also achieve accurate target segmentation and identification through corresponding adjustments and optimizations. It has wide applicability and good scalability. This method has excellent scalability. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the overall processing flow of the method of the present invention;

[0026] Figure 2 This is a structural diagram of the temporal segmentation model proposed in this invention;

[0027] Figure 3 This is the first remote sensing image of the irrigation season;

[0028] Figure 4 This is the second remote sensing image of the tillering stage;

[0029] Figure 5 This is the third remote sensing image during the grouting period;

[0030] Figure 6 This is a mature fourth-generation remote sensing image;

[0031] Figure 7 This is a label image of a rice paddy. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example

[0034] Please see Figures 1-2 The illustrated method for extracting rice paddy planting area based on LSTM-Attention-UNet includes the following steps:

[0035] Step S1: Acquire multi-temporal remote sensing images covering key growth stages of rice, including the first remote sensing image of the irrigation stage, the second remote sensing image of the tillering stage, the third remote sensing image of the grain-filling stage, and the fourth remote sensing image of the maturity stage, and construct a time-series image sequence.

[0036] Step S2: Preprocess the first, second, third and fourth remote sensing images to obtain preprocessed first, second, third and fourth remote sensing images, and convert the paddy field vector data into raster label data that is spatially aligned with the remote sensing images;

[0037] Step S3: Construct a time-series remote sensing dataset from the preprocessed first, second, third, and fourth remote sensing images and paddy field label data, which will be used as input for model training.

[0038] Step S4: Input the time-series image sequences in the time-series remote sensing dataset into a convolutional long short-term memory network (ConvLSTM) for recursive time-series feature modeling. Update the spatial features of each time-phase image through a gating structure to generate a time-series feature map containing historical state information.

[0039] Step S5: Input the temporal feature map into the temporal attention mechanism module, assign weight coefficients to each temporal feature frame in the time dimension, generate each temporal weight through the normalization function, and perform weighted fusion of multi-temporal features to obtain temporal aggregated features.

[0040] Step S6: Input the temporal aggregated features into a U-shaped convolutional neural network (UNet) for pixel-level segmentation and output the paddy field segmentation result.

[0041] The multi-temporal remote sensing images in step S1 include Sentinel-2 and Landsat series images, with a unified spatial resolution of 10m and a projection coordinate system of EPSG:4490.

[0042] The preprocessing in step S2 includes radiometric calibration, band selection, reflectance normalization correction, multi-source sensor spectral consistency matching, cloud and snow shadow masking removal, and image median synthesis.

[0043] In step S2, the raster label data is generated by converting the area to a raster, with paddy fields marked as 1 and non-paddy fields marked as 0. The temporal feature map output by ConvLSTM in step S4 serves as the sole input to the temporal attention mechanism module in step S5. Since there are significant differences in spectral features at different growth stages of rice, it is difficult to effectively distinguish between rice and non-paddy fields without temporal modeling of multi-temporal images and screening of key temporal information. Therefore, it is necessary to first extract temporal dependent features through ConvLSTM and then weight the key temporal phases through the temporal attention mechanism.

[0044] In step S5, the temporal attention mechanism is based on global average pooling and 1×1 convolution, and learns to generate normalized weight coefficients for each temporal feature frame.

[0045] The UNet in step S6 includes an encoder and a decoder structure. The encoder extracts multi-scale features by downsampling layer by layer, and the decoder restores spatial resolution by upsampling layer by layer and combining skip connections. The UNet uses a learnable hybrid activation function, which dynamically balances the combination of ReLU and Tanh-Softplus through learnable weight parameters.

[0046] The effectiveness of the method of the present invention will be verified below with specific embodiments. Taking a portion of Raohe County, Shuangyashan City, Heilongjiang Province, my country as the experimental area, representative high-resolution remote sensing image data from this region was selected as the experimental data source. In the data preprocessing stage, cloud, snow, and shadow masking algorithms were constructed for Sentinel-2 and Landsat series images respectively, removing high cloud cover, cloud shadows, and snow-covered interference pixels while retaining high-quality surface reflectance information. Radiometric calibration and band selection were completed for Sentinel-2 images, and for Landsat... The reflectance of the images was normalized and corrected, and the spectral consistency with Sentinel-2 was achieved by adjusting the spectral coefficients to eliminate the differences between the multiple sensor sources. The preprocessed Sentinel-2 and Landsat images were fused temporally, and the images were divided into four time periods according to the key growth stages of crops. The median composite results of the images in each time period were calculated. The band combination was adaptively selected according to the characteristics of different growth stages, using short-wave infrared and red band combinations (B12, B11, B4), near-infrared and red-green band combinations (B8, B4, B3), and near-infrared, short-wave infrared and red band combinations (B8, B11, B4) to highlight the spectral differences and texture features of crops at different growth stages. Finally, all images were unified with a spatial resolution of 10 m, and the projection coordinate system adopted was EPSG:4490. Then, the method proposed in this invention was used to conduct target segmentation and recognition experiments on the preprocessed images, and the results were compared with traditional segmentation methods.

[0047] The vector data of paddy fields was converted into raster data using the "Polygon to Raster" function in ArcGIS 10.8 software. The processing range and cell size were the same as those of the image, and the coordinate system was changed to CGCS2000. The regional data was then reclassified by setting the Value (paddy field planting area) to 1 and NoData to 0 to achieve binary classification of the paddy fields.

[0048] We have one image for each of the four temporal study areas, and one rice paddy raster label. Next, the "Clip_Dataset" program is used to segment the preprocessed images. The image and label sizes are X and Y = 256, and the segmentation step size is set to 256, including the boundaries. The training data for each remote sensing image includes both the image and the label (e.g., ...). Figures 3-7 As shown, where Figure 3 This is the first remote sensing image of the irrigation season. Figure 4 This is the second remote sensing image during the tillering stage. Figure 5 This is the third remote sensing image during the grouting period. Figure 6 This is a mature fourth-generation remote sensing image. Figure 7 (Label for rice paddies).

[0049] This example compares the proposed method with UNet, multi-temporal channel UNet, and LSTM. Under the same experimental environment and data conditions, traditional UNet exhibits problems such as blurred segmentation boundaries and high target recognition error rate when processing remote sensing images with complex terrain and textures. Multi-temporal channel UNet has fewer such problems, but suffers from long recognition time and insufficient utilization of temporal features. The single LSTM structure lacks spatial processing capabilities, requiring more computation to be allocated to spatial feature preservation, resulting in the longest recognition time. In contrast, the proposed method, with its unique ConvLSTM module and attention mechanism, can more accurately capture the spatiotemporal feature information of the target and effectively distinguish the target from the background. It significantly outperforms traditional segmentation networks in both segmentation accuracy and recognition accuracy, highlighting the advantages and practicality of the proposed method.

[0050] Experimental results (as shown in Table 1) demonstrate that the method of this invention achieves superior performance in the paddy field target segmentation and recognition task in this experimental area. The segmentation accuracy is significantly improved, with an overall accuracy of 97.00%, which is 6.54% higher than UNet. The recall rate also reaches 97.06%, effectively reducing misclassification and missed classification. Furthermore, the mIoU and user accuracy are improved by 7.50% and 6.75% respectively compared to the UNet model, enabling more accurate identification of paddy field areas. This fully verifies the effectiveness and practicality of the method of this invention.

[0051] Table 1 Comparison of Model Performance

[0052]

[0053] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for extracting paddy field planting area based on LSTM-Attention-UNet, characterized in that, Includes the following steps: Step S1: Acquire multi-temporal remote sensing images covering key growth stages of rice, including the first remote sensing image of the irrigation stage, the second remote sensing image of the tillering stage, the third remote sensing image of the grain-filling stage, and the fourth remote sensing image of the maturity stage, and construct a time-series image sequence. Step S2: Preprocess the first, second, third and fourth remote sensing images to obtain preprocessed first, second, third and fourth remote sensing images, and convert the paddy field vector data into raster label data that is spatially aligned with the remote sensing images; Step S3: Construct a time-series remote sensing dataset from the preprocessed first, second, third, and fourth remote sensing images and paddy field label data, which will be used as input for model training. Step S4: Input the time-series image sequences in the time-series remote sensing dataset into a convolutional long short-term memory network (ConvLSTM) for recursive time-series feature modeling. Update the spatial features of each time-phase image through a gating structure to generate a time-series feature map containing historical state information. Step S5: Input the temporal feature map into the temporal attention mechanism module, assign weight coefficients to each temporal feature frame in the time dimension, generate each temporal weight through the normalization function, and perform weighted fusion of multi-temporal features to obtain temporal aggregated features. Step S6: Input the temporal aggregated features into a U-shaped convolutional neural network (UNet) for pixel-level segmentation and output the paddy field segmentation result.

2. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 1, characterized in that: The multi-temporal remote sensing images in step S1 include Sentinel-2 and Landsat series images, with a unified spatial resolution of 10m and a projection coordinate system of EPSG:4490.

3. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 2, characterized in that: The preprocessing in step S2 includes radiometric calibration, band selection, reflectance normalization correction, multi-source sensor spectral consistency matching, cloud and snow shadow masking removal, and image median synthesis.

4. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 3, characterized in that: The raster label data in step S2 is generated by converting area to raster, with paddy fields marked as 1 and non-paddy fields marked as 0.

5. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 4, characterized in that: The temporal feature map output by ConvLSTM in step S4 serves as the sole input to the temporal attention mechanism module in step S5.

6. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 5, characterized in that: In step S5, the temporal attention mechanism is based on global average pooling and 1×1 convolution, and learns to generate normalized weight coefficients for each temporal feature frame.

7. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 6, characterized in that: The UNet in step S6 includes an encoder and a decoder structure. The encoder extracts multi-scale features by downsampling layer by layer, and the decoder restores spatial resolution by upsampling layer by layer and combining skip connections.

8. The method for extracting paddy field planting area based on LSTM-Attention-UNet according to claim 7, characterized in that: The UNet employs a learnable hybrid activation function, dynamically balancing the combination of ReLU and Tanh-Softplus through learnable weight parameters.

Images