A method for extracting coconut trees based on SAR remote sensing images and optical remote sensing images
By combining SAR and optical remote sensing imagery with a deep learning network, the problems of low efficiency and insufficient accuracy in coconut tree monitoring have been solved, achieving efficient and accurate spatial distribution monitoring of coconut trees and improving the level of intelligent agricultural management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 海南省航天技术创新中心
- Filing Date
- 2026-03-03
- Publication Date
- 2026-06-02
Smart Images
Figure CN122135212A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of tropical vegetation remote sensing technology, and specifically relates to a method for extracting coconut trees based on SAR remote sensing data and optical remote sensing data. Background Technology
[0002] Coconut palms are one of the most important economic crops in tropical regions, and the coconut industry has developed into a representative pillar industry of agriculture in these areas, playing a vital role in promoting regional economic development, ensuring agricultural production, and protecting the ecological environment. Therefore, accurate monitoring of coconut palm cultivation in tropical regions, and obtaining information on the spatial distribution and planting area of coconut palms, has significant practical importance and application value.
[0003] Existing methods for monitoring coconut tree planting area mainly rely on manual surveys or field statistics, which suffer from low efficiency, long time cycles, high labor costs, and difficulty in achieving large-scale, continuous monitoring. These methods fail to meet the needs of modern agricultural precision management and intelligent monitoring. Therefore, there is an urgent need for a technological means to efficiently and accurately obtain information on the spatial distribution of coconut trees.
[0004] With the development of remote sensing technology and deep learning methods, crop extraction methods based on remote sensing images have been gradually applied. However, existing deep learning-based remote sensing image extraction methods mostly rely on optical remote sensing images as the primary data source. Under tropical monsoon climate conditions, with high cloud cover and frequent rainfall, optical remote sensing images are easily affected by factors such as cloud and fog obstruction and data gaps, limiting further improvements in the accuracy of coconut tree extraction and the reliability of mapping.
[0005] Current methods for monitoring coconut trees primarily rely on manual surveys and field statistics. This approach is not only inefficient, time-consuming, and labor-intensive, but also extremely difficult to achieve large-scale continuous monitoring, failing to meet the demands of modern, precision agricultural management. Furthermore, in tropical monsoon climate zones characterized by severe cloud cover and frequent rainfall, existing deep learning extraction methods overly depend on a single optical remote sensing image data source, making them susceptible to data loss or unstable image quality due to weather interference. This severely limits the accuracy of coconut tree extraction and the reliability of mapping. Summary of the Invention
[0006] To address the above problems, the present invention aims to provide a method for extracting coconut trees based on SAR remote sensing images and optical remote sensing images; The technical solution adopted in this invention: A method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery includes the following steps: S1: Collect SAR remote sensing image data and optical remote sensing image data of the same area, preprocess them, construct data pairs, and build a dataset; S2: Construct a deep learning network model; the deep learning network model includes a shallow semantic extraction module, a spatial attention gating fusion module, a deep semantic extraction module, a multi-scale contextual feature extraction module, and a decoder module; S21: Input the preprocessed SAR remote sensing image data and optical remote sensing image data into the shallow semantic extraction module to obtain SAR shallow features and optical shallow features; the shallow semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer and a downsampling layer. S22: Input SAR shallow features and optical shallow features into the spatial attention-gated fusion module to obtain shallow fused features, which are used in the decoder module; the spatial attention-gated fusion module includes a stitching layer, a convolutional layer, a global descriptor extraction layer, and an attention weight generation layer; the global descriptor extraction layer includes global average pooling and global max pooling; the attention weight generation layer includes a multilayer perceptron and a Softmax normalized weight layer; S23: Input the SAR shallow features and optical shallow features into the deep semantic extraction module to obtain the SAR high-level features and optical high-level features; the deep semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer and a ReLU activation layer; S24: Input SAR high-level features and optical high-level features into the multi-scale context feature extraction module to obtain high-level fused features; S25: Upsample the high-level fusion features to obtain the sampled high-level fusion features, and then concatenate the sampled high-level fusion features and shallow fusion features to obtain the final fusion features. S26: Input the final fused features into the decoder module to obtain the extraction result; the decoder module includes two two-dimensional convolutional layers and upsampling; S3: Train the deep learning network model by inputting SAR remote sensing image data and optical remote sensing image data into the model, calculating the total loss value of the model, performing backpropagation, optimizing the weight parameters of the model through a predetermined optimizer, and obtaining the optimized deep learning network model after multiple rounds of training.
[0007] S4: Input SAR remote sensing image data and optical remote sensing image data into the optimized deep learning network model to obtain accurate extraction results; Preferably, in step S1, SAR remote sensing image data and optical remote sensing image data are collected and preprocessed, and the specific process is as follows: Based on optical remote sensing image data, coordinate system I, image registration, and resampling processing are performed on SAR remote sensing image data. Masking operations are used to ensure that the two types of images are consistent in spatial range and resolution. The processed SAR remote sensing image data and optical remote sensing image data are cropped and sliced, and the sliced samples are labeled based on the optical remote sensing image slices.
[0008] Preferably, in step S21, the preprocessed SAR remote sensing image data and optical remote sensing image data are input into the shallow semantic extraction module to obtain SAR shallow features and optical shallow features. The specific process is as follows: Preprocessed SAR and optical remote sensing image data are input into a two-dimensional convolutional layer to extract edge features. The edge features are then input into a batch normalization layer to obtain a stable distribution of edge features. The stable distribution of edge features is then input into a ReLU activation layer to introduce nonlinearity, and finally input into a downsampling layer to obtain SAR shallow features and optical shallow features.
[0009] Preferably, in step S22, the SAR shallow features and optical shallow features are input into the spatial attention-gated fusion module to obtain shallow fused features. The specific process is as follows: The shallow SAR features and the shallow optical features are input into the stitching layer channel to stitch together to obtain stitched features. The stitched features are then input into the convolutional layer for channel compression to obtain a joint feature map. The joint feature map is then input into the global descriptor extraction layer to obtain the global descriptor. The global descriptor is then input into the attention weight generation layer to obtain adaptive weight coefficients. Finally, the adaptive weight coefficients are input into the convolutional layer for weighted fusion to obtain shallow fused features.
[0010] Preferably, in step S23, the SAR shallow features and optical shallow features are input into the deep semantic extraction module to obtain SAR high-level features and optical high-level features. The specific process is as follows: The SAR shallow features and optical shallow features are respectively input into a two-dimensional convolutional layer to obtain high-level semantic features. The high-level semantic features are then input into a batch normalization layer for stable distribution, and then input into a ReLU activation layer to obtain SAR high-level features and optical high-level features.
[0011] Preferably, in step S26, the final fused features are input into the decoder module to obtain the extraction result, and the specific process is as follows: The final fused features are input into a two-dimensional convolutional layer for feature dimensionality reduction to obtain dimensionality-reduced fused features. The fused features are then input into the next two-dimensional convolutional layer for mapping to the classification channel, and then upsampling is performed to obtain the extraction result.
[0012] The beneficial effects of this invention are: (1) This invention aims to solve the technical challenge of accurately monitoring the spatial distribution of coconut trees under complex meteorological environmental interference. By fusing SAR remote sensing data with high-resolution optical images, it overcomes the limitations of a single data source affected by clouds and fog. This invention is committed to constructing a deep learning network with a dual-branch encoder. By introducing a spatial attention-gated fusion module (SAGate) to adaptively weight multi-source features and utilizing a multi-scale context extraction module (ASPP) to enhance the modeling ability of coconut tree regions at different scales, it achieves efficient and high-precision pixel-level automatic extraction of coconut trees, providing reliable data support for the tropical pillar industry.
[0013] (2) The SAGate gated fusion module is integrated to solve the problem of classification ambiguity caused by the "inefficient fusion" of heterogeneous features. The gated fusion module can dynamically adjust the contribution of the two types of data according to the features of the land cover, strengthen the key features of the coconut tree, suppress background noise, and thus significantly improve the pixel-level classification accuracy.
[0014] (3) Integrating ASPP multi-scale feature extraction to solve the problem of missed detection and false detection caused by "different scales" in the planting area. Coconut tree planting areas are often of different sizes and have complex edge distribution.
[0015] This invention introduces a multi-scale contextual feature extraction module at the high-level encoding stage. By using multiple dilated convolutions with different dilation rates in parallel, the model can expand its receptive field and capture multi-scale features without increasing computational cost.
[0016] (4) Improved the level of intelligence and automation in agricultural management. Compared with the traditional model that relies on manual surveys in the background technology, this invention achieves pixel-level extraction of coconut trees through a deep learning model. This not only greatly improves the efficiency of operations and reduces the cost of human input, but also provides a scientific and efficient technical means for obtaining the accurate spatial distribution and planting area of coconut trees throughout Hainan Island, which has extremely high practical application value. Attached Figure Description
[0017] Figure 1 This is a flowchart of a method for extracting coconut trees based on SAR remote sensing images and optical remote sensing images according to the present invention.
[0018] Figure 2 This invention discloses a deep learning network model for extracting coconut trees based on SAR and optical remote sensing images. (Network structure diagram follows.) Figure 3 This is a structural diagram of each module in the deep learning network model of this invention.
[0019] Figure 4 The structural diagram of the spatial attention gating fusion module of this invention.
[0020] Figure 5The structural diagram of the multi-scale context feature extraction module of this invention. Detailed Implementation
[0021] The following detailed description of the implementation of the present invention is in conjunction with the accompanying drawings. However, these descriptions do not constitute a limitation of the present invention and are merely examples. Through these descriptions, the advantages of the present invention will be more clearly understood. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in the present invention should be considered to be within the scope of protection of the present invention. Other parts not described in detail in the embodiments are all prior art. like Figure 1 As shown, the present invention proposes a method for extracting coconut trees based on SAR remote sensing images and optical remote sensing images, comprising the following steps: S1: Collect SAR remote sensing image data and optical remote sensing image data of the same area, preprocess them, construct data pairs, and build a dataset; Using a coconut grove in the eastern suburbs of Wenchang City, Hainan Province as the experimental area, SAR and optical remote sensing image data covering the same area were acquired. The SAR image data was used to provide information on surface structure and scattering characteristics, while the optical image data was used to provide spectral and texture information of ground features. In this embodiment, the SAR and optical remote sensing image data were acquired at approximately the same time to minimize the impact of surface changes on subsequent processing results.
[0022] Preprocessing operations are performed on the acquired SAR and optical remote sensing image data. Specifically, using the optical remote sensing image data as a reference image, coordinate system unification and georegistration are performed on the SAR remote sensing image data to spatially align the two types of imagery within the same geographic coordinate system. Subsequently, the SAR remote sensing image data is resampled to ensure its spatial resolution matches that of the optical remote sensing image data; and a spatial masking operation is used to crop out the common coverage area of the two types of imagery, thereby guaranteeing consistency in spatial extent and resolution between the SAR and optical remote sensing image data.
[0023] The SAR and optical remote sensing image data were cropped and segmented, dividing the original large-scale image into multiple image tiles of the same size. Based on the optical remote sensing image tiles, corresponding areas were labeled to construct a sample dataset for coconut tree extraction. A deep learning coconut tree extraction network model was constructed based on the sample dataset. The coconut tree extraction model was obtained through training, and the model with the best performance was selected based on the indicators. The coconut tree extraction model is used to infer the remote sensing image to be processed, and the pixel-level classification results of the coconut trees are output, thereby obtaining the distribution area of the coconut trees.
[0024] Optionally, the SAR remote sensing image data may be dual-polarization SAR image data acquired by the HaiShao-1 satellite, with polarization modes including VV polarization and VH polarization; the optical remote sensing image data may be sub-meter resolution optical remote sensing images.
[0025] Optionally, by setting the size of the image slices and performing sliding window slicing on the processed optical and SAR images, the following can be achieved: Indicates the number of rows in the image slice. Indicates the number of columns in the image slice. Indicates the image length. Indicates the image width. Indicates the pixel length of the image slice. Indicates the pixel width of the image slice. Indicates the size of the overlapping pixels between adjacent slices. If we indicate that adjacent slices overlap by a certain number of pixels, then the image slicing formula is as follows:
[0026] In practical use, due to the large range of SAR remote sensing data and optical remote sensing data, direct input would place too high a demand on computing power. To facilitate model network training, in actual operation, the images are cropped to a size of 256*256, and the images are labeled and trained based on this size. S2: Constructing deep learning network models, such as Figure 2 As shown; the deep learning network model includes a shallow semantic extraction module, a spatial attention gating fusion module, a deep semantic extraction module, a multi-scale contextual feature extraction module, and a decoder module; S21: Input the preprocessed SAR remote sensing image data and optical remote sensing image data into the shallow semantic extraction module to obtain SAR shallow features and optical shallow features; the shallow semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, and a downsampling layer, such as... Figure 3 As shown; The process involves inputting preprocessed SAR and optical remote sensing image data into a two-dimensional convolutional layer to extract edge features. The edge features are then input into a batch normalization layer to achieve a stable distribution, which in turn is input into a ReLU activation layer to introduce nonlinearity. Finally, the stable edge features are input into a downsampling layer. This process is repeated three times to obtain SAR shallow features and optical shallow features.
[0027] S22: Input SAR shallow features and optical shallow features into the spatial attention-gated fusion module to obtain shallow fused features, which are then used in the decoder module; the spatial attention-gated fusion module is as follows: Figure 4As shown, it includes a splicing layer, a convolutional layer, a global descriptor extraction layer, and an attention weight generation layer; the global descriptor extraction layer includes global average pooling and global max pooling; the attention weight generation layer includes a multilayer perceptron and a Softmax normalized weight layer; During the encoding process, for shallow features, the deep learning network model introduces a spatial attention-gated fusion module (SAGate). This module dynamically adjusts the fusion weights of SAR and optical features through adaptive learning. First, the two feature streams are concatenated and fused through convolution. Then, global average pooling and global max pooling are used to extract statistical features. After passing through a bottleneck structure, a dual-channel weight vector is generated. The Softmax function is used to normalize the weights in the channel dimension, achieving precise weighting of SAR and low-level optical features. Optionally, SAGate is a spatial attention-gated fusion module for shallow SAR features of the same scale. and optical shallow features The calculation steps for fusion include: (1) Joint feature mapping: for shallow SAR features of the same scale and optical shallow features Channel splicing is performed using a convolutional layer with a kernel size of 1*1. The joint feature map U is obtained using the following formula:
[0028] (2) Gated vector extraction: Global average pooling (GAP) and global max pooling (GMP) are used to statistically converge the joint feature map U to obtain the global descriptor s. The calculation formula of s is as follows:
[0029] (3) Attention weight allocation: The global description operator s is input into the multilayer perceptron network (MLP), and the adaptive weight coefficients for SAR and optical branches are obtained by Softmax normalization. and :
[0030] (4) Feature reconstruction: The original branches are weighted and mapped using weight coefficients to output the fused feature V, as shown in the following formula:
[0031] S23: Input the SAR shallow layer features and optical shallow layer features into the deep semantic extraction module to obtain the SAR high layer features and optical high layer features; the deep semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer and a ReLU activation layer; The SAR shallow features and optical shallow features are respectively input into a two-dimensional convolutional layer to obtain high-level edge features. Then, the high-level edge features are input into a batch normalization layer for stable distribution to obtain high-level stable edge features. Then, they are input into a ReLU activation layer. The whole process is repeated twice to obtain SAR high-level features and optical high-level features.
[0032] S24: Input SAR high-level features and optical high-level features into the multi-scale context feature extraction module to obtain high-level fused features, such as... Figure 5 As shown; The deep learning network model introduces a multi-scale contextual feature extraction module (ASPP). First, it performs channel compression on the high-level features output from the optical and SAR branches respectively, and then concatenates the features along the channel dimension to form a joint high-level feature representation. This joint high-level feature is then input into the multi-scale contextual feature extraction module, which uses multi-scale dilated convolution operations to obtain contextual information under different receptive fields, thereby enhancing the modeling capability of coconut tree regions at different scales.
[0033] Optionally, the ASPP module captures information from different scales by using multiple convolutional kernels with different dilation rates (dilated convolutions) in parallel. Different dilation rate kernels capture different receptive fields, thus extracting multi-scale features. Specifically, dilated convolutions expand the receptive field of the convolution by inserting holes between elements in the standard convolutional kernel without increasing computation or reducing the resolution of the feature map. The structure of the ASPP module is shown below: (1) Multiple dilated convolutions: Convolution operations are performed using different dilation rates (e.g., 1, 6, 12, 18) to capture features at different scales.
[0034] (2) Global average pooling: Pooling is performed on the entire input to obtain global context information. This part of the pooling is then followed by a 1x1 convolution for dimension matching.
[0035] (3) Feature fusion: All these features at different scales are merged in a concatenated manner to form a feature map containing rich multi-scale information. In this way, the ASPP module can effectively capture features at different scales while avoiding the reduction of image resolution; Optionally, the accuracy evaluation indicators include OA, MIoU, and Kappa. Let TN represent the number of pixels correctly classified as background features, TP represent the number of pixels correctly classified as target features, FN represent the number of pixels misclassified as background features, and FP represent the number of pixels misclassified as target features. The formula for the accuracy evaluation index is as follows: The quotient of the sum of the products of the actual and predicted quantities of all land features and the square of the total sample size.
[0036]
[0037]
[0038] Compared to optical imagery, SAR remote sensing imagery data has the advantages of being available all day and all weather, providing high-resolution, large-scale surface information, and supplementing invalid pixels in optical images caused by interference such as clouds and fog. Therefore, this invention can fully utilize multi-source remote sensing imagery information, achieving efficient and high-precision mapping during the coconut tree extraction process by combining SAR remote sensing imagery data with optical imagery data. S25: Upsample the high-level fusion features to obtain the sampled high-level fusion features, and then concatenate the sampled high-level fusion features and shallow fusion features to obtain the final fusion features. S26: Input the final fused features into the decoder module to obtain the extraction result; the decoder module includes two two-dimensional convolutional layers and upsampling; In the decoder section, the model does not directly predict high-level features. Instead, it upsamples the ASPP-enhanced high-level semantic features through bilinear interpolation and concatenates them with the SAGate-processed low-level spatial features to form composite features. This design ensures that the model can obtain both global information and accurately capture detailed information. Finally, it maps to the class space through consecutive 3x3 and 1x1 convolutions. S3: Train the deep learning network model by inputting SAR remote sensing image data and optical remote sensing image data into the model, calculating the total loss value of the model, performing backpropagation, optimizing the weight parameters of the model through a predetermined optimizer, and obtaining the optimized deep learning network model after multiple rounds of training.
[0039] S4: Input SAR remote sensing image data and optical remote sensing image data into the optimized deep learning network model to obtain accurate extraction results.
[0040] Although preferred embodiments of this application have been described, those skilled in the art, once they learn the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. If such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery, characterized in that, include: S1: Collect SAR remote sensing image data and optical remote sensing image data of the same area, preprocess them, construct data pairs, and build a dataset; S2: Construct a deep learning network model; the deep learning network model includes a shallow semantic extraction module, a spatial attention gating fusion module, a deep semantic extraction module, a multi-scale contextual feature extraction module, and a decoder module; S21: Input the preprocessed SAR remote sensing image data and optical remote sensing image data into the shallow semantic extraction module to obtain SAR shallow features and optical shallow features; the shallow semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer and a downsampling layer. S22: Input SAR shallow features and optical shallow features into the spatial attention-gated fusion module to obtain shallow fused features; the spatial attention-gated fusion module includes a stitching layer, a convolutional layer, a global descriptor extraction layer, and an attention weight generation layer; the global descriptor extraction layer includes global average pooling and global max pooling; the attention weight generation layer includes a multilayer perceptron and a Softmax normalized weight layer. S23: Input the SAR shallow features and optical shallow features into the deep semantic extraction module to obtain the SAR high-level features and optical high-level features; the deep semantic extraction module includes a two-dimensional convolutional layer, a batch normalization layer and a ReLU activation layer; S24: Input SAR high-level features and optical high-level features into the multi-scale context feature extraction module to obtain high-level fused features; S25: Upsample the high-level fusion features to obtain the sampled high-level fusion features, and then concatenate the sampled high-level fusion features and shallow fusion features to obtain the final fusion features. S26: Input the final fused features into the decoder module to obtain the extraction result; the decoder module includes two two-dimensional convolutional layers and upsampling; S3: Train the deep learning network model by inputting SAR remote sensing image data and optical remote sensing image data into the model, calculating the total loss value of the model, performing backpropagation, optimizing the weight parameters of the model through a predetermined optimizer, and obtaining an optimized deep learning network model after multiple rounds of training. S4: Input SAR remote sensing image data and optical remote sensing image data into an optimized deep learning network model to obtain accurate extraction results.
2. The method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery according to claim 1, characterized in that, In step S1, SAR remote sensing image data and optical remote sensing image data are collected and preprocessed. The specific process is as follows: Based on optical remote sensing image data, coordinate system I, image registration, and resampling processing are performed on SAR remote sensing image data. Masking operations are used to ensure that the two types of images are consistent in spatial range and resolution. The processed SAR remote sensing image data and optical remote sensing image data are cropped and sliced, and the sliced samples are labeled based on the optical remote sensing image slices.
3. The method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery according to claim 1, characterized in that, In step S21, SAR shallow-layer features and optical shallow-layer features are obtained. The specific process is as follows: Preprocessed SAR and optical remote sensing image data are input into a two-dimensional convolutional layer to extract edge features. The edge features are then input into a batch normalization layer to obtain a stable distribution of edge features. The stable distribution of edge features is then input into a ReLU activation layer to introduce nonlinearity, and finally input into a downsampling layer to obtain SAR shallow features and optical shallow features.
4. The method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery according to claim 1, characterized in that, The shallow fusion features are obtained in S22, and the specific process is as follows: The shallow SAR features and the shallow optical features are input into the stitching layer channel to stitch together to obtain stitched features. The stitched features are then input into the convolutional layer for channel compression to obtain a joint feature map. The joint feature map is then input into the global descriptor extraction layer to obtain the global descriptor. The global descriptor is then input into the attention weight generation layer to obtain adaptive weight coefficients. Finally, the adaptive weight coefficients are input into the convolutional layer for weighted fusion to obtain the shallow fused features.
5. The method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery according to claim 1, characterized in that, In step S23, SAR high-level features and optical high-level features are obtained. The specific process is as follows: The SAR shallow features and optical shallow features are respectively input into a two-dimensional convolutional layer to obtain high-level semantic features. The high-level semantic features are then input into a batch normalization layer for stable distribution, and then input into a ReLU activation layer to obtain SAR high-level features and optical high-level features.
6. The method for extracting coconut trees based on SAR remote sensing imagery and optical remote sensing imagery according to claim 1, characterized in that, In step S26, the final fused features are input into the decoder module to obtain the extraction result. The specific process is as follows: The final fused features are input into a two-dimensional convolutional layer for feature dimensionality reduction to obtain dimensionality-reduced fused features. The fused features are then input into the next two-dimensional convolutional layer for mapping to the classification channel, and then upsampling is performed to obtain the extraction result.