Rice lodging region segmentation method based on multi-feature data fusion

Abstract

The invention discloses a rice lodging region segmentation method based on multi-feature data fusion in the technical field of agricultural remote sensing and deep learning, and the method comprises the following steps: 1, building a rice lodging data set, and exporting an image file in a TIFF format through a large Xinjiang multispectral unmanned plane; the method comprises the following steps of: marking an image file, and effectively converting marked information into raster data through conversion from a vector to a raster in an ArcGIS (ArcGeographic Information System); and finally, performing grid cutting and data enhancement according to the range of the original TIFF image to obtain tag data. 2, constructing a rice lodging segmentation model based on multi-feature data fusion; the objective of the invention is to solve the problems that lodging regions are irregular in shape and large in scale change in rice lodging image segmentation, and lodging rice and healthy rice are difficult to distinguish due to similar spectral features.

Description

Technical Field

[0001] This invention relates to a method for segmenting lodging areas of rice based on multi-feature data fusion in the field of agricultural remote sensing and deep learning technology. Background Technology

[0002] Rice lodging significantly disrupts photosynthesis and grain-filling processes, hindering biomass accumulation and consequently drastically reducing rice yield and quality. Lodging not only causes direct economic losses but also significantly increases the difficulty and cost of mechanized harvesting. Current rice lodging monitoring methods generally rely on traditional manual statistical analysis or deep learning networks, focusing primarily on superficial features such as color or texture. They pay less attention to the complex and varied spatial scale and irregular boundaries of lodging areas, and fail to adequately consider the spectral confusion between lodged and normal rice, resulting in low segmentation accuracy and blurred edges against complex field backgrounds.

[0003] Therefore, the current research focus is on how to quickly and accurately extract the features of lodged areas, propose a high-precision segmentation method for lodged rice areas suitable for complex farmland environments, and solve the problem of segmenting multi-scale and irregular lodged areas. Summary of the Invention

[0004] The purpose of this invention is to provide a rice lodging region segmentation method based on multi-feature data fusion, which aims to solve the problems of irregular lodging region shape, large scale variation, and difficulty in distinguishing lodged rice from healthy rice due to similar spectral characteristics in rice lodging image segmentation.

[0005] To achieve the above objectives, this invention provides a method for segmenting lodging areas in rice fields based on multi-feature data fusion, comprising the following steps:

[0006] Step 1: Create a rice lodging dataset by exporting it as a TIFF image file using a DJI multispectral drone; label the image file and convert the labeled information into raster data in ArcGIS using vector-to-raster conversion; finally, crop the raster and perform data augmentation based on the extent of the original TIFF image to obtain the label data.

[0007] Step 2: Construct a rice lodging segmentation model based on multi-feature data fusion;

[0008] Step 3: Input the multi-feature dataset preprocessed in Step 1 into the model constructed in Step 2 for training, optimize the network parameters, and finally generate a segmentation model for rice lodging areas.

[0009] Step 4: Using the segmentation model trained in Step 3, input the rice TIFF format file to be segmented, detect whether lodging exists, and segment out the lodged areas of the rice.

[0010] As a further improvement to the present invention, the specific content of step 1 is as follows.

[0011] Step 1.1: Use a DJI multispectral drone to collect field images of rice at the ripening stage, including four bands: green light, red light, red edge, and near-infrared. Use ArcGIS software to crop out the lodged rice area, create a vector file in the TIFF image coordinate system to label the target area, and add attribute fields. In the attribute fields, set the lodged rice to 255 to represent the lodged area, and generate the corresponding label map through vector-to-raster technology.

[0012] Step 1.2: Using the sliding window algorithm, the window size is set to 256×256 and the step size is 256. The large multi-channel image and its corresponding label image are cropped into a series of small image blocks. During the cropping process, image blocks containing more than a threshold of invalid data are automatically removed to ensure the effectiveness of the training data.

[0013] Step 1.3: After the data processing is completed, the image patches are augmented by rotation, mirroring, flipping, and contrast adjustment to expand the dataset and improve the model's generalization ability.

[0014] As a further improvement of the present invention, the segmentation model in step 2 adopts an improved UNet++ network architecture, which includes a feature extraction encoder, dense skip connections, and a feature reconstruction decoder. The encoder path uses ResNet101 as the backbone network and cascades a spatial-spectral dual attention mechanism to extract multi-scale features step by step. The encoder end is connected to a hollow spatial pyramid pooling module to capture global context information. The decoder path introduces nested skip connections of a temporal feature fusion module to adaptively fuse and reconstruct the deep semantic features output by the encoder and the upsampled features generated by the decoder.

[0015] As a further improvement to the present invention, the specific content of step 3 is as follows.

[0016] Step 3.1: ResNet101 is used as the feature extraction network. Its original first convolutional layer is removed and replaced with a 7×7 convolutional layer adapted for multi-channel input. An image of size H*W*5 is input into the segmentation model. A percentage-truncated linear stretching strategy is used for normalization. The pixel value of each band is calculated separately, and the pixel values ​​within this range are linearly mapped to the [0, 255] interval to eliminate extreme lighting noise and enhance image contrast. Based on this, an additional vegetation index channel is constructed using band operations to calculate the Normalized Differential Red Edge Vegetation Index (NDRE). The calculation formula is as follows:

[0017] ;

[0018] In the formula, NIR represents the multispectral near-infrared band, and RE represents the multispectral red-edge band;

[0019] The calculated NDRE single-band image is stacked with the original G, R, RedEdge and NIR bands in terms of channel dimension to construct 5-channel input data containing spectral information and vegetation status information; the image is processed by the pre-convolutional layer of ResNet101 and the initial residual module for feature extraction, and the output is a high-dimensional feature map with size H / 4*W / 4*64.

[0020] Step 3.2 introduces a spatial-spectral dual attention mechanism SS-DAM after each feature level of the ResNet101 encoder. This mechanism is composed of a large selectable kernel LSK module and an efficient multi-scale attention module EMA connected in series, which enhances spatial structure features and spectral features respectively.

[0021] Step 3.3: The LSK module adaptively adjusts the receptive field through large kernel convolution sequence decomposition and spatial selection mechanism;

[0022] Step 3.4: EMA recalibrates channel weights through grouped multi-scale processing;

[0023] Step 3.5: At the deepest output layer of the encoder, a Spatial Pyramid Pooling (ASPP) module is introduced to perform multi-scale feature fusion. The specific process is shown in the following equation.

[0024] ;

[0025] in, express The output of the convolution branch, , and These represent expansion ratios of 6, 12, and 18, respectively. The output of the dilated convolution branch, This indicates the output of the global average pooling branch; Cat indicates channel concatenation. This represents the output features that incorporate multi-scale global contextual information;

[0026] Step 3.6: The UNet++ architecture is used as the decoder to gradually restore the spatial resolution of the feature map through nested dense skip connection paths; the decoder node receives upsampled features from the same level and skip connection features from the encoder or previous node.

[0027] Step 3.7: In the feature fusion stage of the decoder, the Temporal Feature Fusion (TFF) module is used to replace the traditional channel splicing operation, fusing features from the encoder. Features from the decoder The difference between the two features is calculated; the feature differences are extracted using depthwise separable convolution, and adaptive fusion weights are generated.

[0028] Step 3.8: Skip connections are made between the downsampled output features of the same layer in U-Net++ and the upsampled features of the current layer, and then they are fused. At the end of the decoder, bilinear interpolation combined with convolution operations is used to restore the feature map to the same spatial resolution as the original input image. The feature channels are mapped to the number of categories through the final 1×1 convolutional layer, and the pixel-level segmentation result is output. The output map and the input map are spatially aligned, achieving accurate segmentation of the target region in the input image.

[0029] As a further improvement to the present invention, the specific content of step 3.3 is as follows:

[0030] First, multi-scale features are extracted through large-kernel convolution with sequence decoupling:

[0031] ;

[0032] ;

[0033] in, As input features, These are local features obtained through 5×5 convolution. In order to be in The receptive field features are obtained through a 7×7 dilated convolution with a dilation rate of d=3. Subsequently, the two feature streams are dimensionality-reduced and concatenated, and selection weights are calculated using a spatial selection mechanism.

[0034] ;

[0035] ;

[0036] Finally, the generated spatial weight map (SA) is used to weight and fuse features at different scales, and then applied to the input features. The spatially augmented output features are obtained. ;

[0037] ;

[0038] in This indicates element-wise multiplication. , These are the two parts of the weighted graph after being segmented along the channel. The output features are spatially augmented. It enhanced the spatial structure information of the collapsed area.

[0039] As a further improvement to the present invention, the specific content of step 3.4 is as follows:

[0040] Output features Input to the EMA module to enhance spectral channel features, and output features Divide the data into G groups along the channel dimension, and extract features through 1×1 and 3×3 branches respectively. Aggregate global information through cross-space learning

[0041] ;

[0042] ;

[0043] in, and They represent the grouping features in Convolutional branches and The output of the convolutional branch, M, represents the intermediate feature map that aggregates global context information through cross-space learning, and W represents the final generated channel attention weights;

[0044] Recalibrating features using weighted graphs:

[0045] ;

[0046] in, This represents the input features after grouping. Reshape indicates a weighted operation, and Reshape indicates restoring the features to their original dimensions. This indicates the output characteristics after recalibration of the spectral channel features.

[0047] As a further improvement to the present invention, the specific content of step 3.6 is as follows:

[0048] By utilizing the layer-by-layer convolution downsampling operation of ResNet101, the spatial resolution is progressively reduced. The input image size is 256*256*5. The image first passes through Layer 0, which includes a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer with a stride of 2. Two consecutive downsampling operations reduce the feature map size from 256×256 to 64×64, expanding the number of channels to 64, generating the first-level feature map. Subsequently, the feature map passes through four residual stages of ResNet101. After each stage, the output is enhanced by the LSK module. The evolution of the feature dimension is as follows: 64*64*256 → 32*32*512 → 16*16*1024 → 8*8*2048. After each downsampling, the number of channels increases with the network depth to enhance the abstract expression of semantic information. This process forms a multi-scale, high-dimensional feature representation, which serves as the basis for skip connections and feature fusion in the subsequent UNet++ decoding path.

[0049] As a further improvement of the present invention, in step 3.7, the difference information of the two features... ;

[0050] in, This represents the deep semantic features from the encoder. This represents the upsampled features from the decoder. Represent the difference between the two features; generate adaptive fusion weights:

[0051] ;

[0052] ;

[0053] Where DSConv represents depthwise separable convolution, and Cat represents channel concatenation. and They represent the features respectively and Generate adaptive fusion weights; output final fusion features:

[0054] .

[0055] As a further improvement to the present invention, the specific content of step 4 is as follows.

[0056] The AdamW optimizer was used, with an initial learning rate set to Weight decay is set to The learning rate is dynamically adjusted using a cosine annealing strategy, and a joint loss function is employed.

[0057] ;

[0058] ;

[0059] ;

[0060] in, This represents the Dice loss function, used to address the imbalance between positive and negative samples. represents the soft cross-entropy loss function, used to smooth the classification probability distribution and prevent the model from overfitting; N represents the total number of pixels in the image; This represents the true label of the i-th pixel. This indicates the probability that the model predicts the pixel to be upside down; To prevent smooth terms with a denominator of zero; This represents the ground truth label after label smoothing, which reduces the interference of noisy labels on model training by introducing a small smoothing factor.

[0061] The trained segmentation model is read, and the original TIFF image to be segmented is input into the segmentation model to detect whether lodging exists and segment out the lodged rice area.

[0062] Compared with existing technologies, the advantages of this invention lie in constructing a segmentation model based on multi-feature fusion for rice lodging segmentation tasks. By introducing the LSK attention mechanism, utilizing its large kernel and dilated convolution characteristics, it can effectively adapt to the characteristics of varied morphology and irregular boundaries of lodging areas, expanding the effective receptive field. The EMA attention mechanism enhances the expression of spectral channel features through cross-dimensional interaction, helping to distinguish the subtle spectral differences between lodged and healthy rice. The ASPP module not only captures the global context but also alleviates the recognition difficulties caused by the varying scales of lodging areas. Finally, the TFF module optimizes the feature fusion process in skip connections, reducing information loss during the encoding and decoding stages. These improvements combined effectively enhance the segmentation accuracy and robustness of rice lodging areas in complex farmland backgrounds. Attached Figure Description

[0063] Figure 1 This is a flowchart of the present invention.

[0064] Figure 2 This is a structural diagram of the model of the present invention.

[0065] Figure 3 This is an aerial photograph of a scene taken by the drone according to the present invention.

[0066] Figure 4 This is a pixel-level segmentation image of lodging rice output by the present invention. Detailed Implementation

[0067] The present invention will be further described below with reference to the accompanying drawings:

[0068] like Figure 1-4 The method for segmenting lodging areas in rice fields based on multi-feature data fusion, as shown, includes the following steps:

[0069] Step 1: Create a rice lodging dataset by exporting it as a TIFF image file using a DJI multispectral drone; label the image file and convert the labeled information into raster data in ArcGIS using vector-to-raster conversion; finally, crop the raster and perform data augmentation based on the extent of the original TIFF image to obtain the label data.

[0070] Step 1.1: Use a DJI multispectral drone to collect field images of rice at the ripening stage, including four bands: green light, red light, red edge, and near-infrared. Use ArcGIS software to crop out the lodged rice area, create a vector file in the TIFF image coordinate system to label the target area, and add attribute fields. In the attribute fields, set the lodged rice to 255 to represent the lodged area, and generate the corresponding label map through vector-to-raster technology.

[0071] Step 1.2: Using the sliding window algorithm, the window size is set to 256×256 and the step size is 256. The large multi-channel image and its corresponding label image are cropped into a series of small image blocks. During the cropping process, image blocks containing more than a threshold of invalid data are automatically removed to ensure the effectiveness of the training data.

[0072] Step 1.3: After the data processing is completed, the image patches are augmented by rotation, mirroring, flipping, and contrast adjustment to expand the dataset and improve the model's generalization ability.

[0073] Step 2: Construct a rice lodging segmentation model based on multi-feature data fusion;

[0074] The segmentation model in step 2 adopts an improved UNet++ network architecture, which includes a feature extraction encoder, dense skip connections, and a feature reconstruction decoder. The encoder path uses ResNet101 as the backbone network and cascades a spatial-spectral dual attention mechanism to extract multi-scale features step by step. The encoder end is connected to a hollow spatial pyramid pooling module to capture global contextual information. The decoder path introduces nested skip connections of a temporal feature fusion module to adaptively fuse and reconstruct the deep semantic features output by the encoder and the upsampled features generated by the decoder.

[0075] Step 3: Input the multi-feature dataset preprocessed in Step 1 into the model constructed in Step 2 for training, optimize the network parameters, and finally generate a segmentation model for rice lodging areas.

[0076] Step 3.1: ResNet101 is used as the feature extraction network. Its original first convolutional layer is removed and replaced with a 7×7 convolutional layer adapted for multi-channel input. An image of size H*W*5 is input into the segmentation model. A percentage-truncated linear stretching strategy is used for normalization. The pixel value of each band is calculated separately, and the pixel values ​​within this range are linearly mapped to the [0, 255] interval to eliminate extreme lighting noise and enhance image contrast. Based on this, an additional vegetation index channel is constructed using band operations to calculate the Normalized Differential Red Edge Vegetation Index (NDRE). The calculation formula is as follows:

[0077] ;

[0078] In the formula, NIR represents the multispectral near-infrared band, and RE represents the multispectral red-edge band;

[0079] The calculated NDRE single-band image is stacked with the original G, R, RedEdge and NIR bands in terms of channel dimension to construct 5-channel input data containing spectral information and vegetation status information; the image is processed by the pre-convolutional layer of ResNet101 and the initial residual module for feature extraction, and the output is a high-dimensional feature map with size H / 4*W / 4*64.

[0080] Step 3.2 introduces a spatial-spectral dual attention mechanism SS-DAM after each feature level of the ResNet101 encoder. This mechanism is composed of a large selectable kernel LSK module and an efficient multi-scale attention module EMA connected in series, which enhances spatial structure features and spectral features respectively.

[0081] Step 3.3: The LSK module adaptively adjusts the receptive field through large kernel convolution sequence decomposition and spatial selection mechanism;

[0082] First, multi-scale features are extracted through large-kernel convolution with sequence decoupling:

[0083] ;

[0084] ;

[0085] in, As input features, These are local features obtained through 5×5 convolution. In order to be in The receptive field features are obtained through a 7×7 dilated convolution with a dilation rate of d=3. Subsequently, the two feature streams are dimensionality-reduced and concatenated, and selection weights are calculated using a spatial selection mechanism.

[0086] ;

[0087] ;

[0088] Finally, the generated spatial weight map (SA) is used to weight and fuse features at different scales, and then applied to the input features. The spatially augmented output features are obtained. ;

[0089] ;

[0090] in This indicates element-wise multiplication. , These are the two parts of the weighted graph after being segmented along the channel. The output features are spatially augmented. It enhanced the spatial structure information of the collapsed area.

[0091] Step 3.4: EMA recalibrates channel weights through grouped multi-scale processing;

[0092] Output features Input to the EMA module to enhance spectral channel features, and output features Divide the data into G groups along the channel dimension, and extract features through 1×1 and 3×3 branches respectively. Aggregate global information through cross-space learning

[0093] ;

[0094] ;

[0095] in, and They represent the grouping features in Convolutional branches and The output of the convolutional branch, M, represents the intermediate feature map that aggregates global context information through cross-space learning, and W represents the final generated channel attention weights;

[0096] Recalibrating features using weighted graphs:

[0097] ;

[0098] in, This represents the input features after grouping. Reshape indicates a weighted operation, and Reshape indicates restoring the features to their original dimensions. This indicates the output characteristics after recalibration of the spectral channel features.

[0099] Step 3.5: At the deepest output layer of the encoder, a Spatial Pyramid Pooling (ASPP) module is introduced to perform multi-scale feature fusion. The specific process is shown in the following equation.

[0100] ;

[0101] in, express The output of the convolution branch, , and These represent expansion ratios of 6, 12, and 18, respectively. The output of the dilated convolution branch, This indicates the output of the global average pooling branch; Cat indicates channel concatenation. This represents the output features that incorporate multi-scale global contextual information;

[0102] Step 3.6: The UNet++ architecture is used as the decoder to gradually restore the spatial resolution of the feature map through nested dense skip connection paths; the decoder node receives upsampled features from the same level and skip connection features from the encoder or previous node.

[0103] By utilizing the layer-by-layer convolution downsampling operation of ResNet101, the spatial resolution is progressively reduced. The input image size is 256*256*5. The image first passes through Layer 0, which includes a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer with a stride of 2. Two consecutive downsampling operations reduce the feature map size from 256×256 to 64×64, expanding the number of channels to 64, generating the first-level feature map. Subsequently, the feature map passes through four residual stages of ResNet101. After each stage, the output is enhanced by the LSK module. The evolution of the feature dimension is as follows: 64*64*256 → 32*32*512 → 16*16*1024 → 8*8*2048. After each downsampling, the number of channels increases with the network depth to enhance the abstract expression of semantic information. This process forms a multi-scale, high-dimensional feature representation, which serves as the basis for skip connections and feature fusion in the subsequent UNet++ decoding path.

[0104] Step 3.7: In the feature fusion stage of the decoder, the Temporal Feature Fusion (TFF) module is used to replace the traditional channel splicing operation, fusing features from the encoder. Features from the decoder The difference between the two features is calculated; the feature differences are extracted using depthwise separable convolution, and adaptive fusion weights are generated.

[0105] Difference information between two features ;

[0106] in, This represents the deep semantic features from the encoder. This represents the upsampled features from the decoder. Represent the difference between the two features; generate adaptive fusion weights:

[0107] ;

[0108] ;

[0109] Where DSConv represents depthwise separable convolution, and Cat represents channel concatenation. and They represent the features respectively and Generate adaptive fusion weights; output final fusion features:

[0110] .

[0111] Step 3.8: Skip connections are made between the downsampled output features of the same layer in U-Net++ and the upsampled features of the current layer, and then they are fused. At the end of the decoder, bilinear interpolation combined with convolution operations is used to restore the feature map to the same spatial resolution as the original input image. The feature channels are mapped to the number of categories through the final 1×1 convolutional layer, and the pixel-level segmentation result is output. The output map and the input map are spatially aligned, achieving accurate segmentation of the target region in the input image.

[0112] Step 4: Using the segmentation model trained in Step 3, input the rice TIFF format file to be segmented, detect whether lodging exists, and segment out the lodged areas of the rice.

[0113] The AdamW optimizer was used, with an initial learning rate set to Weight decay is set to The learning rate is dynamically adjusted using a cosine annealing strategy, and a joint loss function is employed.

[0114] ;

[0115] ;

[0116] ;

[0117] in, This represents the Dice loss function, used to address the imbalance between positive and negative samples. represents the soft cross-entropy loss function, used to smooth the classification probability distribution and prevent the model from overfitting; N represents the total number of pixels in the image; This represents the true label of the i-th pixel. This indicates the probability that the model predicts the pixel to be upside down; To prevent smooth terms with a denominator of zero; This represents the ground truth label after label smoothing. A small smoothing factor is introduced to reduce the interference of noisy labels on model training. The trained segmentation model is read, and the original TIFF image to be segmented is input into the segmentation model to detect whether lodging exists and to segment the lodged rice area.

[0118] The specific functions of this invention are as follows:

[0119] Multi-source, multi-scale feature extraction: Through the improved ResNet101 backbone network, it can process multi-channel input data containing the original spectrum and vegetation index (NDRE), extract feature maps at five scales step by step, and construct a multi-level feature representation adapted to the complex texture of rice lodging area.

[0120] Spatial-spectral synergistic enhancement: A spatial-spectral dual attention mechanism (SS-DAM) is constructed, which integrates large kernel selective attention (LSK) and efficient multi-scale channel attention (EMA) in the encoding stage to provide targeted enhancement for the irregular spatial morphology of the lodged area and the weak spectral differences between different objects of the same spectrum.

[0121] Dynamic receptive field and channel recalibration: The large kernel decomposition convolution of the LSK module is used to adaptively expand the effective receptive field and accurately capture large-scale and sporadic folded features; the cross-dimensional interaction mechanism of the EMA module is used to recalibrate the channel weights, enhance the response of key bands and suppress background noise interference.

[0122] Global multi-scale context aggregation: A hollow spatial pyramid pooling module is integrated at the end of the encoder. Global context information is captured through parallel hollow convolution branches with different sampling rates, which effectively solves the problem of drastic scale changes in lodged areas in farmland scenes.

[0123] Differential Feature Adaptive Fusion: The enhanced multi-scale features are input into the decoding network, and a feature fusion module is introduced at the skip connection. By calculating the feature difference information and generating adaptive fusion weights, the fusion process of deep semantics of the encoder and shallow details of the decoder is optimized.

[0124] Hybrid loss optimization prediction: A joint loss function strategy (Dice Loss + Soft CrossEntropy) is adopted to handle the class imbalance and probability distribution smoothing problems respectively, optimize the model training convergence process, and generate a fine segmentation mask by outputting pixel-level classification probabilities through the terminal convolutional layer.

[0125] Results Evaluation and Visualization: The probability map output by the model is processed by Argmax and binarized to generate the final distribution map of rice lodging areas. The mean cross-union ratio, precision and recall are calculated simultaneously to achieve a quantitative assessment of lodging disaster.

[0126] like Figure 4 As shown, the image displays the pixel-level segmentation visualization results of the model constructed in this invention on the test set. The first row of images is the test image of the input model, which is a standard image patch generated by cropping multispectral data of the field collected by UAV through a sliding window; the second row of images is the prediction mask of the corresponding lodging area of ​​rice output by the model inference, which intuitively reflects the model's accuracy in identifying lodging areas and its ability to capture edge details in complex field backgrounds.

[0127] To verify the advancement of this invention, ablation experiments showed that on a rice lodging segmentation dataset, the proposed method achieved a lodging region detection accuracy of 95.31% and an average intersection-over-union (IoU) of 93.53%. Compared to the baseline UNet++ model (94.62% accuracy, 93.37% mIoU), this represents an improvement of 0.69 and 0.16 percentage points, respectively; the F1 score increased from 94.51% to 94.65%. The steady improvement in these metrics demonstrates that the introduction of multi-feature fusion and spatial-spectral dual attention mechanisms effectively solves the problems of blurred lodging region edges and misjudgments, significantly outperforming mainstream networks such as DeepLabV3+ and PSPNet. On the hardware platform, thanks to the efficient multi-scale attention (EMA) and the lightweight design of depthwise separable convolutions, the model achieves a new high in accuracy (OA reaching 97.77%) while maintaining manageable computational resource consumption, realizing a synergistic optimization of high accuracy, lightweight design, and high speed. It exhibits strong environmental adaptability and can cover complex farmland sub-scensations such as large-area patchy lodging, sporadic lodging, and irregular edges. It is particularly effective in challenging scenarios where the spectra of semi-lodged rice and normal rice are similar. It demonstrates outstanding resistance to changes in light intensity and background interference, and can be adapted to various smart agriculture scenarios such as drone disaster assessment, agricultural insurance loss determination, and precision operation path planning.

[0128] This invention addresses the task of segmenting lodged rice by constructing a segmentation model based on multi-feature fusion. The model encompasses three stages: multi-source feature data construction, segmentation model building, and lodged region segmentation. By introducing the LSK attention mechanism, leveraging its large kernel and dilated convolution characteristics, it effectively adapts to the varied morphology and irregular boundaries of lodged regions, expanding the effective receptive field. The EMA attention mechanism enhances the expression of spectral channel features through cross-dimensional interaction, helping to distinguish subtle spectral differences between lodged and healthy rice. The ASPP module not only captures the global context but also alleviates the recognition difficulties caused by varying scales of lodged regions. Finally, the TFF module optimizes the feature fusion process in skip connections, reducing information loss during the encoding and decoding stages.

[0129] By combining these improvements, this invention effectively enhances the segmentation accuracy and robustness of rice lodging areas in complex farmland settings. Field images are acquired using DJI multispectral drones, a multi-feature dataset containing multispectral information is constructed, and a deep learning model based on an improved UNet++ is established. A spatial-spectral dual attention mechanism is introduced to collaboratively enhance spatial structural and spectral features. A hollow spatial pyramid pooling and temporal feature fusion module is used to capture global contextual information and optimize cross-level interactions. This method overcomes the irregular shapes of lodging areas and interference from foreign objects with the same spectrum in complex farmland settings, significantly improving segmentation accuracy and robustness, achieving rapid and accurate assessment of rice lodging, and demonstrating promising agricultural application prospects.

[0130] This invention is not limited to the above embodiments. Based on the technical solutions disclosed herein, those skilled in the art can make some substitutions and modifications to some of the technical features without creative effort, and all such substitutions and modifications are within the protection scope of this invention.

Claims

1. A method for segmenting lodging areas in rice fields based on multi-feature data fusion, characterized in that: Includes the following steps, Step 1: Create a rice lodging dataset by exporting it as a TIFF image file using a DJI multispectral drone; label the image file and convert the labeled information into raster data in ArcGIS using vector-to-raster conversion; finally, crop the raster and perform data augmentation based on the extent of the original TIFF image to obtain the label data. Step 2: Construct a rice lodging segmentation model based on multi-feature data fusion; Step 3: Input the multi-feature dataset preprocessed in Step 1 into the model constructed in Step 2 for training, optimize the network parameters, and finally generate a segmentation model for rice lodging areas. Step 4: Using the segmentation model trained in Step 3, input the rice TIFF format file to be segmented, detect whether lodging exists, and segment out the lodged areas of the rice.

2. The rice lodging region segmentation method based on multi-feature data fusion according to claim 1, characterized in that: The specific details of step 1 are as follows: Step 1.1: Use a DJI multispectral drone to collect field images of rice at the ripening stage, including four bands: green light, red light, red edge, and near-infrared. Use ArcGIS software to crop out the lodged rice area, create a vector file in the TIFF image coordinate system to label the target area, and add attribute fields. In the attribute fields, set the lodged rice to 255 to represent the lodged area, and generate the corresponding label map through vector-to-raster technology. Step 1.2: Using the sliding window algorithm, the window size is set to 256×256 and the step size is 256. The large multi-channel image and its corresponding label image are cropped into a series of small image blocks. During the cropping process, image blocks containing more than a threshold of invalid data are automatically removed to ensure the effectiveness of the training data. Step 1.3: After the data processing is completed, the image patches are augmented by rotation, mirroring, flipping, and contrast adjustment to expand the dataset and improve the model's generalization ability.

3. The rice lodging region segmentation method based on multi-feature data fusion according to claim 2, characterized in that: The segmentation model in step 2 adopts an improved UNet++ network architecture, which includes a feature extraction encoder, dense skip connections, and a feature reconstruction decoder. The encoder path uses ResNet101 as the backbone network and cascades a spatial-spectral dual attention mechanism to extract multi-scale features step by step. The encoder end is connected to a void space pyramid pooling module to capture global context information; The decoder path introduces nested skip connections of the temporal feature fusion module to adaptively fuse and reconstruct the deep semantic features output by the encoder and the upsampled features generated by the decoder.

4. The rice lodging region segmentation method based on multi-feature data fusion according to claim 3, characterized in that: The specific details of step 3 are as follows: Step 3.1: ResNet101 is used as the feature extraction network. Its original first convolutional layer is removed and replaced with a 7×7 convolutional layer adapted for multi-channel input. An image of size H*W*5 is input into the segmentation model. A percentage-truncated linear stretching strategy is used for normalization. The pixel value of each band is calculated separately, and the pixel values ​​within this range are linearly mapped to the [0, 255] interval to eliminate extreme lighting noise and enhance image contrast. Based on this, an additional vegetation index channel is constructed using band operations to calculate the Normalized Differential Red Edge Vegetation Index (NDRE). The calculation formula is as follows: ； In the formula, NIR represents the multispectral near-infrared band, and RE represents the multispectral red-edge band; The calculated NDRE single-band image is stacked with the original G, R, RedEdge and NIR bands in terms of channel dimension to construct 5-channel input data containing spectral information and vegetation status information; the image is processed by the pre-convolutional layer of ResNet101 and the initial residual module for feature extraction, and the output is a high-dimensional feature map with size H / 4*W / 4*64. Step 3.2 introduces a spatial-spectral dual attention mechanism SS-DAM after each feature level of the ResNet101 encoder. This mechanism is composed of a large selectable kernel LSK module and an efficient multi-scale attention module EMA connected in series, which enhances spatial structure features and spectral features respectively. Step 3.3: The LSK module adaptively adjusts the receptive field through large kernel convolution sequence decomposition and spatial selection mechanism; Step 3.4: EMA recalibrates channel weights through grouped multi-scale processing; Step 3.5: At the deepest output layer of the encoder, a Spatial Pyramid Pooling (ASPP) module is introduced to perform multi-scale feature fusion. The specific process is shown in the following equation. ； in, express The output of the convolution branch, , and These represent expansion ratios of 6, 12, and 18, respectively. The output of the dilated convolution branch, This indicates the output of the global average pooling branch; Cat indicates channel concatenation. This represents the output features that incorporate multi-scale global contextual information; Step 3.6: The UNet++ architecture is used as the decoder to gradually restore the spatial resolution of the feature map through nested dense skip connection paths; the decoder node receives upsampled features from the same level and skip connection features from the encoder or previous node. Step 3.7: In the feature fusion stage of the decoder, the Temporal Feature Fusion (TFF) module is used to replace the traditional channel splicing operation, fusing features from the encoder. Features from the decoder The difference between the two features is calculated; the feature differences are extracted using depthwise separable convolution, and adaptive fusion weights are generated. Step 3.8: Skip connections are made between the downsampled output features of the same layer in U-Net++ and the upsampled features of the current layer, and then they are fused. At the end of the decoder, bilinear interpolation combined with convolution operations is used to restore the feature map to the same spatial resolution as the original input image. The feature channels are mapped to the number of categories through the final 1×1 convolutional layer, and the pixel-level segmentation result is output. The output map and the input map are spatially aligned, achieving accurate segmentation of the target region in the input image.

5. The rice lodging region segmentation method based on multi-feature data fusion according to claim 4, characterized in that: The specific details of step 3.3 are as follows: First, multi-scale features are extracted through large-kernel convolution with sequence decoupling: ； ； in, For input features, These are local features obtained through 5×5 convolution. In order to be in The receptive field features are obtained through a 7×7 dilated convolution with a dilation rate of d=3. Subsequently, the two feature streams are dimensionality-reduced and concatenated, and selection weights are calculated using a spatial selection mechanism. ； ； Finally, the generated spatial weight map (SA) is used to weight and fuse features at different scales, and then applied to the input features. The spatially augmented output features are obtained. ; ； in This indicates element-wise multiplication. , These are the two parts of the weighted graph after being segmented along the channel. The output features are spatially augmented. It enhanced the spatial structure information of the collapsed area.

6. The rice lodging region segmentation method based on multi-feature data fusion according to claim 5, characterized in that: The specific details of step 3.4 are as follows: Output features Input to the EMA module to enhance spectral channel features, and output features Divide the data into G groups along the channel dimension, and extract features through 1×1 and 3×3 branches respectively. Aggregate global information through cross-space learning ； ； in, and They represent the grouping features in Convolutional branches and The output of the convolutional branch, M, represents the intermediate feature map that aggregates global context information through cross-space learning, and W represents the final generated channel attention weights; Recalibrating features using weighted graphs: ； in, This represents the input features after grouping. Reshape indicates a weighted operation, and Reshape indicates restoring the features to their original dimensions. This indicates the output characteristics after recalibration of the spectral channel features.

7. The rice lodging region segmentation method based on multi-feature data fusion according to claim 6, characterized in that: The specific details of step 3.6 are as follows: By utilizing the layer-by-layer convolution downsampling operation of ResNet101, the spatial resolution is progressively reduced. The input image size is 256*256*5. The image first passes through Layer 0, which includes a 7×7 convolutional layer with a stride of 2 and a 3×3 max pooling layer with a stride of 2. Two consecutive downsampling operations reduce the feature map size from 256×256 to 64×64, expanding the number of channels to 64, generating the first-level feature map. Subsequently, the feature map passes through four residual stages of ResNet101. After each stage, the output is enhanced by the LSK module. The evolution of the feature dimension is as follows: 64*64*256 → 32*32*512 → 16*16*1024 → 8*8*2048. After each downsampling, the number of channels increases with the network depth to enhance the abstract expression of semantic information. This process forms a multi-scale, high-dimensional feature representation, which serves as the basis for skip connections and feature fusion in the subsequent UNet++ decoding path.

8. The rice lodging region segmentation method based on multi-feature data fusion according to claim 7, characterized in that: In step 3.7, the difference information between the two features ; in, This represents the deep semantic features from the encoder. This represents the upsampled features from the decoder. Represent the difference between the two features; generate adaptive fusion weights: ； ； Where DSConv represents depthwise separable convolution, and Cat represents channel concatenation. and They represent the features respectively and Generate adaptive fusion weights; output final fusion features: 。 9. A method for segmenting lodging areas of rice based on multi-feature data fusion according to claim 8, characterized in that: The specific details of step 4 are as follows: The AdamW optimizer was used, with an initial learning rate set to Weight decay is set to The learning rate is dynamically adjusted using a cosine annealing strategy, and a joint loss function is employed. ； ； ； in, This represents the Dice loss function, used to address the imbalance between positive and negative samples. represents the soft cross-entropy loss function, used to smooth the classification probability distribution and prevent the model from overfitting; N represents the total number of pixels in the image; This represents the true label of the i-th pixel. This indicates the probability that the model predicts the pixel to be upside down; To prevent smooth terms with a denominator of zero; This represents the ground truth label after label smoothing, which reduces the interference of noisy labels on model training by introducing a small smoothing factor. The trained segmentation model is read, and the original TIFF image to be segmented is input into the segmentation model to detect whether lodging exists and segment out the lodged rice area.

Images