A rice plant counting method based on low-altitude unmanned aerial vehicle image, group perception adaptive mechanism and inverted density scale fusion loss

Abstract

The embodiment of the application provides a rice plant counting method based on a low-altitude unmanned aerial vehicle image, a group perception adaptive mechanism and an inverted density scale fusion loss, which comprises the following steps: S1, data preprocessing; S2, constructing an encoder-decoder network model, wherein the encoder adopts a pre-trained VGG model as a backbone network, and the decoder uses the group perception adaptive mechanism to perform feature enhancement and weight importance screening on the input feature map; S3, training the encoder-decoder network model to obtain a counting model, and using an inverted density scale fusion loss function in the training process; and S4, performing forward propagation on the input rice image by using the counting model, outputting a predicted density map, and obtaining a rice plant counting result by summing the predicted density map. The method enhances the extraction capability of key plant features, realizes multi-scale adaptive supervision, and significantly improves the counting accuracy and robustness in complex field environments such as density and occlusion.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of the cross of deep learning and agricultural information technology, and particularly relates to a rice plant counting method based on low-altitude unmanned aerial vehicle images, group perception adaptive mechanism and inverted density scale fusion loss. BACKGROUND

[0002] As a major grain crop in China, the accurate counting of rice plants is of great significance for yield prediction and agricultural management. Traditional manual counting methods are inefficient and costly, and cannot meet the needs of modern large-scale and precision agriculture.

[0003] With the development of computer vision technology, automated counting methods based on image processing have emerged. Early methods mainly relied on color segmentation, morphological operations combined with traditional machine learning algorithms, and performed well in relatively simple background scenes. However, their stability and accuracy significantly decreased under actual conditions such as complex field lighting, plant overlap and occlusion. In recent years, low-altitude unmanned aerial vehicles have become an important tool for obtaining crop information in the field due to their flexible deployment, high resolution and low cost, providing high-quality image data for plant counting. At the same time, deep learning-based methods have gradually become the mainstream for processing dense plant counting in low-altitude unmanned aerial vehicle images, especially density map regression methods, which indirectly achieve counting by predicting plant distribution density maps, to some extent, alleviating the challenges posed by occlusion. However, existing density regression networks still have limitations in feature extraction and fusion mechanisms, making it difficult to fully capture the subtle features of dense plants captured by low-altitude unmanned aerial vehicles at multiple scales, and lacking effective supervision mechanisms for multi-scale density map prediction, resulting in room for improvement in counting accuracy in complex planting scenarios.

[0004] Therefore, how to combine the characteristics of low-altitude unmanned aerial vehicle images, enhance the feature perception ability of the network in key areas, and design a more reasonable multi-scale loss function has become a key problem for further improving the accuracy of rice plant counting. SUMMARY

[0005] The present application provides a rice plant counting method based on low-altitude unmanned aerial vehicle images, group perception adaptive mechanism and inverted density scale fusion loss, which enhances the extraction ability of key plant features through the group perception adaptive mechanism (GAAM), and realizes multi-scale adaptive supervision by combining the inverted density scale fusion loss (IDSFLoss), thereby significantly improving the counting accuracy and robustness of the model in complex field environments such as dense and occluded plants.

[0006] In a first aspect, a rice plant counting method based on low-altitude unmanned aerial vehicle images, group perception adaptive mechanism and inverted density scale fusion loss is provided, the method comprising:

[0007] S1: Data preprocessing, converting the original rice images and their annotations into training data containing an initial density map and a plant attention map, and performing augmentation processing on the training data;

[0008] S2: Construct an encoder-decoder network model, where the encoder uses a pre-trained VGG model as the backbone network, and the decoder uses a group-aware adaptive mechanism (GAAM) to enhance features and select weight importance for the input feature map.

[0009] S3: Train the encoder-decoder network model to obtain the counting model. During the training process, a fusion loss function is used, which includes binary cross-entropy loss (BCELoss) and inverted density scale fusion loss (IDSFLoss).

[0010] S4: The input rice image is forward-propagated using the counting model to output a predicted density map. The rice plant count is obtained by summing the predicted density map.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, step S1 includes:

[0012] S101: Convert the raw labeled data into a unified format file containing the initial density map and the plant attention map;

[0013] S102: Perform data augmentation and standardization processing on the original image and the uniform format file;

[0014] S103: Perform size adjustment and normalization on the validation set data for use in model validation.

[0015] In conjunction with the first aspect, in certain implementations of the first aspect, the group-aware adaptive mechanism (GAAM) includes:

[0016] The feature enhancement module includes spatial attention, channel attention, and group-aware attention. The group-aware attention mechanism groups the channels of the feature map and calculates independent weights for the pixel values ​​at each spatial location within each channel group.

[0017] The weight importance filtering module filters the fused weights based on a threshold set by a gating mechanism.

[0018] In conjunction with the first aspect, in certain implementations of the first aspect, the execution logic of the group-aware attention mechanism includes:

[0019] Receive input feature maps of shape (B, C, H, W), where B is the batch size, C is the number of channels, H is the height, and W is the width;

[0020] Divide the C channels into N groups, with each group containing C / N channels;

[0021] For each group, a weight value is independently calculated for the pixel value at each spatial location (H, W) of each channel contained therein;

[0022] All weights calculated from the N groups are combined to generate a group-aware attention weight map of shape (B, C, H, W).

[0023] It should be understood that the Group-Aware Adaptive Mechanism (GAAM) proposed in this application achieves more refined and adaptive weight adjustment of feature maps by fusing spatial, channel, and group-aware triple attention, thereby significantly improving the network's ability to perceive and extract key features of rice plants. Traditional attention mechanisms mostly focus on the global spatial or channel dimensions, making it difficult to differentiate and weight the subtle local features of dense plants. The Group-Aware Adaptive Mechanism (GAAM) proposed in this application introduces group-aware attention, grouping channels and independently calculating weights for pixels at each spatial location within the group, thus achieving refined feature enhancement. Subsequently, a threshold-based gating mechanism is used to adjust the fused weights, adaptively strengthening important features highly correlated with plant counts (such as the top of the plant and key areas of the stem) while suppressing interfering features caused by background or occlusion. This mechanism enables the network to more accurately focus on the effective counting area of ​​the plant when decoding fused multi-scale features, especially in complex scenarios with dense plants and severe mutual occlusion, significantly enhancing the model's feature discrimination ability and counting robustness, providing a crucial guarantee for the final generation of high-precision density maps.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, the formula for calculating the inverted density-scale fusion loss (IDSFLoss) is as follows:

[0025]

[0026] Where L0 is the mean squared error loss between the predicted density map and the true density map at the original resolution. For multi-scale density cascade loss, and These are the weighting coefficients.

[0027] In conjunction with the first aspect, in certain implementations of the first aspect, the multi-scale density cascade loss The methods for obtaining it include:

[0028] Multi-scale block summation downsampling is performed on the original predicted density map P and the true density map T to obtain coarse-grained density maps P at Z different scales. i and Ti , where i represents the scale index, and i=0 represents the original scale;

[0029] Calculate the mean square error (MSE) at each scale. i , T i );

[0030] Based on the mean squared error at each scale, the attention weights at each scale during the current training process are calculated using an adaptive weighting formula. ;

[0031] Calculate the penalty term for each scale It is the ratio of the actual error at that scale. Error ratio with theoretical expectation The absolute value of the difference;

[0032] Based on the attention weight and penalty items Calculate the final multi-scale density cascade loss. .

[0033] In conjunction with the first aspect, in certain implementations of the first aspect, the multi-scale density cascade loss The calculation formula is as follows:

[0034] ,

[0035] Where Z represents Z scales, and P0 and T0 represent the original prediction and ground truth density maps, respectively.

[0036] It should be understood that the Inverted Density Scale Fusion Loss (IDSFLoss) proposed in this application significantly enhances the model's consistency in modeling plant density distribution at different resolutions and its training stability by constructing a multi-scale density supervision system and an adaptive error penalty mechanism. This loss function not only calculates the density map error at the original scale but also constructs a multi-level coarse-grained density mapping through block summation and downsampling. Its core advantage lies in the introduction of a dynamic weight allocation mechanism, which adaptively adjusts the training weights according to the current fitting error at each scale, making the network more focused on the difficult-to-learn scale levels and effectively overcoming the training imbalance problem caused by fixed weights in traditional multi-scale losses. Simultaneously, by constructing a penalty term by comparing the theoretical expected error ratio with the actual error ratio, this loss can effectively suppress the abnormal amplification of prediction bias caused by scale changes, avoiding the dominance of a single scale error in the entire training process. This significantly improves the robustness of density regression and the final counting accuracy in complex scenarios with dense and unevenly distributed plants.

[0037] In conjunction with the first aspect, in some implementations of the first aspect, in step S4, the rice plant count result is obtained by calculating the sum of all pixel values ​​in the predicted density map. Attached Figure Description

[0038] Figure 1 The flowchart illustrates a rice plant counting method based on low-altitude UAV images, a group-aware adaptive mechanism, and inverted density scale fusion loss, as provided in this application embodiment.

[0039] Figure 2 A flowchart illustrating a data preprocessing method provided in this application embodiment. Detailed Implementation

[0040] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0041] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0042] Rice is a crucial food crop in my country, and accurate plant counting is essential for yield prediction and agricultural management. Traditional manual counting is inefficient and costly, making it unsuitable for the demands of modern large-scale agriculture. Automated methods based on image processing are effective in simple scenarios, but lack stability under complex lighting conditions, overlapping plants, and occlusion in the field. In recent years, low-altitude drones have provided high-quality image data resources for automated rice plant counting due to their flexible deployment and low cost. Meanwhile, density map regression methods based on deep learning have alleviated the occlusion problem to some extent, but limitations remain, such as insufficient multi-scale feature fusion and a lack of effective multi-scale supervision mechanisms. Therefore, the counting accuracy in dense and complex scenes based on low-altitude drone images needs further improvement.

[0043] This application provides a rice plant counting method based on low-altitude UAV images, employing a group-aware adaptive mechanism and inverted density scale fusion loss. This method can effectively overcome the aforementioned problems.

[0044] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings.

[0045] Figure 1 The flowchart illustrates a rice plant counting method based on low-altitude UAV images, a group-aware adaptive mechanism, and inverted density scale fusion loss, as provided in this application embodiment.

[0046] refer to Figure 1 In some examples, the method includes:

[0047] S1: Data preprocessing, converting the original rice images and their annotations into training data containing an initial density map and a plant attention map, and performing augmentation processing on the training data;

[0048] S2: Construct an encoder-decoder network model, where the encoder uses a pre-trained VGG model as the backbone network, and the decoder uses a group-aware adaptive mechanism (GAAM) to enhance features and select weight importance for the input feature map.

[0049] S3: Train the encoder-decoder network model to obtain the counting model. During the training process, a fusion loss function is used, which includes binary cross-entropy loss (BCELoss) and inverted density scale fusion loss (IDSFLoss).

[0050] S4: The input rice image is forward-propagated using the counting model to output a predicted density map. The rice plant count is obtained by summing the predicted density map.

[0051] In one possible implementation, embodiments of this application employ an encoder-decoder structure. A pre-trained VGG model is used as the backbone network of the feature encoder, and a custom BackEnd neural network module is used as the decoder.

[0052] Figure 2 A flowchart illustrating a data preprocessing method provided in this application embodiment.

[0053] refer to Figure 2 In some examples, step S1 includes:

[0054] S101: Convert the raw labeled data into a unified format file containing the initial density map and the plant attention map;

[0055] S102: Perform data augmentation and standardization processing on the original image and the uniform format file;

[0056] S103: Perform size adjustment and normalization on the validation set data for use in model validation.

[0057] In one possible implementation, the URC dataset used in this application embodiment is a high-throughput rice plant dataset based on drone data collection. It contains 355 RGB rice images with a resolution of 5472 × 3468, collected using a low-altitude drone (DJI Phantom 4 Advanced), and 257,793 manually labeled points. In the URC, 246 images are randomly selected for training, and the remaining 109 images are used for validation. Each image contains 84-1125 rice plants, with an average of 726 plants per image. This invention aims to count rice plants.

[0058] In one possible implementation, the data preprocessing method provided in this application includes: firstly, data format conversion: reading the original rice RGB image and the corresponding MAT format point annotation file, and generating a supervision signal based on the image size and annotation point coordinates. Specifically, a Gaussian kernel with a standard deviation σ=6 is used to convolve each annotation point to generate an initial density map D(x). Next, a plant attention map A(x) is generated based on the pixel values ​​of the initial density map, according to the rule: if D(x... i If )>0.001, then A(x) i=1 otherwise 0. Finally, the initial density map, attention map, and keypoint coordinates are saved as an H5 file. For the training set data, data augmentation is performed after loading the image and the corresponding H5 file. Random scaling, random cropping, horizontal flipping, and brightness adjustment are performed simultaneously on the image and its initial density map. After augmentation, the image is converted into a PyTorch tensor and normalized. At the same time, the initial density map is downsampled by 8 times to generate the final ground truth density map and attention map used for supervised training. For the validation set data, the image size is first adjusted after loading to ensure that the height and width are both integer multiples of 16 to adapt to the network structure. If the smallest side of the image is less than 384 pixels, it is enlarged proportionally to a smallest side of 384. Then, tensor transformation and normalization are performed again, and the processed image and the corresponding total number of rice plants are output for model validation.

[0059] In some examples, the group-aware adaptive mechanism (GAAM) includes:

[0060] The feature enhancement module includes spatial attention, channel attention, and group-aware attention. The group-aware attention mechanism groups the channels of the feature map and calculates independent weights for the pixel values ​​at each spatial location within each channel group.

[0061] The weight importance filtering module filters the fused weights based on a threshold set by a gating mechanism.

[0062] In some examples, the execution logic of the group-aware attention mechanism includes:

[0063] Receive input feature maps of shape (B, C, H, W), where B is the batch size, C is the number of channels, H is the height, and W is the width;

[0064] Divide the C channels into N groups, with each group containing C / N channels;

[0065] For each group, a weight value is independently calculated for the pixel value at each spatial location (H, W) of each channel contained therein;

[0066] All weights calculated from the N groups are combined to generate a group-aware attention weight map of shape (B, C, H, W).

[0067] In a possible implementation, the implementation of the group-aware adaptive mechanism in the decoder feature fusion process includes: taking a certain hierarchical feature map (with the shape of (B, C, H, W)) output by the encoder as the input, and first performing feature enhancement: calculating three groups of weights of spatial attention, channel attention, and group-aware attention in parallel. The spatial attention generates a spatial weight matrix (B, 1, H, W) by performing global average pooling and max pooling on the feature map in the channel dimension; the channel attention generates a channel weight vector (B, C, 1, 1) through global spatial pooling; the innovative group-aware attention divides the C channels into N groups, independently calculates the weight values for each spatial position (a total of H×W positions) of each channel in each group, and finally combines them into a weight map with the same shape as the input (B, C, H, W). Subsequently, the three attention weights are multiplied element-wise to obtain the fused attention weight. After completing the feature enhancement, perform weight importance screening: set a threshold, perform a gating operation on the fused weights to generate two weight maps. For each weight value w i , weight Figure 1 , if w i ≥ threshold, then set it to 1, if w i < threshold, then the value remains unchanged; weight Figure 2 , if w i ≥ threshold, then the value remains unchanged, if w i < threshold, then set it to 0. Then add these two weight maps to obtain the final weight map. This operation strengthens the weights of important features and suppresses the weights of unimportant features, realizing the adaptive screening of features. Finally, multiply the adjusted weight map and the original input feature map element-wise to output the refined and enhanced and screened features. This mechanism enables the network to focus more on the key areas of the plant when fusing multi-scale features, effectively improving the feature discrimination ability in the dense occlusion scenario.

[0068] In some examples, the calculation formula of the inverted density scale fusion loss (IDSFLoss) is as follows:

[0069]

[0070] where, L0 is the mean square error loss between the predicted density map and the true density map at the original resolution, is the multi-scale density cascade loss, and are the weight coefficients.

[0071] In some examples, the method for obtaining the multi-scale density cascade loss includes:

[0072] Multi-scale block summation downsampling is performed on the original predicted density map P and the true density map T to obtain coarse-grained density maps P at Z different scales. i and T i , where i represents the scale index, and i=0 represents the original scale;

[0073] Calculate the mean square error (MSE) at each scale. i , T i );

[0074] Based on the mean squared error at each scale, the attention weights at each scale during the current training process are calculated using an adaptive weighting formula. ;

[0075] Calculate the penalty term for each scale It is the ratio of the actual error at that scale. Error ratio with theoretical expectation The absolute value of the difference;

[0076] Based on the attention weight and penalty items Calculate the final multi-scale density cascade loss. .

[0077] In one possible implementation, a logarithmic function is used to compress the range of the MSE loss, preventing large errors from dominating training. The final result is the scaled prediction and ground truth density map-penalized MSE loss:

[0078]

[0079] In some examples, the multi-scale density cascade loss The calculation formula is as follows:

[0080] ,

[0081] Where Z represents Z scales, and P0 and T0 represent the original prediction and ground truth density maps, respectively.

[0082] In some examples, in step S4, the rice plant count result is obtained by calculating the sum of all pixel values ​​in the predicted density map.

[0083] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or variations made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A rice plant counting method based on low-altitude UAV imagery, group-aware adaptive mechanism, and inverted density scale fusion loss, characterized in that... The method includes: S1: Data preprocessing, converting the original rice images and their annotations into training data containing an initial density map and a plant attention map, and performing augmentation processing on the training data; S2: Construct an encoder-decoder network model, where the encoder uses a pre-trained VGG model as the backbone network, and the decoder uses a group-aware adaptive mechanism (GAAM) to enhance features and select weight importance for the input feature map. The group-aware adaptive mechanism (GAAM) includes: The feature enhancement module includes spatial attention, channel attention, and group-aware attention. The group-aware attention mechanism groups the channels of the feature map and calculates independent weights for the pixel values ​​at each spatial location within each channel group. The weight importance filtering module sets a threshold based on a gating mechanism to filter the fused weights. The execution logic of the group-aware attention mechanism includes: receiving an input feature map of shape (B, C, H, W), where B is the batch size, C is the number of channels, H is the height, and W is the width; dividing the C channels into N groups, each group containing C / N channels; for each group, independently calculating a weight value for the pixel value of each spatial position (H, W) of each channel contained therein; and combining all the weights calculated for the N groups to generate a group-aware attention weight map of shape (B, C, H, W). The formula for calculating the inverted density-scale fusion loss (IDSFLoss) is as follows: in, L 0 This represents the mean squared error loss between the predicted density map and the true density map at the original resolution. For multi-scale density cascade loss, and These are the weighting coefficients; The multi-scale density cascade loss The methods for obtaining it include: Multi-scale block summation downsampling is performed on the original predicted density map P and the true density map T to obtain coarse-grained density maps P at Z different scales. i and T i , where i represents the scale index, and i=0 represents the original scale; Calculate the mean square error (MSE) at each scale. i , T i Based on the mean squared error of each scale, the attention weights for each scale during the current training process are calculated using an adaptive weighting formula. ; Calculate the penalty term for each scale It is the ratio of the actual error at that scale. Error ratio with theoretical expectation The absolute value of the difference; Based on the attention weight and penalty items Calculate the final multi-scale density cascade loss. ; The multi-scale density cascade loss The calculation formula is as follows: ， Where Z represents Z scales, and P0 and T0 represent the original prediction and ground truth density maps, respectively; S3: Train the encoder-decoder network model to obtain the counting model. During the training process, a fusion loss function is used, which includes binary cross-entropy loss (BCELoss) and inverted density scale fusion loss (IDSFLoss). S4: The input rice image is forward-propagated using the counting model to output a predicted density map. The rice plant count is obtained by summing the predicted density map.

2. The method according to claim 1, characterized in that, Step S1 includes: S101: Convert the raw labeled data into a unified format file containing the initial density map and the plant attention map; S102: Perform data augmentation and standardization processing on the original image and the uniform format file; S103: Perform size adjustment and normalization on the validation set data for use in model validation.

3. The method according to any one of claims 1 to 2, characterized in that, In step S4, the rice plant count result is obtained by calculating the sum of all pixel values ​​in the predicted density map.

Images