Lightweight field wheat ear detection method and device based on improved RTDETR
By improving the RTDETR model, introducing a lightweight FasterNet network and an adaptive feature normalization module, and combining it with a matching-perceptual loss function, the problems of occlusion and light variation in wheat ear detection in the field were solved, achieving efficient and stable wheat ear detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies struggle to balance detection accuracy and real-time performance in complex field environments where wheat ears are severely obscured and lighting conditions vary greatly. YOLO series algorithms also suffer from high computational overhead and limited real-time performance in scenarios with densely distributed wheat ears.
An improved RTDETR model is adopted, a lightweight FasterNet network is introduced as the backbone network, the AIFI module is replaced with the AIFI-DyT module, and the DRBNCSPELAN4 module and the matching-aware loss function are used to optimize feature extraction and loss function, thereby enhancing the model's detection capability under different lighting conditions and performance in densely occluded scenes.
While reducing the number of model parameters and computational cost, it improves the accuracy and speed of wheat detection, enhances the stability of the model under different lighting conditions and the detection performance in occluded scenarios, and simplifies the training process.
Smart Images
Figure CN121904552A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning crop ear detection technology, and in particular to a lightweight field wheat ear detection method and device based on an improved RTDETR. Background Technology
[0002] Wheat, as one of the world's major food crops, directly impacts food security and agricultural economic stability through its yield and quality. According to data from the Food and Agriculture Organization of the United Nations (FAO), global wheat production in 2023 was approximately 785 million tons. Research shows a significant positive correlation between wheat yield and the number of wheat ears. Against this backdrop, timely and accurate detection of wheat ears in the field is particularly important. Wheat ear detection is not only used for yield estimation and growth monitoring, but also assists in disease identification and precision fertilization, thereby effectively controlling losses and improving agricultural production efficiency. On the one hand, efficient detection methods can provide farmers with decision-making support and reduce disease spread; on the other hand, it helps improve detection efficiency, reduce labor costs, and promote the development of smart agriculture.
[0003] Traditional methods for wheat ear detection mainly rely on manual visual inspection or traditional image processing-based methods. Manual detection depends on human experience for identification and counting, which is not only labor-intensive and inefficient, but also highly susceptible to subjective factors, making it difficult to guarantee consistency and accuracy. Traditional image processing methods typically achieve target detection through threshold segmentation, morphological operations, or edge feature extraction. However, these methods are highly dependent on environmental conditions; under varying lighting conditions or when wheat ears are partially obscured, it is difficult to consistently extract effective features, leading to missed detections and false detections, making them unsuitable for large-scale, automated field detection scenarios.
[0004] With the development of deep learning technology, object detection algorithms based on convolutional neural networks have been increasingly applied in agricultural vision to achieve automatic detection of crop targets such as wheat ears. Among them, single-stage object detection algorithms, represented by the YOLO series, achieve a certain balance between detection speed and accuracy by predicting the target location and category in a single network, and have been widely used in field wheat ear detection tasks. However, YOLO series algorithms generally require post-processing steps such as non-maximum suppression (NMS) to filter the prediction results. In scenarios with densely distributed wheat ears, the number of candidate targets increases significantly, and the post-processing computational cost increases accordingly, which limits real-time performance to some extent. Summary of the Invention
[0005] To address the challenges of existing technologies in complex field environments, such as severe wheat shading, significant variations in lighting conditions, and the difficulty in balancing detection accuracy and real-time performance, the primary objective of this invention is to provide a lightweight field wheat detection method based on an improved RTDETR that reduces the number of model parameters and computational load while maintaining efficient feature extraction capabilities for wheat targets, improves the model's wheat detection capabilities under different lighting conditions, and enhances the model's detection performance and convergence speed in densely shaded scenarios.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: a lightweight field wheat ear detection method based on an improved RTDETR, the method comprising the following sequential steps:
[0007] (1) Obtain wheat ear image data in the field environment and preprocess it. Divide the preprocessed wheat ear image data into training set, validation set and test set. Expand the training set, validation set and test set. The expanded training set, validation set and test set together form the wheat ear detection dataset.
[0008] (2) Improve the RTDETR model to obtain the improved RTDETR model, and use the improved RTDETR model as the wheat ear detection model; the RTDETR model includes a backbone network, an encoder and a decoder. Replace the backbone network of the RTDETR model with a FasterNet network, replace the AIFI module in the encoder of the RTDETR model with an AIFI-DyT module, replace the RepC3 module in the encoder of the RTDETR model with a DRBNCSPELAN4 module; replace the zoom focus loss function of the RTDETR model with a matching perception loss function;
[0009] (3) The wheat ear detection model is trained using the expanded training set to obtain the trained wheat ear detection model;
[0010] (4) Input the expanded test set into the trained wheat ear detection model to detect wheat ear targets and obtain wheat ear detection results. The wheat ear detection results refer to the location information of the wheat ear targets.
[0011] In step (1), the preprocessing refers to: cropping the acquired wheat ear image data into images of uniform pixel size, filtering the cropped images, and labeling each wheat ear target in them.
[0012] In step (2), the FasterNet network consists of an embedding layer, four cascaded feature extraction layers, and a fusion layer.
[0013] The embedding layer is used to perform initial feature embedding and dimension mapping on the input wheat ear image. It obtains the initial feature representation through a convolution operation with a kernel size of 4×4 and a stride of 4, and after batch normalization.
[0014] The four feature extraction layers process the embedded features layer by layer in a cascaded manner. Each feature extraction layer consists of multiple FasterNet Blocks, and the calculation formula for each FasterNet Block is as follows:
[0015] ;
[0016] In the formula, Represents FasterNet Block; Indicates the input feature map; This represents a 3×3 partial convolution operation; This represents a 1×1 convolution operation; Indicates the activation function; Represents residual join operation; BN represents batch normalization;
[0017] The fusion layer is used to integrate and downsample the features output by the feature extraction layer. It is performed by a convolution operation with a kernel size of 2×2 and a stride of 2, and after batch normalization, the output is sent to the next stage for deep feature learning.
[0018] In step (2), the AIFI-DyT module is obtained by improving the AIFI module, that is, replacing the layer normalization of the AIFI module with the DyT module;
[0019] Learnable scaling parameters Adaptive feature normalization is achieved using the hyperbolic tangent function, thus optimizing the feature normalization process in the AIFI module.
[0020] The calculation formula for the AIFI-DyT module is as follows:
[0021] ;
[0022] In the formula, This is used to adaptively adjust the scaling range based on the input feature range, avoiding gradient instability or information loss caused by fixed scaling; and These are all channel-wise affine transformation parameters, used to restore feature representation and adjust the output distribution.
[0023] In step (2), the DRBNCSPPELAN4 module is obtained by adding the first and second DRBNCSP modules to the GELAN architecture. The GELAN architecture includes the first, second, third, and fourth convolutional blocks, the Split module, and the first splicing module. The first DRBNCSP module is added between the Split module and the second convolutional block, and the second DRBNCSP module is added between the second convolutional block and the third convolutional block.
[0024] First, the input feature map (Input) is fed into the first convolutional block to obtain feature map I1. The first convolutional block uses a 1×1 kernel, a stride of 1, and zero padding to adjust the channels of the input feature map. Feature map I1 is then fed into the Split module, which divides it into two parts along the channel dimension, denoted as feature map I2 and feature map I3. Feature map I2 is then fed into the first stitching module, while feature map I3 is simultaneously fed into both the first stitching module and the first DRBNCSP module. The first DRBNCSP module processes feature map I3 to obtain feature map I4. Feature map I4 is then fed into the second convolutional block to obtain feature map I5. The second convolutional block uses a 3×3 kernel, a stride of 1, and zero padding to extract local features. Spatial details: Feature map I5 is simultaneously fed into the first stitching module and the second DRBNCSP module. The second DRBNCSP module processes feature map I5 to obtain feature map I6. Feature map I6 is then fed into the third convolutional block to obtain feature map I7. The third convolutional block uses a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1 to further extract local spatial details. Feature map I7 is then fed into the first stitching module. The first stitching module stitches feature maps I2, I3, I5, and I7 along the channel dimension to obtain feature map I8. Feature map I8 is then fed into the fourth convolutional block to obtain the output feature map. The fourth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to perform cross-channel information fusion and channel adjustment on the stitched feature map.
[0025] The first and second DRBNCSP modules have the same structure. The first DRBNCSP module includes a fifth, sixth, and seventh convolutional block, a second concatenation module, and N DRBNBottleneck modules. Feature map I3 is simultaneously fed into the fifth and sixth convolutional blocks. The fifth convolutional block processes feature map I3 to obtain feature map I9. The fifth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 for feature map channel adjustment. Subsequently, feature map I9 is sequentially fed into the N DRBNBottleneck modules to obtain feature map I. 10 , feature map I 10 The data is fed into the second concatenation module; the sixth convolutional block processes feature map I3 to obtain feature map I. 11The sixth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0, for feature map channel adjustment; the sixth convolutional block converts feature map I... 11 The feature map I is sent to the second stitching module, which then stitches it into the second stitching module. 10 and Feature Map I 11 By concatenating along the channel dimension, feature map I is obtained. 12 , feature map I 12 The feature map I4 is obtained by feeding it into the seventh convolutional block. The seventh convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0. It is used to perform cross-channel information fusion and channel adjustment on the stitched feature map.
[0026] The N DRBNBottleneck modules include the first to the Nth DRBNBottleneck modules, and the N DRBNBottleneck modules have the same structure. The first DRBNBottleneck module includes a Dilated ReparamBlock module and an eighth convolutional block. The feature map I9 is input to the Dilated Reparam Block module and the residual branch, respectively. The Dilated Reparam Block module outputs feature map I9. 13 The feature map I is obtained by feeding it into the eighth convolutional block. 14 The eighth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to adjust the number of channels in the output feature; the feature map I... 14 The residual branch is added element-wise to generate feature map I'.
[0027] In step (2), the matching-aware loss function modifies the target label of the positive sample to... And abandon manually set balance parameters Based directly on the matching quality score of This enables differentiated learning, thereby simplifying hyperparameter tuning and enhancing optimization for low-quality matching samples;
[0028] The formula for the matching-perceptual loss function is expressed as follows:
[0029] ;
[0030] In the formula, Predict scores for the model; The quality score for the match between the predicted bounding box and the ground truth target; The value is 1 for positive samples and 0 for negative samples. The modulation coefficient is used to control the balance between easy and difficult samples; To match quality scores The power-modulated weights are used to dynamically adjust the learning intensity of positive samples based on the matching quality. For the model to predict scores The power modulation factor is used to suppress the dominant role of easily classified samples in the loss.
[0031] Another object of the present invention is to provide an electronic device comprising:
[0032] Processor; and
[0033] A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the lightweight field wheat ear detection method based on the improved RTDETR as described above.
[0034] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the lightweight field wheat ear detection method based on the improved RTDETR as described above.
[0035] As can be seen from the above technical solutions, the beneficial effects of this invention are as follows: First, this invention addresses the difficulty of real-time wheat ear detection in the field by improving upon the RTDETR model by introducing a lightweight FasterNet model as the basic backbone network. This reduces the number of model parameters and computational cost while maintaining efficient feature extraction capabilities for wheat ear targets. Second, the AIFI-DyT module in this invention replaces the traditional layer normalization within the module with a DyT design and combines learnable scaling parameters to achieve adaptive feature normalization. This suppresses the influence of illumination changes on the distribution of wheat ear features and effectively enhances the representation stability of key wheat ear features, thereby improving the model's wheat ear detection capability under different illumination conditions. Third, the DRBNCSPELAN4 module in this invention leverages the efficient layer aggregation and gradient path optimization characteristics of the GELAN architecture and uses a multi-branch dilated convolution structure to capture and model multi-scale contextual information. Furthermore, during the inference stage, structural reparameterization technology is used to convert the multi-branch dilated convolution into a single large-scale convolution. The convolutional kernel effectively expands the receptive field without additional computational overhead. Fourth, this invention introduces a matching-aware loss function to address the problem of insufficient optimization of low-quality matching caused by initial sparseness and dense occlusion in wheat ear detection. An adaptive correlation mechanism between matching degree and loss weight is established, assigning higher gradient weights to low IoU matches in the early stage of training, thereby enhancing the optimization signal for matched boxes with localization deviations and strengthening the gradient feedback of occluded samples, which is beneficial to improving the detection performance and convergence speed of the model in densely occluded scenes. This loss simplifies the calculation of loss weights, avoids over-optimization of high-quality matching samples, and makes the training process more stable. Attached Figure Description
[0036] Figure 1 This is a flowchart of the method of the present invention;
[0037] Figure 2 A structural diagram of the improved RTDETR model;
[0038] Figure 3 Here is a structural diagram of the AIFI-DyT module;
[0039] Figure 4 This is a structural diagram of the DRBNCSPELAN4 module;
[0040] Figure 5 The image shows the detection results of the RTDETR model in a scene with strong light and dense shading from wheat ears;
[0041] Figure 6 The image shows the detection results of the improved RTDETR model in a scene with strong light and dense shading from wheat ears.
[0042] Figure 7 The image shows the detection results of the RTDETR model in a scene with low light and dense shading from wheat ears;
[0043] Figure 8 The image shows the detection results of the improved RTDETR model in a scene with low light and dense shading from wheat ears. Detailed Implementation
[0044] like Figure 1 As shown, a lightweight field wheat ear detection method based on an improved RTDETR is described, which includes the following sequential steps:
[0045] (1) Obtain wheat ear image data in the field environment and preprocess it. Divide the preprocessed wheat ear image data into training set, validation set and test set. Expand the training set, validation set and test set. The expanded training set, validation set and test set together form the wheat ear detection dataset.
[0046] (2) The RTDETR model is improved to obtain the improved RTDETR model, such as Figure 2As shown, the improved RTDETR model is used as the wheat detection model. The RTDETR model includes a backbone network, an encoder, and a decoder. The backbone network of the RTDETR model is replaced with a FasterNet network to reduce the number of model parameters and computational complexity. The AIFI module in the encoder of the RTDETR model is replaced with an AIFI-DyT module to enhance the feature representation of the target region. The RepC3 module in the encoder of the RTDETR model is replaced with a DRBNCSPELAN4 module to expand the effective receptive field without increasing computational overhead and optimize the feature fusion efficiency. The zoom focus loss function of the RTDETR model is replaced with a matching perception loss function to improve the optimization effect on low-quality matching samples.
[0047] (3) The wheat ear detection model is trained using the expanded training set to obtain the trained wheat ear detection model;
[0048] (4) Input the expanded test set into the trained wheat ear detection model to detect wheat ear targets and obtain wheat ear detection results. The wheat ear detection results refer to the location information of the wheat ear targets.
[0049] In step (1), the preprocessing refers to: cropping the acquired wheat ear image data into images of uniform pixel size, filtering the cropped images, and labeling each wheat ear target in them.
[0050] Wheat ears were photographed in a field environment, resulting in 525 raw images with a resolution of 3072×3072 pixels. These raw images were then cropped to 640×640 pixels, and further filtered to obtain 1563 valid images. The LabelImg annotation tool was used to annotate the bounding boxes of the wheat ears in these valid images, generating corresponding .txt annotation files. The annotated dataset was then divided into a training set of 1094 images, a validation set of 312 images, and a test set of 157 images in a 7:2:1 ratio. Data augmentation was performed on each subset, including random brightness and contrast adjustments, hue and saturation transformations, geometric flipping, image sharpening, motion blur, and Gaussian noise reduction. After data augmentation, the training set was expanded to 2437 images, the validation set to 696 images, and the test set to 349 images.
[0051] In step (2), the FasterNet network consists of an embedding layer, four cascaded feature extraction layers, and a fusion layer.
[0052] The embedding layer is used to perform initial feature embedding and dimension mapping on the input wheat ear image. It obtains the initial feature representation through a convolution operation with a kernel size of 4×4 and a stride of 4, and after batch normalization.
[0053] The four feature extraction layers process the embedded features layer by layer in a cascaded manner. Each feature extraction layer consists of multiple FasterNet Blocks, and the calculation formula for each FasterNet Block is as follows:
[0054] ;
[0055] In the formula, Represents FasterNet Block; Indicates the input feature map; This represents a 3×3 partial convolution operation; This represents a 1×1 convolution operation; Indicates the activation function; Represents residual join operation; BN represents batch normalization;
[0056] Each FasterNet Block uses partial convolution as its core operator, performing spatial feature extraction only on a portion of the input channels, while the remaining channels are directly preserved and passed to the output through identity mapping. This helps reduce feature information loss and improve feature reuse efficiency.
[0057] The fusion layer is used to integrate and downsample the features output by the feature extraction layer. It is performed by a convolution operation with a kernel size of 2×2 and a stride of 2, and after batch normalization, the output is sent to the next stage for deep feature learning.
[0058] like Figure 3 As shown, in step (2), the AIFI-DyT module is obtained by improving the AIFI module, that is, replacing the layer normalization of the AIFI module with the DyT module;
[0059] Learnable scaling parameters Adaptive feature normalization is achieved using the hyperbolic tangent function, thus optimizing the feature normalization process in the AIFI module.
[0060] The calculation formula for the AIFI-DyT module is as follows:
[0061] ;
[0062] In the formula, This is used to adaptively adjust the scaling range based on the input feature range, avoiding gradient instability or information loss caused by fixed scaling; and These are all channel-wise affine transformation parameters, used to restore feature representation and adjust the output distribution.
[0063] The AIFI-DyT module utilizes the nonlinear compression properties of DyT to suppress extreme input values, while simultaneously employing learnable scaling parameters. The scaling intensity is automatically adjusted according to the range of input features, which can optimize the feature processing in the RTDETR model. This helps to alleviate the differences in feature distribution under different lighting conditions, enhances the representation stability of key features of wheat ears, and enables the model to maintain good adaptability in scenarios with changing lighting.
[0064] In step (2), as Figure 4 As shown, the DRBNCSPPELAN4 module is obtained by adding the first and second DRBNCSP modules to the GELAN architecture. The GELAN architecture includes the first, second, third, and fourth convolutional blocks, the Split module, and the first splicing module. The first DRBNCSP module is added between the Split module and the second convolutional block, and the second DRBNCSP module is added between the second convolutional block and the third convolutional block.
[0065] First, the input feature map (Input) is fed into the first convolutional block to obtain feature map I1. The first convolutional block uses a 1×1 kernel, a stride of 1, and zero padding to adjust the channels of the input feature map. Feature map I1 is then fed into the Split module, which divides it into two parts along the channel dimension, denoted as feature map I2 and feature map I3. Feature map I2 is then fed into the first stitching module, while feature map I3 is simultaneously fed into both the first stitching module and the first DRBNCSP module. The first DRBNCSP module processes feature map I3 to obtain feature map I4. Feature map I4 is then fed into the second convolutional block to obtain feature map I5. The second convolutional block uses a 3×3 kernel, a stride of 1, and zero padding to extract local features. Spatial details: Feature map I5 is simultaneously fed into the first stitching module and the second DRBNCSP module. The second DRBNCSP module processes feature map I5 to obtain feature map I6. Feature map I6 is then fed into the third convolutional block to obtain feature map I7. The third convolutional block uses a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1 to further extract local spatial details. Feature map I7 is then fed into the first stitching module. The first stitching module stitches feature maps I2, I3, I5, and I7 along the channel dimension to obtain feature map I8. Feature map I8 is then fed into the fourth convolutional block to obtain the output feature map. The fourth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to perform cross-channel information fusion and channel adjustment on the stitched feature map.
[0066] The first and second DRBNCSP modules have the same structure. The first DRBNCSP module includes a fifth, sixth, and seventh convolutional block, a second concatenation module, and N DRBNBottleneck modules. Feature map I3 is simultaneously fed into the fifth and sixth convolutional blocks. The fifth convolutional block processes feature map I3 to obtain feature map I9. The fifth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 for feature map channel adjustment. Subsequently, feature map I9 is sequentially fed into the N DRBNBottleneck modules to obtain feature map I. 10 , feature map I 10 The data is fed into the second concatenation module; the sixth convolutional block processes feature map I3 to obtain feature map I. 11 The sixth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0, for feature map channel adjustment; the sixth convolutional block converts feature map I... 11 The feature map I is sent to the second stitching module, which then stitches it into the second stitching module. 10 and Feature Map I 11 By concatenating along the channel dimension, feature map I is obtained. 12 , feature map I 12 The feature map I4 is obtained by feeding it into the seventh convolutional block. The seventh convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0. It is used to perform cross-channel information fusion and channel adjustment on the stitched feature map.
[0067] The N DRBNBottleneck modules include the first to the Nth DRBNBottleneck modules, and the N DRBNBottleneck modules have the same structure. The first DRBNBottleneck module includes a Dilated ReparamBlock module and an eighth convolutional block. The feature map I9 is input to the Dilated Reparam Block module and the residual branch, respectively. The Dilated Reparam Block module outputs feature map I9. 13 The feature map I is obtained by feeding it into the eighth convolutional block. 14 The eighth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to adjust the number of channels in the output feature; the feature map I... 14 The residual branch is added element-wise to generate feature map I'.
[0068] The DRBNCSPELAN4 module replaces the RepC3 module in the neck network to alleviate the problems of convolutional redundancy and insufficient long-range dependency modeling in traditional feature fusion. The DRBNCSPELAN4 module is based on the GELAN architecture, which integrates the cross-stage connectivity of CSPNet and the efficient layer aggregation mechanism of ELAN to achieve gradient path optimization and efficient parameter utilization, and supports flexible stacking. The DRBNCSPELAN4 module retains the multi-branch aggregation and residual structure of CSP-ELAN, and replaces the core small kernel convolution with a dilated reparameter block. During training, the dilated reparameter block models multi-scale contextual information through multi-branch dilated convolution; during inference, it is equivalent to a single large convolution kernel by using structural reparameterization, which expands the receptive field and enhances feature representation without increasing the computational cost.
[0069] In step (2), the matching-aware loss function modifies the target label of the positive sample to... And abandon manually set balance parameters Based directly on the matching quality score of This enables differentiated learning, thereby simplifying hyperparameter tuning and enhancing optimization for low-quality matching samples;
[0070] The formula for the matching-perceptual loss function is expressed as follows:
[0071] ;
[0072] In the formula, Predict scores for the model; The quality score for the match between the predicted bounding box and the ground truth target; The value is 1 for positive samples and 0 for negative samples. The modulation coefficient is used to control the balance between easy and difficult samples; To match quality scores The power-modulated weights are used to dynamically adjust the learning intensity of positive samples based on the matching quality. For the model to predict scores The power modulation factor is used to suppress the dominant role of easily classified samples in the loss.
[0073] A Matching Awareness Loss Function (MAL) is introduced to replace the original VFL zoom focus loss function. This addresses the issue of insufficient low-quality matching optimization caused by initial query sparsity and dense occlusion in wheat ear detection by establishing an adaptive correlation mechanism between matching degree and loss weights. This design assigns higher gradient weights to low IoU matches in the early stages of training, thereby enhancing the optimization signal for matched boxes with localization errors and strengthening gradient feedback for occluded samples. This improves the model's detection performance and convergence speed in densely occluded scenes. The difference between MAL and VFL is that VFL includes manually set balancing parameters. and The target label for the positive samples is The MAL modifies the target label of the positive sample to And abandon manually set balance parameters Based directly on the matching quality score of This enables differentiated learning, thereby simplifying hyperparameter tuning and enhancing optimization for low-quality matching samples.
[0074] To verify the effectiveness of this invention, it was compared with current mainstream target detection models. The experimental results are shown in Table 1. This invention demonstrates a significant advantage in both detection accuracy and computational efficiency. In terms of detection accuracy, this invention achieves the best overall performance among all compared models. Its average precision (mAP50) reaches 93.5%, surpassing the second-best performing YOLOv8s (93.1%). Simultaneously, the model's recall rate is 89.0%, exceeding all compared models. It maintains a high precision rate (92.5%), comparable to the most precise YOLOv8s (92.6%). In terms of computational and storage efficiency, this invention's advantages are particularly prominent. Its computational complexity (GFLOPs) is only 21.2G, significantly lower than all RT-DETR series models, almost on par with the lightest YOLOv13s (20.7G), yet its precision (mAP50) is 1.0 percentage point higher. Compared to YOLOv8s (28.4G), which has the closest mAP50 accuracy, this invention reduces computational cost by approximately 25.4% while improving accuracy by 0.4%. In terms of model storage, this invention has 9.4MB of parameters and a model size of 18.1MB, comparable to the compact YOLOv11s (9.4MB params, 18.3MB size), significantly outperforming the RT-DETR series and demonstrating excellent lightweight design and deployment potential.
[0075] In summary, compared with the lightweight models of the YOLO series, the model of this invention achieves higher detection accuracy and recall while maintaining a compact structure. Compared with the RT-DETR series models, this invention achieves superior detection performance with a smaller parameter scale and computational overhead. The research results fully demonstrate that the model of this invention achieves an excellent balance between detection accuracy, inference speed, and model size, and has good engineering application and promotion value.
[0076] Table 1 Comparison of the model of this invention with other models.
[0077] like Figure 5 As shown, in scenarios with strong lighting and dense shading of wheat ears, the RTDETR model misses some severely shaded wheat ears.
[0078] like Figure 6 As shown, Figure 6 compared to Figure 5 The detection results of the RTDETR model show that the improved RTDETR model did not miss any detections in the same scenario, and the detection completeness was improved.
[0079] like Figure 7 As shown, in scenes with weak lighting and dense shading from wheat ears, the RTDETR model exhibited both missed detections and false detections.
[0080] like Figure 8 As shown, Figure 8 and Figure 7 Compared with the detection results of the RTDETR model in the previous scenario, the improved RTDETR model showed improvements in both false negatives and false positives in this scenario, resulting in enhanced overall detection performance.
[0081] exist Figure 5 , 6 In Figures 7 and 8, the red boxes represent the target detection boxes, and the green circles are the markers for false positives and false negatives.
[0082] In summary, this invention addresses the challenges of real-time wheat ear detection in the field by improving upon the RTDETR model. It introduces a lightweight FasterNet model as the foundational backbone network to reduce the number of model parameters and computational cost while maintaining efficient feature extraction capabilities for wheat ears. The AIFI-DyT module replaces traditional layer normalization with a DyT design and incorporates learnable scaling parameters to achieve adaptive feature normalization. This suppresses the impact of illumination variations on wheat ear feature distribution, effectively enhancing the stability of key wheat ear feature representations and thus improving the model's wheat ear detection capabilities under different illumination conditions. The DRBNCSPELAN4 module leverages the efficient layer aggregation and gradient path optimization features of the GELAN architecture and uses a multi-branch dilated convolution structure to capture and model multi-scale contextual information. Furthermore, during the inference phase, structural reparameterization technology is used to convert the multi-branch dilated convolution into a single large-scale convolution. The convolutional kernel effectively expands the receptive field without additional computational overhead. This invention introduces a matching-aware loss function to address the problem of insufficient optimization of low-quality matching caused by initial sparseness and dense occlusion in wheat ear detection. It establishes an adaptive correlation mechanism between matching degree and loss weight, assigning higher gradient weights to low IoU matches in the early stage of training, thereby enhancing the optimization signal for matched boxes with localization deviations and strengthening gradient feedback for occluded samples. This is beneficial to improving the detection performance and convergence speed of the model in densely occluded scenes. This loss simplifies the calculation of loss weights, avoids over-optimization of high-quality matching samples, and makes the training process more stable.
[0083] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A lightweight field wheat ear detection method based on an improved RTDETR, characterized in that: The method includes the following steps in sequence: (1) Obtain wheat ear image data in the field environment and preprocess it. Divide the preprocessed wheat ear image data into training set, validation set and test set. Expand the training set, validation set and test set. The expanded training set, validation set and test set together form the wheat ear detection dataset. (2) Improve the RTDETR model to obtain the improved RTDETR model, and use the improved RTDETR model as the wheat ear detection model; the RTDETR model includes a backbone network, an encoder and a decoder. Replace the backbone network of the RTDETR model with a FasterNet network, replace the AIFI module in the encoder of the RTDETR model with an AIFI-DyT module, replace the RepC3 module in the encoder of the RTDETR model with a DRBNCSPELAN4 module; replace the zoom focus loss function of the RTDETR model with a matching perception loss function; (3) The wheat ear detection model is trained using the expanded training set to obtain the trained wheat ear detection model; (4) Input the expanded test set into the trained wheat ear detection model to detect wheat ear targets and obtain wheat ear detection results. The wheat ear detection results refer to the location information of the wheat ear targets.
2. The lightweight field wheat ear detection method based on improved RTDETR according to claim 1, characterized in that: In step (1), the preprocessing refers to: cropping the acquired wheat ear image data into images of uniform pixel size, filtering the cropped images, and labeling each wheat ear target in them.
3. The lightweight field wheat ear detection method based on the improved RTDETR according to claim 1, characterized in that: In step (2), the FasterNet network consists of an embedding layer, four cascaded feature extraction layers, and a fusion layer. The embedding layer is used to perform initial feature embedding and dimension mapping on the input wheat ear image. It obtains the initial feature representation through a convolution operation with a kernel size of 4×4 and a stride of 4, and after batch normalization. The four feature extraction layers process the embedded features layer by layer in a cascaded manner. Each feature extraction layer consists of multiple FasterNet Blocks, and the calculation formula for each FasterNet Block is as follows: ； In the formula, Represents FasterNet Block; Indicates the input feature map; This represents a 3×3 partial convolution operation; This represents a 1×1 convolution operation; Indicates the activation function; Represents residual join operation; BN represents batch normalization; The fusion layer is used to integrate and downsample the features output by the feature extraction layer. It is performed by a convolution operation with a kernel size of 2×2 and a stride of 2, and after batch normalization, the output is sent to the next stage for deep feature learning.
4. The lightweight field wheat ear detection method based on improved RTDETR according to claim 1, characterized in that: In step (2), the AIFI-DyT module is obtained by improving the AIFI module, that is, replacing the layer normalization of the AIFI module with the DyT module; Learnable scaling parameters Adaptive feature normalization is achieved using the hyperbolic tangent function, optimizing the feature normalization process in the AIFI module; The calculation formula for the AIFI-DyT module is as follows: ； In the formula, This is used to adaptively adjust the scaling range based on the input feature range, avoiding gradient instability or information loss caused by fixed scaling; and These are all channel-wise affine transformation parameters, used to restore feature representation and adjust the output distribution.
5. The lightweight field wheat ear detection method based on the improved RTDETR according to claim 1, characterized in that: In step (2), the DRBNCSPPELAN4 module is obtained by adding the first and second DRBNCSP modules to the GELAN architecture. The GELAN architecture includes the first, second, third, and fourth convolutional blocks, the Split module, and the first splicing module. The first DRBNCSP module is added between the Split module and the second convolutional block, and the second DRBNCSP module is added between the second convolutional block and the third convolutional block. First, the input feature map (Input) is fed into the first convolutional block to obtain feature map I1. The first convolutional block uses a 1×1 kernel, a stride of 1, and zero padding to adjust the channels of the input feature map. Feature map I1 is then fed into the Split module, which divides it into two parts along the channel dimension, denoted as feature map I2 and feature map I3. Feature map I2 is then fed into the first stitching module, while feature map I3 is simultaneously fed into both the first stitching module and the first DRBNCSP module. The first DRBNCSP module processes feature map I3 to obtain feature map I4. Feature map I4 is then fed into the second convolutional block to obtain feature map I5. The second convolutional block uses a 3×3 kernel, a stride of 1, and zero padding to extract local features. Spatial details: Feature map I5 is simultaneously fed into the first stitching module and the second DRBNCSP module. The second DRBNCSP module processes feature map I5 to obtain feature map I6. Feature map I6 is then fed into the third convolutional block to obtain feature map I7. The third convolutional block uses a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1 to further extract local spatial details. Feature map I7 is then fed into the first stitching module. The first stitching module stitches feature maps I2, I3, I5, and I7 along the channel dimension to obtain feature map I8. Feature map I8 is then fed into the fourth convolutional block to obtain the output feature map. The fourth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to perform cross-channel information fusion and channel adjustment on the stitched feature map. The first and second DRBNCSP modules have the same structure. The first DRBNCSP module includes a fifth, sixth, and seventh convolutional block, a second concatenation module, and N DRBNBottleneck modules. Feature map I3 is simultaneously fed into the fifth and sixth convolutional blocks. The fifth convolutional block processes feature map I3 to obtain feature map I9. The fifth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 for feature map channel adjustment. Subsequently, feature map I9 is sequentially fed into the N DRBNBottleneck modules to obtain feature map I. 10 , feature map I 10 The feature map I3 is fed into the second concatenation module; the sixth convolutional block processes the feature map I3 to obtain the feature map I. 11 The sixth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0, for feature map channel adjustment; the sixth convolutional block converts feature map I... 11 The feature map I is sent to the second stitching module, which then stitches it into the second stitching module. 10 and Feature Map I 11 By concatenating along the channel dimension, feature map I is obtained. 12 , feature map I 12 The feature map I4 is obtained by feeding it into the seventh convolutional block. The seventh convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0. It is used to perform cross-channel information fusion and channel adjustment on the stitched feature map. The N DRBNBottleneck modules include the first to the Nth DRBNBottleneck modules, and the N DRBNBottleneck modules have the same structure. The first DRBNBottleneck module includes a Dilated ReparamBlock module and an eighth convolutional block. Feature map I9 is input to the Dilated Reparam Block module and the residual branch, respectively. The Dilated Reparam Block module outputs feature map I. 13 The feature map I is obtained by feeding it into the eighth convolutional block. 14 The eighth convolutional block uses a convolutional layer with a kernel size of 1×1, a stride of 1, and padding of 0 to adjust the number of channels in the output feature; the feature map I... 14 The residual branch is added element-wise to generate feature map I'.
6. The lightweight field wheat ear detection method based on the improved RTDETR according to claim 1, characterized in that: In step (2), the matching-aware loss function modifies the target label of the positive sample to... And abandon manually set balance parameters Based directly on the matching quality score of This enables differentiated learning, thereby simplifying hyperparameter tuning and enhancing optimization for low-quality matching samples; The formula for the matching-perceptual loss function is expressed as follows: ； In the formula, Predict scores for the model; The quality score for the match between the predicted bounding box and the ground truth target; The value is 1 for positive samples and 0 for negative samples. The modulation coefficient is used to control the balance between easy and difficult samples; To match quality scores The power-modulated weights are used to dynamically adjust the learning intensity of positive samples based on the matching quality. For the model to predict scores The power modulation factor is used to suppress the dominant role of easily classified samples in the loss.
7. An electronic device, comprising: processor; as well as A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the lightweight field wheat ear detection method based on the improved RTDETR as described in any one of claims 1-6.
8. A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the lightweight field wheat ear detection method based on the improved RTDETR as described in any one of claims 1-6.