Pseudo-ginseng flower maturity detection method based on improved RT-DETR model
By improving the RT-DETR model and utilizing CSPNet, EfficientViMBlock, and DynamicPosBias-Attention mechanisms, the problems of small targets being easily missed and branches and leaves obscuring the flowers in Panax notoginseng flower detection were solved, achieving efficient and accurate Panax notoginseng flower maturity detection, and supporting intelligent harvesting robots and the upgrading of the Panax notoginseng industry.
Patent Information
- Application Number
- CN202511260629.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-04
- Publication Date
- 2025-12-12
AI Technical Summary
Existing technologies for detecting Panax notoginseng flowers in natural field environments suffer from problems such as easy omission of small targets, misjudgment due to obstruction by branches and leaves, and difficulty in distinguishing between the flowering period and the seed-setting period. These issues result in insufficient detection accuracy, making it difficult to meet the needs of intelligent harvesting robots and the intelligent upgrading of the Panax notoginseng industry.
Based on the improved RT-DETR model, CSPNet is used to replace the original backbone network in the backbone network. EfficientViMBlock, Content-Guided Attention Fusion module and CrossFormer's DynamicPosBias-Attention mechanism are introduced to optimize the model structure and training mechanism, thereby improving the detection accuracy of small objects and in complex backgrounds.
It significantly reduced the rates of missed and false detections, improved detection speed and overall accuracy, and enabled precise positioning and classification of Panax notoginseng flowers at different growth stages in complex field scenarios, meeting the needs of intelligent harvesting robots and the intelligent upgrading of the Panax notoginseng industry.
Smart Images

Figure CN121121267A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of agricultural intelligent detection and deep learning applications, and in particular to a method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model. Background Technology
[0002] Panax notoginseng, a traditional and rare medicinal plant, has a significantly higher content of saponins in its flower spikes than in its roots, stems, and leaves, thus possessing high medicinal value. However, its medicinal value varies at different growth stages (flowering, seed formation, and seed harvesting). Accurately identifying the maturity of Panax notoginseng flowers is of great significance for the development of harvesting robots and the upgrading of the industrial chain.
[0003] However, conducting detection in natural field environments still faces multiple challenges: First, field lighting conditions vary significantly, and the color of leaves and branches is similar to that of flowers, easily interfering with feature extraction; second, Panax notoginseng flowers are small (approximately 1–3 cm in diameter), often obscured by branches and leaves or exhibiting clustering and overlap, increasing the risk of missed detection; third, the morphological differences between the flowering and seed-setting stages are subtle, making accurate differentiation difficult using traditional methods. These factors lead to insufficient detection accuracy, limiting automated applications.
[0004] Existing methods for detecting crop maturity mainly fall into two categories: traditional machine learning and deep learning. While single-stage models such as the YOLO series have fast inference speeds, their detection accuracy is limited in scenarios involving small targets, occlusion, and overlap, making it difficult to balance speed and accuracy. Two-stage models such as Faster R-CNN have higher detection accuracy, but their complex structure, high computational cost, and insufficient real-time performance make them unsuitable for agricultural field applications.
[0005] Therefore, there is an urgent need for an efficient, accurate, and robust detection scheme capable of precisely locating and classifying different growth stages of Panax notoginseng flowers in natural settings. Based on this, this invention improves and optimizes the RT-DETR r18 model, effectively addressing issues such as small target detection, occlusion, and differentiation of easily confused stages. It provides a solution for Panax notoginseng flower maturity detection that balances accuracy and speed, and offers core technical support for intelligent harvesting robots and the intelligent upgrading of the Panax notoginseng industry. Summary of the Invention
[0006] This invention addresses the problems in the detection of Panax notoginseng flowers in natural environments, such as the easy omission of small targets, misjudgment caused by foliage obstruction, and difficulty in distinguishing between the flowering and seed-setting stages. It proposes a Panax notoginseng flower maturity detection method based on an improved RT-DETR model. Through optimization of the model structure and training mechanism, this method can more accurately identify and locate different growth stages of Panax notoginseng flowers in complex field scenarios, effectively reducing the missed detection and false detection rates, while improving detection speed and overall accuracy. This meets the practical needs of intelligent harvesting robots in the application of Panax notoginseng flower maturity discrimination and classification.
[0007] This invention provides a method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model, comprising the following steps:
[0008] Step (1) Collect images of Panax notoginseng flowers at different growth stages in natural environment and create a dataset;
[0009] Step (2) Optimize the RT-DETR model to form an improved RT-DETR model;
[0010] Step (3) Train the improved RT-DETR model;
[0011] Step (4) Deploy the model to the mobile device to obtain the detection results.
[0012] The dataset was created as follows: Using a DSLR camera in a Panax notoginseng plantation in Yunnan, images were taken during the natural light cycle from 10:00 AM to 8:00 PM. Three typical shooting angles were used: upward (lens elevation 15°-30°), eye-level (horizontal view), and downward (45°-60°). Macro shots were taken at a distance of 15-40cm, under different lighting conditions (strong light: direct midday sunlight; normal light: morning diffused light; weak light: evening oblique sunlight; dim light: shaded light). The collected images cover three growth stages of Panax notoginseng flowers: flowering stage (petals fully open), seed formation stage (green seeds appearing), and harvesting stage (red seeds mature). A Panax notoginseng flower maturity detection dataset containing 2000 original samples was ultimately constructed. To balance model training efficiency and feature integrity, bilinear interpolation was used to uniformly scale all images to a resolution of 640×640 pixels. This size can preserve the detailed features of the flowers (the pixel ratio of each flower is ≥5%), while also meeting the memory optimization requirements of GPU batch computing.
[0013] The dataset creation also includes expanding the dataset using data augmentation techniques. In addition to classic data augmentation methods such as horizontal and vertical flipping and random translation, random color enhancement was performed on the images, including noise addition and brightness and contrast adjustments, to further improve the model's robustness. After data augmentation, the images were randomly divided into training, validation, and test sets in an 8:1:1 ratio, combined with the original data. The training and validation sets were used for single-round model training and result evaluation, while the test set was used for the final evaluation of the model's recognition performance.
[0014] The optimization and improvement of the RT-DETR model to form the improved RT-DETR model includes: replacing the original backbone network with CSPNet in the RT-DETR backbone network; and replacing the traditional convolutional modules with EfficientViMBlock.
[0015] The basic idea of CSPNet is to split the feature maps of the network at each stage, which can be represented as follows:
[0016] x0 = [x′0, x″0]
[0017] By segmenting feature maps, CSPNet reduces the computational cost of each layer and balances the computational tasks, effectively improving the utilization of computing resources. Because CSPNet effectively reduces computational cost, it can achieve accuracy comparable to or even higher than traditional models at a lower computational cost. Especially in complex backgrounds, CSPNet's cross-stage feature fusion strategy reduces background interference while strengthening the spatial relationship modeling between multiple targets, improving detection accuracy. For complex backgrounds and spatial relationships between targets in Panax notoginseng flower images, CSPNet improves its learning ability under these complex conditions by reducing redundant computations and optimizing gradient information flow. CSPNet's design allows it to reduce computational and memory consumption while maintaining or improving detection accuracy, meeting the needs of real-time detection. Whether on edge devices, mobile devices, or embedded devices, CSPNet provides efficient inference performance, ensuring real-time and accurate target detection in complex scenes.
[0018] Traditional convolutional neural networks (CNNs) can be susceptible to background noise, making it difficult to accurately distinguish targets. EfficientViMBlock addresses this by introducing an HSM-SSD layer, employing a global information aggregation approach based on hidden states to shift computation from the traditional feature map space to a compressed hidden state space. This design significantly reduces computational costs and, while maintaining global dependencies, enables the network to more accurately focus on target regions in complex backgrounds, effectively suppressing background noise. Specifically, the HSM-SSD layer performs linear transformations and weighting operations on the input features, aggregating information in the hidden state space to obtain h, expressed by the following formula:
[0019]
[0020] Compared to traditional self-attention mechanisms, EfficientViMBlock employs a simplified global token mixing approach, reducing computational complexity while enhancing its ability to model spatial relationships between targets. This mechanism can be expressed as follows: First, the network performs a weighted summation of tokens at different locations to obtain the hidden state h. Then, a shared global hidden state h is used to model the relationships between targets. The specific process can be described by the following formula:
[0021] x out =f(Ch⊙σ(W) z h))
[0022] Furthermore, EfficientViMBlock introduces a multi-stage hidden state fusion (MSF) technique, which further enhances the network's representation capabilities by fusing predicted logits from different stages. Specifically, for the hidden state h at each stage... (s) Calculate its global representation And the final prediction result is obtained by weighted summation:
[0023]
[0024] EfficientViMBlock enhances detection capabilities in multi-target scenarios and complex backgrounds by combining HSM-SSD layers and multi-stage hidden state fusion, while optimizing computational efficiency and inference speed.
[0025] The optimization and improvement of the RT-DETR model to form the improved RT-DETR model also includes: introducing a Content-Guided Attention Fusion module into the neck network of the RT-DETR.
[0026] CGA Fusion aims to enhance the ability of multi-level feature fusion, thereby improving the model's performance in complex backgrounds and multi-object detection. Traditional feature fusion methods (such as addition or concatenation) often fail to effectively address the mismatch in receptive domains between low-level and high-level features, leading to information loss or redundancy. CGA Fusion, by calculating content-guided spatial weighting coefficients, dynamically adjusts the fusion ratio of low-level and high-level features based on the characteristics of each channel, thus preserving more effective information during the fusion process. This mechanism precisely controls the combination of low-level features (such as edges and textures) and high-level features (such as contextual information) by generating spatial weighting coefficients, avoiding information loss in traditional methods. To address the vanishing gradient problem and accelerate network learning, CGA Fusion also introduces skip connections, directly adding input features to the fused features, ensuring information flow and improving network learning efficiency. By introducing 1×1 convolutional layers to project the fused features, a more accurate and lower-dimensional feature representation is obtained, further optimizing computational and memory overhead, as shown in the following formula:
[0027] F fuse =C 1×1 (F low ·W+F high ·(1-W)+F low +F high )
[0028] In the improved RTDETR, CGA Fusion not only enhances the network's feature fusion capabilities, enabling it to utilize low-level and high-level features more efficiently, but also significantly improves the model's robustness and accuracy in multi-object detection tasks. Because this mechanism intelligently adjusts the feature fusion ratio, the model exhibits stronger adaptability when handling complex backgrounds, while reducing computational complexity and improving inference speed and response efficiency in real-time detection tasks.
[0029] The optimization and improvement of the RT-DETR model to form the improved RT-DETR model further includes: Based on the RT-DETR model, to improve the model's detection accuracy in scenarios with small targets, occlusion, and overlapping targets, we optimized the AIFI module by adopting the DynamicPosBias-Attention mechanism from CrossFormer. This mechanism dynamically adjusts the relative position bias, enabling the model to better handle the detection of multi-scale targets, especially in complex scenarios with small targets and occluded targets, significantly improving the model's detection performance.
[0030] The DynamicPositionBias (DPB) module introduced by CrossFormer, as a novel positional coding method, overcomes the limitations of traditional Relative Position Bias (RPB) on fixed image sizes. Unlike RPB with a fixed image size, DPB can dynamically calculate the positional bias based on changes in image or group size, providing greater adaptability and flexibility, especially excelling when processing variable-sized input images.
[0031] The core idea of the DPB module is to dynamically generate the relative positional bias between two embeddings using a multilayer perceptron (MLP) network. Specifically, DPB calculates the bias based on the relative distance Δx between the embeddings. ij and Δy ij A bias matrix is generated, which is added to the self-attention calculation to optimize the model's spatial awareness, as shown in the following formula:
[0032] B i,j =DPB(Δx) ij ,Δy ij )
[0033] Where Δx ij and Δy ij This represents the relative coordinate distance between two embeddings, and the DPB module uses this distance to calculate the relative position offset.
[0034] Compared to the traditional relative position bias (RPB), DPB has the following advantages:
[0035] Flexibility: DPB can adapt to different image sizes or group sizes, especially in the detection of multi-scale targets, and can avoid the performance bottleneck caused by fixed-position encoding.
[0036] High efficiency: DPB can effectively improve the model's target recognition ability in complex backgrounds and multi-scale environments by optimizing the calculation of relative positions. Especially in dense prediction tasks (such as target detection and instance segmentation), DPB can significantly improve detection accuracy compared with traditional methods.
[0037] By introducing the DPB mechanism into the AIFI module, the model can more accurately model the spatial relationship between small targets and occluded targets, enhancing its ability to recognize targets at different scales. Combined with the Cross-Scale Embedding Layer (CEL) mechanism in CrossFormer, DPB further optimizes the model's perception of spatial features, enabling the AIFI module to better handle target detection tasks in complex environments.
[0038] In CrossFormer, DPB is combined with a self-attention mechanism, and the calculation method is as follows:
[0039]
[0040] Here, B is the relative position bias matrix generated by the DPB module, which dynamically adjusts the relative positions between targets. Through this optimization, the target detection accuracy of the model is significantly improved under different scales and complex backgrounds.
[0041] Experimental results show that the RT-DETR model using DPB can more accurately locate Panax notoginseng flowers at different growth stages in the maturity detection task compared with traditional methods, especially in complex scenarios with small targets, occlusion, and target overlap, significantly improving detection accuracy.
[0042] The improved RT-DETR model obtained after training improvements refers to the following: Image resolution was compressed to 640×640 pixels to ensure model training speed; the GPU was GeForce RTX 4090; Python version was 3.11; CUDA version was 11.8; and the deep learning framework was PyTorch 2.0.0. The RT-DETR r18 model was used for experimentation, with a depth factor of 1, a width factor of 1, and a maximum number of channels of 1024. SGD was used as the optimizer, combined with a cosine decay strategy to adjust the learning rate; the initial learning rate was 0.01; a total of 200 training iterations were performed; the batch size was 8; and the momentum was set to 0.937. During training, no pre-trained weights were used in any network, and data augmentation methods such as horizontal and vertical flipping and noise addition were used. The original RT-DETR model and the improved RT-DETR model were trained and optimized using the above training parameters, and the training results were compared to draw conclusions.
[0043] The deployment of the model to mobile devices to obtain detection results refers to: evaluating the robustness and generalization of the model on a self-built Panax notoginseng flower maturity detection dataset, performing lightweight optimization of the RT-DETR model, and deploying it on edge computing devices such as Jetson Nano to test the network's inference efficiency and memory usage under the ARM architecture. Images of Panax notoginseng flowers are captured in real-time using the camera on mobile devices (such as Android tablets) and transmitted via Wi-Fi to the edge device where the model is deployed for real-time detection. The detection results are displayed in a visual interface (labeling flower location, maturity category, and confidence level), and the results (including images, category labels, and timestamps) are stored in the device's local database. This deployment scheme enables accurate positioning and classification of Panax notoginseng flower growth stages in natural environments, providing real-time visual decision support for intelligent harvesting robots.
[0044] Compared with existing technologies, the beneficial effects of adopting the above technical solution are as follows:
[0045] (1) The RT-DETR model is improved in multiple dimensions to address the complex background and spatial relationships between targets in Panax notoginseng flower images: CSPNet is used to replace the original backbone network in the RT-DETR backbone network. The computational cost is reduced by cross-stage feature fusion, while the ability to model the spatial relationships between multiple targets is improved. It is particularly suitable for Panax notoginseng flower target detection in complex backgrounds. EfficientViMBlock is used to replace the traditional convolution module in the convolution module. The computational cost is significantly reduced and the focusing ability of the target region in the background is improved by the global information aggregation method based on the hidden state, thus optimizing the accuracy of multi-target detection. The CGA Fusion module is introduced into the neck network. By calculating the content-guided spatial weighting coefficient, the fusion ratio of low-level and high-level features is dynamically adjusted, making the multi-level feature fusion more accurate. This further improves the accuracy of multi-target detection in complex backgrounds, reduces computational complexity, and improves the inference speed and response efficiency in real-time detection tasks. By introducing the DPB mechanism into the AIFI module, the model can more accurately model the spatial relationship between small targets and occluded targets, enhancing the ability to recognize targets of different scales. By combining the Cross-Scale Embedding Layer (CEL) mechanism in CrossFormer, DPB further optimizes the model's perception of spatial features, enabling the AIFI module to better handle target detection tasks in complex environments.
[0046] (2) The improved RT-DETR model and deployment scheme proposed in this invention demonstrate outstanding application value and industrial application potential. First, by adopting a lightweight structural design and a cross-scale attention mechanism, the model complexity and runtime latency are significantly reduced while maintaining high mAP. Similar to the excellent performance achieved by lightweight detection networks such as EdgeYOLO and EDNet on Jetson series devices, this model is optimized to adapt to edge platforms such as JetsonNano, enabling high frame rate real-time detection through local inference. Second, the model can be seamlessly deployed on mobile terminals, such as Android tablets, to collect images through the terminal and transmit the data to the edge device via Wi-Fi, then perform visualization detection and storage, thereby constructing a complete visual picking ecosystem. Attached Figure Description
[0047] Figure 1 This is an overall flowchart of a method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model provided in the embodiments of this application;
[0048] Figure 2 This is a dataset of Panax notoginseng flowers under different light conditions in the embodiments of this application;
[0049] Figure 3This is a schematic diagram of the EfficientViMBlock module in this application;
[0050] Figure 4 This is a schematic diagram of the CGA Fusion module in this application;
[0051] Figure 5 This is a schematic diagram of the DPB module in this application. Detailed Implementation
[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The examples provided are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments described, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0053] See Figure 1 The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model provided in this embodiment of the invention specifically includes the following steps:
[0054] Step (1): Collect images of Panax notoginseng flowers during their growth stages and create a dataset.
[0055] First, images were collected in the natural environment of a Panax notoginseng plantation in Yunnan Province from 10:00 AM to 4:00 PM, covering various lighting conditions including strong light, normal light, weak light, and low light. Shooting angles included upward, eye-level, and downward views, with shooting distances controlled within the range of 15-40 cm to ensure coverage of the three different growth stages: flowering, seed formation, and seed harvesting. To accurately reflect the natural planting environment, we manually screened and removed blurry images caused by focusing issues and abnormal samples caused by human interference. To improve the stability and accuracy of model training, we specifically selected clear images under different angles, distances, and lighting conditions to ensure sufficient diversity in the dataset.
[0056] Next, all images were uniformly scaled to 640×640 pixels to suit the needs of model training. Data augmentation of the training set was performed using Python tools, employing classic techniques such as horizontal flipping, scaling, rotation, noise addition, contrast adjustment, and brightness adjustment, expanding the training set from 1600 images to 4800.
[0057] Finally, manual labeling was performed using the LabelImg tool, labeling the flowering period as "f", the seed-setting period (green seeds) as "g", and the harvesting period (red seeds) as "s". The label frame is the smallest bounding rectangle encompassing the entire Panax notoginseng flower, and a corresponding txt file is generated, containing the category, center coordinates, and width and height information. See also... Figure 2The image shown is a sample image of Panax notoginseng flower dataset collected under different lighting conditions.
[0058] Step (2): Optimize and improve the RT-DETRr18 model to form an improved RT-DETR model.
[0059] First, the improved model replaces the original backbone network with the CSPNet network. The design philosophy of CSPNet is to segment feature maps, reducing the computational cost per layer and optimizing the use of computational resources. The basic representation of feature map segmentation is as follows:
[0060] x0 = [x′0, x″0]
[0061] By segmenting feature maps, CSPNet can significantly reduce computational costs while maintaining model accuracy. Especially in complex backgrounds, cross-stage feature fusion can effectively reduce background interference and enhance the modeling ability of spatial relationships between targets, thereby improving detection accuracy. CSPNet's advantages enable it to provide more efficient processing capabilities for complex backgrounds in Panax notoginseng flower images without increasing computational resource consumption, ensuring the need for real-time detection.
[0062] Secondly, the original convolutional modules are replaced by EfficientViMBlock, which optimizes the computation process by introducing HSM-SSD layers. Traditional convolutional networks struggle to distinguish between targets and backgrounds, while EfficientViMBlock employs a global information aggregation method based on hidden states. This effectively shifts computation from the traditional feature map space to a lower-dimensional hidden state space, reducing computational cost and enhancing focus on target regions. (See [link to relevant documentation]). Figure 3 The diagram shown is a structural schematic of the EfficientViMBlock module in this application, expressed by the formula:
[0063] x out =f(Ch⊙σ(W) z h))
[0064] By combining multi-stage hidden state fusion (MSF) technology, the model can further improve the performance of multi-object detection by fusing the prediction results from different stages.
[0065] Next, the neck network of RT-DETR introduces a Content-Guided Attention Fusion (CGAFusion) module to improve the fusion capability of multi-level features. See also Figure 4 The diagram shown is a structural schematic of the CGA Fusion module in this application. CGAFusion dynamically adjusts the fusion ratio of low-level and high-level features, avoiding information loss in traditional methods. The fusion of low-level and high-level features uses the following method:
[0066] F fuse =C 1×1 (F low ·W+F high ·(1-W)+F low +F high )
[0067] In complex contexts, CGAFusion allows the model to focus more on effective information, improving the accuracy of object detection and reducing computational overhead. The computational cost formula is as follows:
[0068]
[0069] Finally, to address the insufficient spatial relationship modeling of small targets, occluded objects, and overlapping samples in detection, the model introduces the DynamicPosBias-Attention mechanism proposed by CrossFormer in the AIFI module. This mechanism compensates for the shortcomings of traditional fixed-position bias in complex scenes by dynamically generating relative position biases. Dynamic PositionBias (DPB) uses a multilayer perceptron (MLP) to model the relative positions (Δx, Δy) between feature points and generates a bias matrix that is added to the attention calculation, thus providing stronger adaptability in spatial modeling. In this way, the model can dynamically adjust the attention distribution according to the input features, paying more attention to the feature interactions of small targets and occluded targets, and avoiding the problem of small-scale target information loss caused by fixed-position encoding. See also Figure 5 The diagram shown is a structural schematic of the DPB module in this application.
[0070] Compared to the traditional AIFI module, DPB-Attention can effectively model the spatial relationships between targets at different scales and in complex backgrounds, improving the model's robustness and detection accuracy in complex scenes. In the Panax notoginseng flower maturity detection task, this mechanism can significantly improve the model's ability to identify flowers at different growth stages, especially when small targets and overlapping targets appear densely, demonstrating stronger detection performance.
[0071] The formula for calculating DynamicPosBias-Attention is as follows:
[0072] B i,j =DPB(Δx) i,j ,Δy i,j )
[0073]
[0074] Through this improvement, the RT-DETR model achieves a balance between accuracy and computational efficiency, while significantly improving detection performance in complex field environments, especially in scenarios with small targets, occlusion, and overlapping samples, effectively enhancing the accuracy of target detection.
[0075] Step (3): Train the improved RT-DETR model.
[0076] The model was trained on a system running Windows 11, using an NVIDIA RTX 4090 graphics card, PyCharm as the development environment, and PyTorch as the deep learning framework. The initial learning rate was set to 0.01, and the RT-DETRr18 model was used for experimentation. The selected model had a depth factor of 1, a width factor of 1, and a maximum number of channels of 1024. SGD was used as the optimizer, combined with a cosine decay strategy to adjust the learning rate. The initial learning rate was 0.01, and the training iterations totaled 200, with a batch size of 8 and a momentum of 0.937. During training, no pre-trained weights were used in any network, and data augmentation methods such as horizontal flipping, scaling, rotation, noise addition, contrast adjustment, and brightness adjustment were employed.
[0077] To objectively evaluate algorithm performance, precision, recall, mean precision (mAP), frame rate (FPS), and computational cost (GFLOPs) are used as evaluation metrics. Precision and recall are defined as follows:
[0078]
[0079] TP, FP, and FN represent true positive, false positive, and false negative, respectively.
[0080] Real-time performance is measured in frame rate and converted to total latency per frame (milliseconds):
[0081]
[0082] Among them, T pre T represents the image preprocessing time. infer T represents the model inference time. nms This refers to post-processing time.
[0083] The key parameter for evaluating the model is mAP, which is calculated as follows:
[0084]
[0085] Wherein, mAP50 represents the average accuracy of each category at an IoU threshold of 0.50, and mAP50:95 represents the average detection accuracy at 10 thresholds with an IoU threshold ranging from 0.50 to 0.95 and a step size of 0.05.
[0086] Step (4) Deploy the model to the mobile device to obtain the detection results.
[0087] To further verify the feasibility of the model in practical applications, robustness and generalization tests were conducted on a self-built Panax notoginseng flower maturity detection dataset (containing complex samples such as strong light interference and target occlusion) and publicly available flower datasets. Subsequently, the improved RT-DETR model was deployed to embedded terminals such as Jetson Nano (ARM architecture) to evaluate its inference speed and storage consumption under low-power hardware conditions, in order to verify the model's portability and lightweight effect.
[0088] Building upon this foundation, the model was also deployed on mobile devices such as Android tablets, using the terminal's camera to capture images of Panax notoginseng flowers in real time and transmitting them to edge devices via wireless network for detection. The detection results are output in a visual interface, displaying the flower's location, maturity category, and confidence level. Simultaneously, information such as images, category labels, and timestamps are recorded and saved to a local database, enabling traceability and management of the detection results.
[0089] The proposed RT-DETR method for detecting the maturity of Panax notoginseng flowers addresses the challenges of small target omissions, occlusion-related false detections, and difficulties in stage classification in natural environments. It effectively optimizes this method by replacing the original RT-DETR network backbone with CSPNet, significantly reducing redundant computation. The traditional convolutional modules are replaced with EfficientViMBlock, improving multi-target detection performance and enhancing the model's robustness and adaptability. The neck network incorporates the CGAFusion module, significantly improving multi-level feature fusion capabilities. Furthermore, the AIFI module utilizes the DynamicPosBias-Attention mechanism proposed by CrossFormer, exhibiting stronger detection performance when small and overlapping targets are densely distributed.
[0090] This detection solution not only enables real-time acquisition of images of Panax notoginseng flowers in natural environments and the display and storage of detection results, but also allows for remote, contactless large-scale plantation inspections by combining drones and other mechanical equipment, significantly improving the efficiency and intelligence level of Panax notoginseng flower maturity detection.
Claims
1. A method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model, characterized in that, Includes the following steps: Step (1) Collect images of Panax notoginseng flowers at different maturity levels and create a dataset; Step (2) optimizes and improves the RT-DETR model to form an improved RT-DETR model; Step (3) Train the improved RT-DETR model; Step (4) Deploy the model to the mobile device to obtain the detection results.
2. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, Using a camera, between 10:00 AM and 8:00 PM, take clear images of different growth stages of Panax notoginseng flowers, namely the flowering period, seed-setting period, and seed-harvesting period, from different angles such as looking up, looking at eye level, and looking down, within a shooting distance of 15-40cm, and under various lighting conditions such as strong light, normal light, weak light, and dark light. A Panax notoginseng flower maturity detection dataset was also created.
3. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, After acquiring images of Panax notoginseng flowers in their natural environment, this study employed various data augmentation methods, including horizontal flipping, scaling, and rotation, to expand the data. Simultaneously, color enhancement was performed, encompassing noise addition, brightness and contrast adjustments, to improve the model's robustness. Subsequently, the LabelImg tool was used to annotate the images: "f" for the flowering period, "g" for green seeds during the seed-setting period, and "s" for red seeds during the harvest period. The annotation boxes used the smallest bounding rectangle encompassing the entire Panax notoginseng flower to minimize background interference. Finally, the dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio.
4. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, The RT-DETR model was optimized and improved by replacing the original backbone network with CSPNet. Based on CSPNet, the traditional convolutional modules were replaced with EfficientViMBlock as the core module. By performing channel mixing operations on the compressed hidden states, the computational complexity was significantly reduced, enabling the network to extract spatial features more efficiently. This also enhanced the model's ability to extract Panax notoginseng flower features in complex backgrounds, allowing for more accurate differentiation of Panax notoginseng flowers at different flowering stages. This resulted in a more stable and efficient model performance in practical applications.
5. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 4, characterized in that, Further optimizations to the RT-DETR model resulted in an improved RT-DETR model. These improvements included: introducing a Content-Guided Attention Fusion module into the neck network of RT-DETR, which guides the model to focus on the most critical regions in the feature maps by assigning a unique Spatial Importance Map (SIM) to each channel; combining FPN (Feature Pyramid Network) and PAN (Path Aggregation Network) structures for feature fusion, fusing low-level feature maps with high-level features through upsampling operations to achieve more refined target detection; and introducing a CGAF module that adjusts feature interactions by calculating spatial weights during feature fusion, effectively alleviating the receptive field mismatch problem in traditional methods and enhancing the detection capability for small targets such as Panax notoginseng flowers.
6. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, The improved RT-DETR model further optimizes the AIFI module by introducing the DynamicPosBias-Attention mechanism from CrossFormer to enhance its target detection capabilities, particularly in complex environments with small targets, severe occlusion, and overlap. Specifically, DynamicPosBias-Attention optimizes the spatial focusing ability of the self-attention mechanism by introducing a dynamic positional bias attention mechanism. This allows the model to more effectively capture key information about small and partially occluded targets in complex scenes, thus improving detection accuracy. Compared to the traditional AIFI module, DynamicPosBias-Attention better handles spatial relationships between targets, especially in cases of significant occlusion and overlap, further enhancing the model's learning ability and robustness. Therefore, using the improved AIFI module, the model exhibits stronger detection performance in the growth stage detection task of Panax notoginseng flowers, more accurately identifying difficult-to-distinguish targets, with a significant improvement in detection accuracy, especially in complex environments.
7. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, The improved RT-DETR model was trained as follows: image resolution was compressed to 640×640 pixels to ensure training speed; the GPU was GeForce RTX 4090, Python version 3.11, CUDA version 11.8, and the deep learning framework was PyTorch 2.0.0; the RT-DETR r18 model was used for experimentation, with a depth factor of 1, a width factor of 1, and a maximum number of channels of 1024; SGD was used as the optimizer, combined with a cosine decay strategy to adjust the learning rate; the initial learning rate was 0.01, with a total of 200 training iterations, a batch size of 8, and a momentum of 0.937; during training, no pre-trained weights were used in any network, and horizontal flipping, scaling, rotation, noise addition, contrast adjustment, and brightness adjustment were used to augment the images and label files; the original RT-DETR model and the improved RT-DETR model were trained and optimized using the above training parameters, and the training results were compared to draw conclusions.
8. The method for detecting the maturity of Panax notoginseng flowers based on an improved RT-DETR model according to claim 1, characterized in that, The method also includes: evaluating the robustness and generalization performance of the trained improved RT-DETR model on a self-built Panax notoginseng flower maturity detection dataset, and further optimizing the network structure by combining lightweight design; deploying the model on a mobile terminal device to achieve real-time localization and classification detection of Panax notoginseng flower growth stages, and using the obtained detection results as input to the core vision algorithm of the harvesting robot to provide technical support for automated harvesting of Panax notoginseng flowers.
Citation Information
Cited By
Model dynamic combination-based complex scene target detection method
CN121459137A
