Chinese herbal medicine plant recognition and detection method based on improved YOLO11n

Abstract

A Chinese herbal medicine plant recognition method based on improved YOLOv1n is proposed. A reparameterization heterogeneous multi-scale module is constructed in the main network, which significantly enhances the model's ability to represent multi-scale morphological features of plants. An efficient up-sampling convolution module is introduced in the neck network, effectively suppressing the loss of spatial information during feature transmission and achieving more accurate target boundary positioning. A multi-scale convolution attention mechanism is integrated before the detection head, which effectively suppresses complex background noise in the wild by widening the receptive field. Experimental results show that the mAP50 of REM-YOLO on the Chinese herbal medicine dataset reaches 85.7%, which is 1.5% higher than the original YOLOv1n. At the same time, the computational complexity is compressed to 5.3 GFLOPs, which is 28.4% lower than YOLOv8n, and the inference speed is greatly increased to 198.2 fps. This algorithm maintains real-time detection speed while showing strong robustness and generalization ability, providing an efficient and reliable technical solution for automatic recognition and resource investigation of Chinese herbal medicine.

Description

Technical Field

[0001] This invention relates to the field of Chinese herbal medicine plant identification technology, specifically a Chinese herbal medicine plant identification and detection method based on an improved YOLO11n. Background Technology

[0002] Because the visual characteristics of the leaves of living medicinal plants in their natural habitats, such as morphology, color, and texture, are easily confused, traditional manual identification methods are often ineffective. Machine learning algorithms can be used to mine the apparent geometry and texture information of medicinal plants. Kaur et al. explored the feasibility of using machine learning to extract macroscopic physical features such as leaf shape, texture, and veins for automated plant identification, but this method relies on manually designed physical features such as leaf shape and veins. Ibrahim et al. further compared the performance of HOG, LBP, and SURF texture features combined with multi-class support vector machines, confirming the high robustness of manually explicit local feature descriptors in leaf identification. However, this method struggles to extract stable and continuous descriptions when faced with actual growth environments with dense, accompanying wild weeds.

[0003] With the development of deep learning, automatic feature extraction based on neural networks has gradually become mainstream. Zhao et al. proposed a part-priority attention-guided CNN model for complex field backgrounds, achieving more accurate fine-grained classification through adaptive localization and feature enhancement of medicinal plant organs. Vani et al. proposed a dual-modal deep learning architecture combining image analysis and natural language processing. By mining multi-scale visual features of leaves through HFEN and fusing botanical text descriptions with CCIN, they achieved high-precision multimodal classification of Chinese herbal medicines, but still could not handle multi-target localization in complex field scenarios. Vo et al. proposed an improved recognition algorithm combining VGG16 and LightGBM, which automatically extracts implicit features using deep residual blocks, achieving high-precision species classification in natural environments, but still could not solve the complex bounding box localization and detection needs of multiple intertwined plants and multiple parallel targets in actual wild environments.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the above-mentioned technical defects and provide a Chinese herbal medicine plant identification and detection method based on the improved YOLO11n. It introduces the reparameterized heterogeneous multi-scale module RepHMS to enhance the integration of multi-scale features, and introduces the efficient upsampling convolution module EUCB and the multi-scale convolutional attention module MSCA, aiming to enhance the network's ability to perceive key semantic regions and optimize the spatial information recovery in the feature transfer process.

[0006] To address the aforementioned problems, the technical solution of this invention is a method for identifying and detecting Chinese herbal medicine plants based on an improved YOLO11n, comprising the following steps: Step 1: Introduce the Reparameterized Heterogeneous Multiscale Module (RepHMS) to replace the original C3k2 structure. This module achieves efficient extraction and fusion of feature information at different scales through a multi-branch cascade structure and a reparameterized convolution mechanism, thereby improving the feature representation capability of the network while ensuring inference efficiency. Step 2: Introduce an upsampled convolutional module (EUCB) into the neck network to replace the traditional upsampled operator. EUCB achieves high-quality spatial feature reconstruction while reducing computational redundancy through lightweight multi-scale modeling and channel refinement mechanism. Step 3: Integrate a multi-scale convolutional attention module (MSCA) before the detection layer. The MSCA simulates the large kernel convolution effect through lightweight deep bar convolution to obtain multi-scale contextual information. The MSCA module adopts a parallel multi-branch architecture to reconstruct the input features.

[0007] Preferably, in step one, the input features of the RepHMS module are first channel-adjusted through 1×1 convolution, and then segmentation is performed to divide the feature map into N parallel information streams. In the multi-branch architecture, the first branch directly retains the shallow feature information of the input to maintain spatial fidelity. Starting from the second branch, the input information is sequentially passed through M cascaded blocks to deeply mine semantic representation.

[0008] Preferably, in step one, RepHMS employs a reparameterization strategy. During the training phase, the module utilizes a multi-branch heterogeneous structure that includes 3×3 convolution, 1×1 convolution, and identity mapping to enhance learning capabilities. During the inference phase, the module leverages the fusionability of linear operators to fuse multiple parallel parameters into an equivalent single-path 3×3 convolution operator through mathematical transformations, thereby obtaining a wider receptive field while maintaining zero additional inference overhead.

[0009] Preferably, in step two, EUCB expands the spatial dimension of the input feature map by a factor of 2 through interpolation, completing the initial resolution restoration; the module uses 3×3 depthwise separable convolution to process the magnified feature map, accurately capturing the texture features of plant leaf edges and fine organs; 1×1 convolution is used to precisely adjust the number of channels, filtering out information redundancy while ensuring that the output features and subsequent fusion layers achieve precise dimensional matching. The EUCB module calculation process is as follows: In the formula, , Features of module input and output; This is an upsampling operation; For depthwise separable convolution; This is a normalization operation; For activation function, This is a compressed convolutional layer.

[0010] Preferably, in step three, the input features undergo a depthwise convolution for local information aggregation. One branch uses an identity mapping to preserve the original feature information, while the other three branches extract contextual features at different scales using depthwise strip convolutions with kernel sizes of 1×7 and 7×1, 1×11 and 11×1, and 1×21 and 21×1, respectively. The output features of each branch are fused variably, and channel information interaction and attention weight generation are completed through a 1×1 convolution, forming a multi-scale spatial attention map. The calculation process of MSCA is as follows: In the formula, , Features of module input and output; These are the attention weights calculated by the multi-scale convolutional attention module; This represents the stepwise matrix multiplication operation; For depthwise convolution; To indicate the first Each scale branch.

[0011] The advantages of this invention compared to existing technologies are: 1. This invention introduces a reparameterized heterogeneous multiscale module (RepHMS) into the backbone network. By fusing heterogeneous multiscale convolutional kernels through reparameterization technology, the network’s ability to extract subtle morphological features of plants and its spatial receptive field are significantly improved without increasing inference energy consumption.

[0012] 2. In the feature fusion stage, this invention uses an efficient upsampling convolutional module (EUCB) instead of traditional upsampling. This module utilizes multi-scale depthwise separable convolution and gated attention mechanisms to achieve high-quality spatial feature recovery with extremely low computational cost, effectively enhancing the recognition of target plants against a weedy background.

[0013] 3. This invention integrates a multi-scale convolutional attention module (MSCA) in front of the detection head, capturing extensive contextual information through the attention mechanism of the convolutional structure. This module effectively addresses the problems of varying plant sizes and mutual occlusion through multi-scale feature interaction, improving the algorithm's localization accuracy in complex field environments. Attached Figure Description

[0014] Figure 1 This is a diagram of the REM-YOLO network structure of the present invention.

[0015] Figure 2 This is a structural diagram of the RepHMS module of the present invention.

[0016] Figure 3 This is a structural diagram of the EUCB module of the present invention.

[0017] Figure 4 This is a structural diagram of the MSCA module of the present invention.

[0018] Figure 5 This is a graph showing the REM-YOLO detection results of the present invention.

[0019] Figure 6 This is a comparison diagram of the model heat map of the present invention. Detailed Implementation

[0020] To make the content of this invention easier to understand, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings.

[0021] like Figure 1-4 As shown, a method for identifying and detecting Chinese herbal medicine plants based on an improved YOLO11n includes the following steps: Step 1: Introduce the Reparameterized Heterogeneous Multiscale Module (RepHMS) to replace the original C3k2 structure. This module achieves efficient extraction and fusion of feature information at different scales through a multi-branch cascade structure and a reparameterized convolution mechanism, thereby improving the feature representation capability of the network while ensuring inference efficiency.

[0022] The input features of the RepHMS module are first channel-adjusted through 1×1 convolutions, followed by segmentation, dividing the feature map into N parallel information streams. In the multi-branch architecture, the first branch directly retains the shallow feature information of the input to maintain spatial fidelity. Starting from the second branch, the input information is sequentially passed through M cascaded blocks to mine semantic representations at a deeper level. RepHMS introduces a cross-branch information stream mechanism, where each branch, while processing features independently, can utilize the shallow representations propagated from the previous branch across layers. This cascaded structure not only enhances the effective propagation of gradients but also significantly enriches the depth and diversity of feature learning. Finally, feature refinement and channel alignment are achieved through cascaded integration and 1×1 convolutions.

[0023] RepHMS employs a reparameterization strategy to improve detection accuracy while maintaining deployment efficiency for edge devices. During the training phase, the module utilizes a multi-branch heterogeneous structure that includes 3×3 convolution, 1×1 convolution, and identity mapping to enhance learning capabilities. During the inference phase, the module leverages the fusionability of linear operators to fuse multiple parallel parameters into an equivalent single-path 3×3 convolution operator through mathematical transformations, thereby obtaining a wider receptive field while maintaining zero additional inference overhead.

[0024] By flexibly adjusting hyperparameters M and N, RepHMS achieves hierarchical fusion from local texture to global context, effectively alleviating the problem of missed detection caused by drastic scale changes in the identification of Chinese herbal medicines. While significantly enhancing the ability to model fine-grained features, it maintains a very high lightweight advantage, providing robust feature support for plant detection in natural habitats.

[0025] Step 2: Introduce an upsampling convolution module (EUCB) into the neck network to replace the traditional upsampling operator. This addresses the problem of spatial information loss and detail blurring of Chinese herbal medicine plant features during the upsampling process in natural habitats. EUCB achieves high-quality spatial feature reconstruction while reducing computational redundancy through lightweight multi-scale modeling and channel refinement mechanisms.

[0026] EUCB expands the spatial dimension of the input feature map by a factor of 2 through interpolation, achieving initial resolution restoration. The module uses 3×3 depthwise separable convolutions to process the magnified feature map, accurately capturing the texture features of plant leaf edges and subtle organs. It utilizes batch normalization and ReLU activation function feature distribution, enhancing the network's non-linear expressive power to effectively suppress unstructured noise interference in complex field backgrounds. 1×1 convolutions are used to precisely adjust the number of channels, filtering out information redundancy while ensuring accurate dimensional matching between the output features and subsequent fusion layers. The EUCB module calculation process is as follows: In the formula, , Features of module input and output; This is an upsampling operation; For depthwise separable convolution; This is a normalization operation; For activation function, This is a compressed convolutional layer. Through the integration of this module, REM-YOLO significantly enhances the feature perception accuracy for scale-variable targets while maintaining its real-time inference advantages on edge devices.

[0027] Step 3: Integrate a multi-scale convolutional attention module (MSCA) before the detection layer to further enhance the model's ability to capture the salient features of Chinese herbal plants in complex field backgrounds. The MSCA simulates the large kernel convolution effect through lightweight deep strip convolution to obtain multi-scale contextual information. The MSCA overcomes the limitation of high computational cost of traditional self-attention mechanisms. The module adopts a parallel multi-branch architecture to reconstruct input features.

[0028] The input features are processed through a depthwise convolution for local information aggregation. One branch uses an identity mapping to preserve the original feature information, while the other three branches use depthwise strip convolutions with kernel sizes of 1×7 and 7×1, 1×11 and 11×1, and 1×21 and 21×1, respectively, to extract contextual features at different scales. This design significantly reduces the number of parameters and computational complexity of the model while maintaining a very wide receptive field by splitting the ultra-large kernel convolution into horizontal and vertical component operations. The output features of each branch are fused variably, and channel information interaction and attention weight generation are completed through 1×1 convolutions to form a multi-scale spatial attention map. The calculation process of MSCA is as follows: In the formula, , Features of module input and output; These are the attention weights calculated by the multi-scale convolutional attention module; This represents the stepwise matrix multiplication operation; For depthwise convolution; To indicate the first Multiple scale branches. By deploying MSCA in front of the detection head, REM-YOLO can more accurately focus on subtle visual differences between similar species at a fine-grained scale, providing more discriminative feature representations for target detection in complex natural scenes.

[0029] Experiments and Results

[0030] The experiments for this invention were conducted on the Ubuntu 22.04.4 LTS operating system. The hardware platform consisted of an Intel Xeon Gold 6133 processor and four NVIDIA GeForce RTX 4090 D GPUs (each with 24GB of VRAM). The software environment was based on the PyTorch 2.4.1 deep learning framework and the Python 3.8.20 programming language, with CUDA 12.1 used for hardware acceleration.

[0031] During model training, the input image size was uniformly adjusted to 640×640 pixels, the batch size was set to 64, and the total number of epochs was 250. The SGD optimizer was used for network optimization, with momentum and weight decay configured to 0.937 and 0.0005, respectively. The initial learning rate was set to 0.001, and a cosine annealing strategy was used to dynamically adjust the learning rate, eventually decaying it to 1% of its initial value.

[0032] Dataset description and evaluation metrics

[0033] This invention constructs an image dataset containing 10 common Chinese medicinal herbs. The species include the morphologically similar patchouli and mint, as well as agrimony, magnolia bark, lysimachia, pinellia, dandelion, dryopteris, astragalus, and cassia. The original images in the dataset are primarily sourced from the Plant Image Database of China (PPBC). To enhance the habitat diversity of the data, the astragalus category additionally includes field images collected from a planting base in Hohhot, Inner Mongolia.

[0034] The constructed dataset of Chinese herbal medicine plants contains 3115 original images. Before the experiment, the dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. To simulate the complex and diverse lighting environments, changing climate conditions, and differences in shooting angles in nature, this invention performed data augmentation on the training set, specifically including random rotation, random scaling, mosaic enhancement, HSV color space perturbation, and adaptive histogram equalization (AHE). Through these operations, the size of the training set was expanded to four times the original, resulting in a complete dataset of 10591 images. This dataset provides a solid data foundation for the model's fine-grained feature learning and robustness verification in natural habitats.

[0035] like Figure 5The results shown are those of REM-YOLO. To objectively evaluate the performance of the REM-YOLO model in the detection of Chinese herbal plants, this invention selects precision (P), recall (R), average precision (AP), and mean average precision (mAP) as core evaluation metrics.

[0036] Precision (P) represents the proportion of actual positive samples among those predicted as positive by the model; recall (R) represents the proportion of all actual positive samples successfully predicted by the model; AP is defined as the area under the PR curve, reflecting the model's detection stability for a single medicinal material category; mAP is the arithmetic mean of AP values ​​for all k categories, and is a key indicator for measuring the overall detection performance of the model. The calculation formula is as follows: Experimental results

[0037] To verify the specific contributions of the proposed RepHMS, EUCB, and MSCA modules to the performance of the YOLOv11 model, this invention designed and conducted ablation experiments under the same dataset and experimental environment. The experimental results are shown in Table 1.

[0038] Table 1 Ablation Experiment

[0039] By introducing the improved modules one by one, it can be seen that the RepHMS module enhances the network's multi-scale feature representation capability through a reparameterization mechanism, resulting in a steady increase in mAP50 to 84.5%. Adding the EUCB module alone, the model shows a significant gain in precision, reaching 87.2%, indicating its advantage in efficient feature reconstruction and contextual information fusion. The MSCA module utilizes a multi-scale spatial attention mechanism, enabling the model to focus more accurately on the target region, further boosting mAP50 to 85.2%. In the module combination experiment, the model integrating both RepHMS and EUCB showed a good synergistic effect, outperforming the single-module improvement scheme in all metrics. When all three modules are integrated into the improved model, the performance reaches its optimal state, with precision, recall, mAP50, and mAP50-95 reaching 87.9%, 74.7%, 85.7%, and 64.8%, respectively. Compared to the baseline model, the final improved scheme achieved a significant increase of 1.5% in mAP50 and also improved by 1.2% in the complex accuracy mAP50-95 index. This fully verifies the effectiveness of the proposed module in enhancing the model's detection capabilities, improving fine-grained recognition accuracy, and maintaining robustness, providing reliable technical support for the identification of Chinese herbal plants in complex natural environments.

[0040] To further verify the advancement of the REM-YOLO model, this study selected the mainstream algorithms YOLOv8n, YOLOv11n, YOLOv12n and YOLOv26n, which are representative in the current field of object detection, for comparative experiments. All models were tested under the same experimental environment and training parameters. The specific results are shown in Table 2.

[0041] Table 2 Comparative Tests

[0042] Experimental analysis shows that REM-YOLO has broken performance records for similar models across multiple core evaluation metrics. In terms of detection accuracy, REM-YOLO achieves an mAP50 of 85.7%, a significant improvement of 1.1% compared to the second-best performing YOLOv12n (84.6%), and achieves the highest score of 64.8% in the mAP50-90 metric, which reflects high-precision localization capabilities. REM-YOLO's advantages are particularly prominent in terms of hardware friendliness and real-time performance. Its computational cost (GFLOPS) is compressed to 5.3, a 28.4% reduction compared to YOLOv8n's 7.4, demonstrating extremely high computational efficiency. Simultaneously, its inference speed (FPS) has increased dramatically to 198.2, approximately 16.4% faster than the second-fastest detection speed YOLOv26n. In conclusion, REM-YOLO achieves a significant leap in detection accuracy while maintaining extremely low computational cost, fully demonstrating the algorithm's efficiency and practicality in real-time target detection tasks.

[0043] Visualization results

[0044] To further understand the performance of REM-YOLO in detection tasks, a comparative heatmap with YOLO11n is presented. Figure 6 As shown, taking the recognition of Agrimonia pilosa against simple and complex backgrounds as an example, although YOLO11n can capture the target of Chinese herbal medicine plants, its heat map has problems such as thermal response area shift and failure to completely cover the core growth part of the plant. Moreover, the response intensity distribution is scattered under background interference, resulting in insufficient robustness of feature extraction.

[0045] The REM-YOLO model exhibited a clearer and more focused thermal response region in all three samples, with its region of interest covering the core leaf cluster of the Agrimonia pilosa plant, indicating that the model is more accurate in target localization. In contrast, REM-YOLO's superior performance is attributed to the synergistic effect of the three improved modules, RepHMS, EUCB, and MSCA. Through reparameterized multi-scale feature extraction, efficient upsampling information restoration, and a multi-scale attention mechanism, it effectively corrected the original model's lack of core feature perception, achieving precise focusing on the key biological characteristics of Agrimonia pilosa.

[0046] The present invention and its embodiments have been described above, and such description is not restrictive. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for identifying and detecting Chinese herbal medicine plants based on an improved YOLO11n, characterized in that: Includes the following steps: Step 1: Introduce a reparameterized heterogeneous multi-scale module to replace the original C3k2 structure. This module achieves efficient extraction and fusion of feature information at different scales through a multi-branch cascade structure and a reparameterized convolution mechanism, thereby improving the feature representation capability of the network while ensuring inference efficiency. Step 2: Introduce an upsampling convolution module in the neck network to replace the traditional upsampling operator. The upsampling convolution module achieves high-quality spatial feature reconstruction while reducing computational redundancy through lightweight multi-scale modeling and channel refinement mechanism. Step 3: Integrate a multi-scale convolutional attention module before the detection layer. The multi-scale convolutional attention module simulates the large kernel convolution effect through lightweight deep bar convolution to obtain multi-scale contextual information. The multi-scale convolutional attention module adopts a parallel multi-branch architecture to reconstruct the input features.

2. The method for identifying and detecting Chinese herbal medicine plants based on the improved YOLO11n according to claim 1, characterized in that: In step one, the input features of the reparameterized heterogeneous multi-scale module are first adjusted by 1×1 convolution, and then a segmentation operation is performed to divide the feature map into N parallel information streams. In the multi-branch architecture, the first branch directly retains the shallow feature information of the input to maintain spatial fidelity. Starting from the second branch, the input information is sequentially passed through M cascaded blocks to deeply mine semantic representation.

3. The method for identifying and detecting Chinese herbal medicine plants based on the improved YOLO11n according to claim 1, characterized in that: In step one, the reparameterized heterogeneous multiscale module adopts a reparameterization strategy. During the training phase, the module uses a multi-branch heterogeneous structure containing 3×3 convolution, 1×1 convolution, and identity mapping to enhance learning ability. During the inference phase, the module utilizes the fusionability of linear operators to fuse multiple parallel parameters into an equivalent single-path 3×3 convolution operator through mathematical transformation, thereby obtaining a wider receptive field while maintaining zero additional inference overhead.

4. The method for identifying and detecting Chinese herbal medicine plants based on the improved YOLO11n according to claim 1, characterized in that: In step two, the upsampling convolution module expands the spatial dimension of the input feature map by a factor of 2 through interpolation, completing the initial resolution restoration. The module uses a 3×3 depthwise separable convolution to process the magnified feature map, accurately capturing the texture features of plant leaf edges and subtle organs. A 1×1 convolution is used to precisely adjust the number of channels, filtering out information redundancy while ensuring that the output features and subsequent fusion layers achieve precise dimensional matching. The calculation process of the upsampling convolution module is as follows: In the formula, , Features of module input and output; This is an upsampling operation; For depthwise separable convolution; This is a normalization operation; For activation function, This is a compressed convolutional layer.

5. The method for identifying and detecting Chinese herbal medicine plants based on the improved YOLO11n according to claim 1, characterized in that: In step three, the input features undergo a depthwise convolution for local information aggregation. One branch uses an identity mapping to preserve the original feature information, while the other three branches extract contextual features at different scales using depthwise strip convolutions with kernel sizes of 1×7 and 7×1, 1×11 and 11×1, and 1×21 and 21×1, respectively. The output features of each branch are fused variably, and channel information interaction and attention weight generation are completed through a 1×1 convolution, forming a multi-scale spatial attention map. The calculation process of the multi-scale convolutional attention module is as follows: In the formula, , Features of module input and output; These are the attention weights calculated by the multi-scale convolutional attention module; This represents the stepwise matrix multiplication operation; For depthwise convolution; To indicate the first Each scale branch.

Images