A method and device for constructing a rotating frame model, detecting abnormal tobacco shreds, and measuring phenotypic characteristics based on YOLOv5.

Abstract

This invention relates to a method and device for constructing a YOLOv5-based rotating bounding box model, detecting abnormal tobacco shreds, and measuring phenotypic characteristics. Belonging to the field of cigarette manufacturing, it is based on the YOLOv5 model architecture. A feature extraction module incorporating multi-scale attention mechanisms and dynamic serpentine convolutions is constructed in the backbone network. A bidirectional feature pyramid network based on context anchors is designed in the neck network to capture long-distance contextual information and extract features bidirectionally. The KLD loss function is introduced in the head network to generate a YOLOv5-based rotating bounding box model for target detection using rotating bounding boxes. This overcomes the bottleneck of the YOLOv5 model being limited to horizontal bounding box labeling. It detects abnormal tobacco shreds from an original image containing several tobacco shreds, obtaining the phenotypic parameters of the abnormal tobacco shreds. This not only accurately obtains the characteristics of abnormal tobacco shreds and reduces interference from background and adjacent targets, but also provides an online detection scheme for the accurate counting of abnormal tobacco shreds and the precise grading control and optimization of shredded tobacco quality, contributing to the modernization of the tobacco processing technology.

Description

Technical Field

[0001] This invention relates to a method and equipment for constructing a rotating frame model based on YOLOv5, detecting abnormal tobacco shreds, and measuring phenotypic characteristics, belonging to the field of cigarette manufacturing. Background Technology

[0002] The shredding process is a crucial step in cigarette manufacturing, directly impacting the quality, taste, and combustion characteristics of the final product. However, due to equipment and process limitations, abnormal tobacco shreds often occur during shredding, such as scalloped tobacco, loose tobacco, rolled tobacco, and serrated tobacco. These abnormalities not only affect quality but can also interfere with subsequent processes and even cause material blockage. Therefore, accurate detection of abnormal tobacco shreds is essential. Traditional detection methods rely on manual visual and tactile assessment, supplemented by limited offline instrument analysis, resulting in low efficiency and insufficient representativeness. In recent years, deep learning technology has made rapid progress in the field of artificial intelligence. Deep learning-based target detection technology, with its powerful feature extraction capabilities and high precision, achieves end-to-end detection and demonstrates good adaptability to different scenes, colors, and scales, providing strong support for the automation of cigarette manufacturing. A series of studies have applied deep learning to various stages of cigarette production. For example, Zhang Pengfei et al. designed a cigar quality detection device that achieves the detection of defects on the cigarette support surface, with an average accuracy (mAP) of 87.7%. Xiao Leiyu et al. constructed a tobacco stem classification and recognition model by improving the YOLOv3 model, achieving a recognition accuracy of 95.01%. Hong Jinhua et al. utilized data augmentation and k-means++ clustering, based on the YOLOv3 model, to achieve accurate identification of tobacco worm locations, tobacco shreds, and tobacco dust in tobacco worm images. Liu Jun et al., addressing the safety hazards of pedestrians entering or materials scattering in material conveying tunnels, proposed an improved YOLOv4 model, introducing SE attention mechanism and GhostNet for feature extraction, achieving an average accuracy of 98.48% and a false detection rate of 0.62%. Wang Ruiqi et al. constructed a detection model for tobacco leaves of different maturity levels using the YOLOv5 model, achieving accurate detection of fresh tobacco leaf grades. The aforementioned YOLO (You Only Look Once) is a real-time object detection algorithm based on deep learning. Its core idea is to transform the object detection task into a single global inference problem, directly predicting the bounding box and category of objects on the image.

[0003] However, it's easy to see that the above detection methods are all based on horizontal bounding box detection models, which have inherent limitations. For example, they are weak in handling occluded and overlapping targets, and struggle to cope with interference from complex background environments. Furthermore, for rotated or tilted irregular targets, horizontal bounding boxes introduce a large amount of background noise, resulting in numerous irrelevant areas within the labeled bounding box, thus reducing labeling accuracy.

[0004] In the tobacco manufacturing process, the detection of abnormal tobacco shreds still faces significant challenges due to factors such as dense stacking and mutual occlusion. In the task of detecting abnormal tobacco shreds and measuring phenotypic parameters, the following issues need special consideration: (1) Unlike natural scene images, abnormal tobacco shreds are usually arranged at an angle during actual tobacco processing. Using a horizontal bounding box will result in a large amount of background area being included, thus reducing detection accuracy; (2) Abnormal tobacco shreds are easily affected by occlusion and overlap with normal tobacco shreds, and some abnormal tobacco shreds are incomplete, making accurate detection and counting difficult based solely on appearance features; (3) In a densely stacked state, the target scale differences are significant, and there are many small targets, leading to a decrease in detection accuracy and making it difficult to accurately measure phenotypic parameters. Therefore, in order to optimize the tobacco cutting process, improve product quality and production efficiency, a robust, efficient, and accurate method for detecting abnormal tobacco shreds that adapts to the actual production environment is urgently needed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for constructing a rotating frame model based on YOLOv5, enabling target detection using a rotating frame while ensuring high detection speed and meeting detection accuracy requirements. Furthermore, this invention provides a method and device for detecting abnormal tobacco shreds and measuring their phenotypic characteristics based on a rotating frame model of YOLOv5, to address the problems of low detection accuracy and inability to obtain phenotypic information of tobacco shreds in the complex environment of tobacco processing.

[0006] To achieve the above objectives, this invention proposes a method for constructing a rotated box model based on YOLOv5, comprising:

[0007] Based on the YOLOv5 model architecture, a feature extraction module containing multi-scale attention mechanism and dynamic snake convolution is constructed in the backbone network;

[0008] A context-anchor-based bidirectional feature pyramid network is designed in the neck network to capture long-range contextual information and extract features bidirectionally;

[0009] The KLD loss function is introduced into the head network to generate a YOLOv5-based rotated box model for object detection using rotated boxes.

[0010] Furthermore, the feature extraction module is constructed through the following steps:

[0011] 1) Combine multi-scale attention mechanisms and dynamic serpentine convolutions to generate DEblock modules;

[0012] 2) The feature extraction module is obtained by following the order of the convolution module, the DEblock module and the convolution module.

[0013] Furthermore, the feature extraction module is combined using an arrangement of two convolutional modules, several deblocking modules, and one convolutional module.

[0014] Furthermore, the Deblock module is composed of a dynamic serpentine convolution layer, two standard convolutional layers, and a multi-scale attention mechanism module arranged in that order.

[0015] Furthermore, a context-anchor-based bidirectional feature pyramid network for capturing long-range contextual information and extracting features bidirectionally is designed in the neck network using the following method:

[0016] In the path aggregation network structure, the context anchor-based bidirectional feature pyramid network is obtained by bidirectional information fusion and the introduction of a context anchor attention mechanism to capture long-distance context information.

[0017] Furthermore, the KLD loss function is expressed by the following formula:

[0018]

[0019] Among them, L kld Let represent the regression loss of the bounding box; D be the loss value; sqrt(D) be a nonlinear function containing the loss value D; τ be a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula:

[0020]

[0021] Among them, u p The mean of the predicted distribution; u t Tr is the mean of the true distribution; Tr is the trace of the covariance matrix of the true distribution. The covariance matrix of the true distribution; It is the covariance matrix of the predicted distribution.

[0022] On the other hand, this invention also proposes a method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a rotating frame model of YOLOv5, comprising:

[0023] A pre-built YOLOv5-based rotating bounding box model is invoked to detect abnormal tobacco shreds from an original image containing several tobacco shreds, and the phenotypic parameters of the abnormal tobacco shreds are obtained. The YOLOv5-based rotating bounding box model is constructed through the following steps:

[0024] 1) Based on the YOLOv5 model architecture, a feature extraction module containing multi-scale attention mechanism and dynamic snake convolution is built in the backbone network; 2) In the neck network, a bidirectional feature pyramid network based on context anchor is designed to capture long-distance context information and extract features bidirectionally; 3) In the head network, the KLD loss function is introduced to generate a YOLOv5-based rotated box model for object detection using rotated boxes.

[0025] Furthermore, the feature extraction module is constructed through the following steps:

[0026] 1) Combine multi-scale attention mechanisms and dynamic serpentine convolutions to generate DEblock modules;

[0027] 2) The feature extraction module is obtained by following the order of the convolution module, the DEblock module and the convolution module.

[0028] Furthermore, the feature extraction module is combined using an arrangement of two convolutional modules, several deblocking modules, and one convolutional module.

[0029] Furthermore, the Deblock module is composed of a dynamic serpentine convolution layer, two standard convolutional layers, and a multi-scale attention mechanism module arranged in that order.

[0030] Furthermore, a context-anchor-based bidirectional feature pyramid network for capturing long-range contextual information and extracting features bidirectionally is designed in the neck network using the following method:

[0031] In the path aggregation network structure, the context anchor-based bidirectional feature pyramid network is obtained by bidirectional information fusion and the introduction of a context anchor attention mechanism to capture long-distance context information.

[0032] Furthermore, the KLD loss function is expressed by the following formula:

[0033]

[0034] Among them, L kld Let represent the regression loss of the bounding box; D be the loss value; sqrt(D) be a nonlinear function containing the loss value D; τ be a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula:

[0035]

[0036] Among them, u p The mean of the predicted distribution; u t Tr is the mean of the true distribution; Tr is the trace of the covariance matrix of the true distribution. The covariance matrix of the true distribution; It is the covariance matrix of the predicted distribution.

[0037] Furthermore, the phenotypic parameters include the length and / or width of the abnormal tobacco shreds.

[0038] On the other hand, the present invention also proposes an abnormal tobacco detection and phenotypic measurement device based on a rotating frame model of YOLOv5, including a processor, the processor being used to execute the above-described abnormal tobacco detection and phenotypic measurement method based on a rotating frame model of YOLOv5.

[0039] The beneficial effects of this invention are as follows: Based on the YOLOv5 model architecture, a feature extraction module in the backbone network, incorporating multi-scale attention mechanisms and dynamic serpentine convolutions, is constructed to more effectively focus on the region of interest and improve detection accuracy; a context-anchor-based bidirectional feature pyramid network is designed in the neck network to capture long-distance contextual information and extract features bidirectionally, enhancing the ability of multi-scale feature fusion, alleviating the problem of inaccurate recognition caused by occlusion and overlap, and improving the model's performance in densely stacked scenarios; and the KLD loss function is introduced in the head network to solve the periodic angle problem, which is particularly suitable for... This method is used for detecting abnormal tobacco shreds with large aspect ratios, thereby improving positioning accuracy. It generates a YOLOv5-based rotating bounding box model for target detection, overcoming the bottleneck of the YOLOv5 model being limited to horizontal bounding box marking. By calling the YOLOv5-based rotating bounding box model, abnormal tobacco shreds are detected from the original image containing several tobacco shreds, and the phenotypic parameters of the abnormal tobacco shreds are obtained. This not only accurately obtains the characteristics of abnormal tobacco shreds and reduces interference from the background and adjacent targets, but also provides an online detection solution for the accurate counting of abnormal tobacco shreds and the precise grading control and optimization of the shredded tobacco quality, which helps to promote the modernization of the tobacco processing technology. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the EAM structure in a practical application scenario of the YOLOv5-based method for constructing a rotating frame model proposed in this invention.

[0041] Figure 2 This is a schematic diagram comparing the sampling points between standard convolution and dynamic serpentine convolution in a practical application scenario of the method for constructing a YOLOv5-based rotated box model proposed in this invention.

[0042] Figure 3 This is a schematic diagram of the CAA module structure in a practical application scenario of the method for constructing a YOLOv5-based rotated frame model proposed in this invention.

[0043] Figure 4 This is a schematic diagram of the FPN structure in the prior art;

[0044] Figure 5 This is a schematic diagram of the PANet structure in the prior art;

[0045] Figure 6 This is a schematic diagram of the structure of a bidirectional feature pyramid network (CAA-BiFPN) based on context anchors in a practical application scenario, based on the YOLOv5-based method for constructing a rotated box model proposed in this invention.

[0046] Figure 7 This is a schematic diagram of the C3-Deblock structure in a practical application scenario of the method for constructing a YOLOv5-based rotating frame model proposed in this invention.

[0047] Figure 8 This is an overall structural diagram of a YOLOv5-based rotating frame model constructed in a practical application scenario using the YOLOv5-based rotating frame model construction method proposed in this invention.

[0048] Figure 9 This is a schematic diagram illustrating the photogrammetric relationship between the camera and the target in a practical application scenario for an abnormal tobacco detection and phenotypic measurement method based on a YOLOv5 rotating frame model proposed in this invention.

[0049] Figure 10 This is a schematic diagram of the acquisition device and tobacco image sample in a practical application scenario of the abnormal tobacco detection and phenotypic measurement method based on the YOLOv5 rotating frame model proposed in this invention.

[0050] Figure 11 This is the original image and its annotated schematic diagram of the method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on the YOLOv5 rotating frame model proposed in this invention in a practical application scenario.

[0051] Figure 12 This is a schematic diagram showing the comparison of results between the Rot-YOLOv5 model and the YOLOv5 model in a comparative experiment for the abnormal tobacco detection and phenotypic measurement method based on the YOLOv5 rotating frame model proposed in this invention.

[0052] Figure 13 This is a schematic diagram comparing the results of the horizontal detection box algorithm YOLOv5 and the rotation detection algorithm Rot-YOLOv5 in abnormal tobacco target detection in a comparative experiment, based on the rotating frame model of YOLOv5 proposed in this invention.

[0053] Figure 14 This is a schematic diagram showing the results of the experimental analysis of the detection effect of the model using Grad-CAM, based on the rotating frame model of YOLOv5 proposed in this invention for abnormal tobacco detection and phenotypic measurement.

[0054] Figure 15 This is a schematic diagram showing the results of abnormal tobacco counting performance in a model performance experiment of an abnormal tobacco counting method based on a YOLOv5 rotating frame model proposed in this invention.

[0055] Figure 16 This is a schematic diagram comparing the automatically extracted and measured values ​​of six phenotypic traits in the abnormal tobacco shred detection and phenotypic measurement and analysis method based on the YOLOv5 rotating frame model proposed in this invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] The inventive concept of this invention lies in the following: Since the YOLOv5 model is mainly based on a horizontal bounding box detection model, it is difficult to cope with interference from complex background environments when detecting objects. At the same time, considering the complex state of densely stacked tobacco shreds and mutual occlusion in the tobacco processing scene, a feature extraction module including a multi-scale attention mechanism and dynamic serpentine convolution is added to the YOLOv5 model architecture; a bidirectional feature pyramid network based on context anchors is used to enhance the model's performance in densely stacked scenes; the KLD loss function is introduced to improve the model's localization accuracy, and a rotating bounding box is generated to accurately lock the target detection region, avoiding the inclusion of a lot of background in the bounding box. This not only enhances the feature extraction of the region of interest, but also improves the performance in densely stacked scenes, and can also cope with the detection of abnormal tobacco shreds with large length and width, achieving accurate detection of abnormal tobacco shreds and reducing interference from background areas.

[0058] Specific implementation method 1 for constructing a rotated frame model based on YOLOv5:

[0059] On the one hand, this invention proposes a method for constructing a YOLOv5-based rotated box model. The YOLOv5 model is the fifth version of the YOLO series, consisting of a backbone network, a neck network, and a head network, with five different versions. YOLOv5 represents a significant leap forward in performance and ease of use, and is also simple to use and convenient to deploy. Therefore, this invention proposes a method for constructing a YOLOv5-based rotated box model, including steps S11, S12, and S13, specifically:

[0060] First, in step S11, based on the YOLOv5 model architecture, a feature extraction module incorporating multi-scale attention mechanisms and dynamic serpentine convolutions is constructed in the backbone network. It should be noted that among the five different versions of the YOLOv5 model, YOLOv5s has the smallest network structure, exhibiting high detection speed and operational efficiency. Considering the densely packed and irregularly shaped abnormal tobacco shreds encountered in tobacco manufacturing applications, this invention preferably utilizes the YOLOv5s model architecture within the YOLOv5 model architecture. Simultaneously, in the YOLOv5 model, the backbone network is responsible for extracting features from the input image; the neck network fuses features at different scales to handle complex scene variations; and the head network performs classification and regression.

[0061] The multi-scale attention mechanism is a technique that introduces attention weights at multiple scales, enabling the model to understand and process complex data more deeply. Specifically, by capturing data features at different scales, the model can simultaneously focus on local details and global structure, improving its understanding of details and contextual information. In practical applications, considering the complexity of manufacturing abnormal tobacco shred morphology, an efficient multi-scale attention (EMA) mechanism is adopted. This mechanism captures pixel-level relationships through cross-dimensional interaction, significantly improving the model's ability to process spatial features. Preferably, such as... Figure 1 The diagram shown is a schematic representation of the structure of EAM in a practical application scenario of the YOLOv5-based method for constructing a rotating frame model proposed in this invention.

[0062] The Dynamic Snake Convolution (DSConv) employs a deformable convolution kernel composed of non-standard sampling points, such as... Figure 2 The diagram shows a comparison of sampling points between standard convolution and dynamic serpentine convolution in a practical application scenario for the YOLOv5-based rotating bounding box model construction method proposed in this invention. (a) represents the sampling points of standard convolution, and (b) represents the sampling points of dynamic serpentine convolution. However, completely unconstrained convolutional deformation can lead to the loss of minor details. Therefore, dynamic serpentine convolution introduces continuity constraints to ensure continuity, enabling DSConv to adaptively prioritize slender and tortuous local structures, thereby accurately capturing the features of complex object detection. Simultaneously, during model construction, learnable offsets are introduced to enable the model to more flexibly capture complex deformed objects.

[0063] Secondly, in step S12, a context-anchor-based bidirectional feature pyramid network is designed in the neck network to capture long-range contextual information and extract features bidirectionally. It should be noted that the context anchor refers to the Context Anchor Attention (CAA) mechanism. CAA is an attention mechanism designed for target detection in remote sensing images, aiming to capture long-range contextual information and enhance the expressive power of central features. It generates a weighted attention map through operations such as pooling, convolution, and sigmoid, allowing each region in the feature map to be weighted according to its global contextual information. This enables features of different importance to be selectively enhanced or suppressed, significantly improving the effectiveness of feature expression and detection performance. Preferably, such as... Figure 3 The diagram shows the structure of the CAA module in a practical application scenario of the YOLOv5-based method for constructing a rotating frame model proposed in this invention.

[0064] Meanwhile, in the YOLOv5 model, the neck network (Neck layer) adopts the Path Aggregation Network (PANet) structure. PANet is developed based on the Feature Pyramid Network (FPN). Unlike FPN, PANet introduces a bottom-up feature fusion path, enabling low-level image features to be effectively transferred to high-level layers, fully utilizing low-level information, and overcoming the limitation of FPN that only relies on top-down feature transfer. Figure 4 and Figure 5 The diagrams shown are schematics of the FPN and PANet structures, respectively. However, when performing feature fusion, FPN and PANet typically only perform simple summarization and fail to effectively extract features from elongated targets with large size variations. Therefore, to better utilize the local feature information of shallow networks and the global feature information of deep networks, and to improve the performance of the detection algorithm in target detection localization and classification, this invention introduces a bidirectional feature pyramid network based on context anchors. Preferably, as shown... Figure 6 The diagram shows the structure of a bidirectional feature pyramid network (CAA-BiFPN) based on context anchors in a practical application scenario, which is a method for constructing a YOLOv5-based rotated box model proposed in this invention.

[0065] Finally, among traditional YOLO series object detection algorithms, the rotational object detection algorithm faces challenges in learning periodic angle parameters, especially the discontinuity at the periodic boundaries, which may lead to a sudden increase in the loss function value and affect the training stability of the network. Meanwhile, rotational object detection utilizes angled rectangles to provide target position and orientation information, significantly improving detection accuracy and robustness, and has broad application prospects in fields such as autonomous driving, remote sensing, industrial inspection, and agriculture. (For example, to address the insufficient accuracy of soybean branch detection and the problem of rapid acquisition of phenotypic parameters, Guo Xiyue et al. proposed Re-YOLOv5 and a detection region search algorithm, introducing circular smooth labels, a coordinated attention mechanism, and a RepVGG structure, improving mAP by 1.70%, reducing the number of parameters by 0.17M, and improving stem node accuracy by 9.90%. For accurate wheat ear localization and counting in complex environments, Yu Junwei et al. proposed an improved Oriented R-CNN, integrating SPCSPC, PANet, E2CBAM, and Soft-NMS, improving mAP by 2.02% compared to the original model, outperforming mainstream rotational object detection models.) In step S13, the KLD loss function is introduced into the head network to generate a YOLOv5-based rotated bounding box model for object detection. Here, the KLD loss function refers to the Kullback-Leibler divergence loss, an asymmetric metric that measures the difference between two probability distributions. In rotated object detection, the angle parameter can be modeled as a probability distribution to represent the possible rotation angle of the target. Since the discontinuity of periodic angles can cause abrupt changes in the probability distribution at the boundaries, the KLD method is used to measure the difference between adjacent periodic probability distributions and incorporate it into the loss function.

[0066] Through steps S11-S13, a feature extraction module incorporating multi-scale attention mechanisms and dynamic serpentine convolutions is constructed in the backbone network based on the YOLOv5 model. This allows for more effective focus on regions of interest, improving detection accuracy. In the neck network, a bidirectional feature pyramid network based on context anchors is designed to enhance multi-scale feature fusion capabilities, alleviate inaccurate recognition caused by occlusion and overlap, and improve the model's performance in densely stacked scenarios. In the head network, the KLD loss function is introduced to address the periodic angle problem, which is particularly suitable for detecting abnormal tobacco shreds with large aspect ratios, thereby improving localization accuracy. This generates a YOLOv5-based rotated bounding box model for target detection using rotated bounding boxes.

[0067] Specific implementation method 2 for constructing a rotated frame model based on YOLOv5:

[0068] Following the specific implementation described above, in step S11, the feature extraction module is constructed through the following steps:

[0069] 1) The DEblock module combines multi-scale attention mechanisms and dynamic serpentine convolutions. Within the DEblock module, the arrangement of convolutional layers (multi-scale attention, dynamic serpentine convolutions, and standard convolutions) is tailored to different detection task requirements, object characteristics, and training sample quantities. For example, for simple tasks like handwritten digit recognition, basic features can be extracted using only 2-3 standard convolutional layers. For complex tasks like facial recognition, which requires processing both local facial features and global contours, feature extraction can be performed through a custom layered design. In the abnormal tobacco detection task, the preferred combination of the Deblock module is one layer of dynamic serpentine convolution, two layers of standard convolutional layers, and one layer of multi-scale attention mechanism.

[0070] 2) Following the order of the convolutional module, the DEblock module, and the convolutional module, the feature extraction module is obtained. The input data first passes through the first convolutional module to reduce the number of channels, then through the Deblock module for feature extraction, and finally the outputs of each module are concatenated along the channel dimension, and the final features are generated through the last convolutional module. Furthermore, the feature extraction module is introduced into the C3 module of the backbone network to generate a new C3-DEBlock module, such as... Figure 7 The diagram shows the C3-Deblock structure of the YOLOv5-based rotating box model construction method proposed in this invention in a practical application scenario. Similarly, the design and arrangement of the feature extraction modules are designed according to different detection task requirements, different detected object features, and different training sample numbers. In the abnormal tobacco shred detection task, the preferred combination of feature extraction modules is an arrangement of two convolutional modules, several Deblock modules, and one convolutional module.

[0071] By addressing the limitations of YOLOv5 models that use convolution kernels with fixed shapes and parameters in standard convolutions, which prevent optimal feature recognition of targets with different characteristics, particularly for original images containing multiple tobacco strands, the model overcomes the challenge of optimally recognizing these long, tubular structures due to their elongated and varied shapes. The introduction of a multi-scale attention mechanism enables cross-dimensional interactive capture of pixel-level relationships, significantly improving the model's ability to process spatial features. Furthermore, the introduction of dynamic serpentine convolutions adaptively prioritizes elongated and tortuous local structures, thereby accurately capturing the features of complex targets.

[0072] Detailed Implementation Method 3 for Constructing a Rotated Box Model Based on YOLOv5:

[0073] Following the specific implementation described above, in step S12, a context-anchor-based bidirectional feature pyramid network for capturing long-range contextual information and extracting features bidirectionally is designed in the neck network using the following method:

[0074] In the path aggregation network structure, a context anchor-based bidirectional feature pyramid network is obtained through bidirectional information fusion and the introduction of a context anchor attention mechanism to capture long-distance context information. It should be noted that, based on PANet (path aggregation network structure), learnable parameters are added to each path in the module to solve the problem of uneven contribution of different input features to the output in the original network. In addition, by combining the skip connection mechanism, features of different resolutions are fused to achieve efficient integration of multi-scale features, thus improving PANet. In the improved PANet, a context anchor attention mechanism is introduced to mine the interdependencies between long-distance contexts and enhance the expressive power of central features, resulting in a context anchor-based bidirectional feature pyramid network. This strengthens the ability of multi-scale feature fusion, alleviates the problem of inaccurate recognition caused by complex environments, and improves the performance of the model in densely stacked scenarios.

[0075] In step S13, in the head network, the Kullback-Leibler Divergence (KLD) loss function calculates the center point coordinates, width, height, and angle information (x, y, w, h) of the predicted bounding box. ), and the center coordinates, width, height, and angle information of the real bounding box are converted into... The transformation formula for the two-dimensional Gaussian distribution is as follows:

[0076]

[0077]

[0078] in, R represents the covariance matrix of the true distribution; R represents the rotation matrix; S represents the diagonal matrix of the eigenvalues.

[0079] The center point coordinates of the target object are treated as positional information distributed in a two-dimensional Gaussian distribution. A rotation matrix is ​​used to transform the object's width, height, and angle into scale parameters distributed in a Gaussian distribution, thus mapping the rotated bounding box to a two-dimensional Gaussian distribution. This not only solves the problem of angle periodicity but also eliminates the problem of boundary commutativity. The distance between the ground truth bounding box and the predicted bounding box in the two-dimensional Gaussian distribution is calculated as the regression loss to optimize the box's position and scale. The KLD loss function is expressed by the following formula:

[0080]

[0081] Among them, L kldLet represent the regression loss of the bounding box; D is the loss value; sqrt(D) is a nonlinear function incorporating the loss value D to smooth the loss; τ is a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula:

[0082]

[0083] Where D refers to the KLD loss value, which measures the difference between two probability distributions; u p The mean of the predicted distribution; u t Tr is the mean of the true distribution; Tr is the trace of the covariance matrix of the true distribution. The covariance matrix of the true distribution; It is the covariance matrix of the predicted distribution.

[0084] For objects with large aspect ratios: the KLD loss function is scale-invariant, meaning the loss value is the same at different scales, reducing the impact of differences in predicted bounding box size on the loss and improving detection accuracy and robustness. Furthermore, as the aspect ratio of the target increases, the derivative of the KLD loss function focuses more on angle optimization, effectively reducing the impact of angle differences on large aspect ratio targets. Therefore, the KLD loss function is particularly suitable for detecting objects with large aspect ratios and can provide higher accuracy results. In abnormal tobacco shred detection, the KLD regression loss function improves the model's sensitivity and detection accuracy by quantifying the difference between the predicted and true distributions. It is suitable for capturing subtle features and multimodal distributions of abnormal tobacco shreds. Compared to traditional loss functions, KLD can more effectively handle uncertain data and can be combined with generative models to optimize the latent space representation, thereby ensuring model reliability. For example, for abnormal tobacco shreds and stems, the regression loss of the bounding box of the abnormal tobacco shreds can be calculated using the following formula:

[0085]

[0086] Among them, L kld The regression loss is for the bounding box. To adjust the hyperparameters of the entire loss function.

[0087] Specific implementation method 4 for constructing a rotated box model based on YOLOv5:

[0088] like Figure 8The diagram shows the overall structure of the YOLOv5-based rotating bounding box model constructed in a practical application scenario using the YOLOv5-based model construction method proposed in this invention. The Rot-YOLOv5 model consists of three parts: Backbone, Neck, and Head. The Backbone extracts features from the input image and constructs a C3-DEblock module by combining the EMA attention mechanism with the DSConv module. This allows the network to more effectively focus on the region of interest, thereby improving detection accuracy. The Neck fuses features at different scales to handle complex scene variations and introduces a CAA-BiFPN module to enhance multi-scale feature fusion capabilities, alleviating inaccurate recognition caused by occlusion and overlap, and improving the model's performance in densely stacked scenes. The Head performs classification and regression, using the KLD loss function to regress the five parameters of the rotating bounding box (center point coordinates, width, height, and angle information). The KLD loss function treats the rotation angle as a probability distribution, solving the periodic angle problem and is particularly suitable for detecting abnormal tobacco shreds with large aspect ratios, thereby improving localization accuracy.

[0089] Specific Implementation Method 1 for Abnormal Tobacco Detection and Phenotypic Measurement Method Based on YOLOv5 Rotating Frame Model:

[0090] On the other hand, this invention proposes a method for detecting and measuring abnormal tobacco strands based on a YOLOv5-based rotating frame model. The method involves calling a pre-built YOLOv5-based rotating frame model to detect abnormal tobacco strands from an original image containing several tobacco strands and obtaining the phenotypic parameters of the abnormal tobacco strands. The phenotypic parameters include the length and / or width of the abnormal tobacco strands. The types of abnormal tobacco strands include, but are not limited to, fused strands, stems, and broken pieces.

[0091] For example, after an industrial camera acquires an RGB color image (original image) containing several tobacco strands, the Rot-YOLOv5 recognition model (i.e., a YOLOv5-based rotated bounding box model) backbone network is used for feature extraction. Then, feature pyramids at different scales are fused, and the result is output to the detection layer for prediction of abnormal tobacco strands and their phenotypes. Specifically, the minimum bounding rectangle of the rotated bounding box is used to extract the width and length information of the strands, stems, and strips to obtain the pixel values ​​of the objects. At the same time, by calculating the number of anchor boxes, the number of each target category is calculated separately, and the results are displayed in the upper left corner of the image, thus completing the detection of the number of abnormal tobacco strands in the original image or image.

[0092] Furthermore, since the extracted information on the width and length of the strips, stems, and segments is all individual abnormal tobacco shreds at the pixel level, this invention also converts the phenotypic parameters of the obtained abnormal tobacco shreds into actual physical dimensions. Specifically, the physical dimension reconstruction process infers or reconstructs the actual physical dimensions of the object in the real world based on the pixel dimensions of the image. This process is based on the camera's intrinsic and extrinsic parameters, first determining the spatial relationship between the measured plane and the camera's imaging unit, and then calculating the correspondence between the real physical space and the pixel distance through a proportional relationship. For example... Figure 9 The diagram illustrates the photogrammetric relationship between the camera and the target in a practical application scenario for an abnormal tobacco detection and phenotypic measurement method based on a YOLOv5 rotating frame model proposed in this invention. In (a) and (b), s represents the camera's coordinate position, plane abc represents the actual physical space, and plane a'b'c' represents a point on the camera's imaging unit (i.e., image space). (b) is obtained by symmetrically transforming the image plane in (a), and plane a”b”c” in (b) is the actual physical space corresponding to the image space. The significance of the reconstruction lies in obtaining the true physical size of plane abc through the image size of plane a”b”c” in (b). Experimental calculations show that this invention preferably uses K=1.23mm / pixel as the conversion coefficient between pixel distance and physical space distance to obtain the actual physical size distribution of abnormal tobacco.

[0093] Specific Implementation Method 2 for Abnormal Tobacco Detection and Phenotypic Measurement Method Based on YOLOv5 Rotating Frame Model:

[0094] Following the specific embodiments described above, the construction of the YOLOv5-based rotated frame model involves the following steps:

[0095] Step 1: Baseline Model Construction: The YOLOv5 model consists of a backbone network, a neck network, and a head network, with five different versions. Among them, YOLOv5s has the smallest network structure and high detection speed and operating efficiency. Since abnormal tobacco shreds with dense stacking and various shapes often occur during the tobacco processing, YOLOv5s can ensure a fast detection speed while meeting accuracy requirements. Therefore, further improvements and applications will be based on the YOLOv5s model structure.

[0096] Step 2: Construction of C3-Deblock in the backbone network: In tobacco images, abnormal tobacco strands and stems differ from regular tobacco pieces, exhibiting elongated tubular structures with varied morphologies. YOLOv5's standard convolution uses kernels with fixed shapes and parameters, which cannot optimally identify these elongated tubular structures. Figure 2Dynamic snake convolution employs deformable convolution kernels composed of non-standard sampling points and introduces learnable offsets, enabling the model to flexibly capture complex deformable objects. However, completely unconstrained convolutional deformation may lead to the loss of minor details. Therefore, DSConv also introduces continuity constraints to ensure the continuity of the receptive field. Here, the continuity of the receptive field refers to the reasonable correlation between feature responses at adjacent spatial locations when the convolutional neural network extracts features, rather than abrupt or inconsistent changes. By introducing continuity constraints, DSConv effectively solves the problem of discontinuous receptive fields that may occur in depthwise separable convolutions. In standard depthwise separable convolutions, the kernel weights of channel-wise convolutions are spatially independent, which may lead to a lack of continuity in the receptive fields at adjacent locations, especially in dynamic or conditional convolutions. To address this, continuity constraints ensure the continuity and consistency of the receptive field during feature extraction by forcing smooth changes in the kernel weights at adjacent spatial locations. This mechanism not only conforms to the spatial continuity of natural signals but also improves optimization stability and avoids training getting trapped in local optima. With this design, DSConv can adaptively prioritize slender and tortuous local structures, thereby accurately capturing the characteristics of complex tobacco shreds.

[0097] Considering the complexity of anomalous tobacco morphology in densely stacked environments, an efficient multi-scale attention module is introduced. This module captures pixel-level relationships through cross-dimensional interaction, significantly improving the model's ability to process spatial features. To further enhance feature extraction, this study combines DSC, EMA, and the C3 module in the backbone network to construct C3-DEblock. This module retains the advantages of DSConv in extracting elongated local features while utilizing EMA to improve sensitivity to anomalous tobacco features in complex and dense environments. Figure 7 As shown, the DEblock consists of one DSConv layer, two standard convolutional layers, and an EMA module combined sequentially, with residual connections added to enhance the reuse and propagation of input features. The C3-DEblock consists of two convolutional modules, n DEblock modules, and another convolutional module. The input data first passes through the first convolutional module to reduce the number of channels, then through the n DEblocks for feature extraction, and finally the outputs of each module are concatenated along the channel dimension, and the final features are generated through the last convolutional module.

[0098] Step 3: Context Anchor Bidirectional Feature Pyramid Network in the Neck Network: The Neck layer (i.e., the neck network) in the YOLOv5 algorithm adopts a path aggregation network (PANet) structure, see... Figure 5PANet is an evolution of the Feature Pyramid Network (FPN). Unlike FPN, PANet introduces a bottom-up feature fusion path, enabling lower-level image features to be effectively passed to higher levels, fully utilizing low-level information and overcoming the limitation of FPN, which relies solely on top-down feature transmission. However, both FPN and PANet typically perform only simple summarization during feature fusion, failing to effectively extract features from elongated targets with large variations in size.

[0099] To better utilize the local feature information of shallow networks and the global feature information of deep networks, and to improve the performance of the detection algorithm in the localization and classification of abnormal tobacco shreds, a context-anchor-weighted bidirectional feature pyramid network (CAA-BiFPN) is introduced. (See...) Figure 6 This method improves upon PANet by adding learnable parameters to each path within the module, effectively addressing the uneven contribution of different input features to the output in the original network. Furthermore, by incorporating a skip connection mechanism, it fuses features at different resolutions, achieving efficient integration of multi-scale features.

[0100] To further explore the interdependencies between remote contexts and enhance the expressive power of central features, CAA-BiFPN introduces a context anchor attention mechanism (CAA), the specific structure of which is shown below. Figure 3 This mechanism generates weighted attention maps through operations such as pooling, convolution, and sigmoid, allowing each region in the feature map to be weighted according to its global context information. In this way, features of different importance can be selectively enhanced or suppressed, thereby significantly improving the effectiveness of feature representation and detection performance.

[0101] Step 4: Calculation of Rotation Box Regression Loss in the Head Network: Compared with traditional YOLO series object detection algorithms, rotation object detection algorithms face challenges in learning periodic angle parameters, especially the discontinuity at the periodic boundaries, which can lead to a sudden increase in the loss function value and affect the training stability of the network. To address this issue, Kullback-Leibler divergence (KLD) is introduced. KLD is a method for measuring the difference between two probability distributions. In rotation object detection, the angle parameter can be modeled as a probability distribution to represent the possible rotation angle of the target. Since the discontinuity of periodic angles can cause abrupt changes in the probability distribution at the boundaries, the KLD method is used to measure the difference between adjacent periodic probability distributions and incorporate it into the loss function.

[0102] The Kullback-Leibler Divergence (KLD) loss function will be composed of (x, y, w, h, ... The center coordinates, width, height, and angle information of the predicted bounding box and the ground truth bounding box are converted into... The transformation process of the two-dimensional Gaussian distribution is represented by the following formula:

[0103]

[0104]

[0105] in, R represents the covariance matrix of the true distribution; R represents the rotation matrix; S represents the diagonal matrix of the eigenvalues.

[0106] The center point coordinates of the target object are treated as positional information distributed in a two-dimensional Gaussian distribution. A rotation matrix is ​​used to transform the object's width, height, and angle into scale parameters distributed in a Gaussian distribution, thus mapping the rotated bounding box to a two-dimensional Gaussian distribution. This not only solves the problem of angle periodicity but also eliminates the problem of boundary commutativity. The distance between the ground truth bounding box and the predicted bounding box in the two-dimensional Gaussian distribution is calculated as the regression loss to optimize the box's position and scale. The KLD loss function is expressed by the following formula:

[0107]

[0108] Among them, L kld Let represent the regression loss of the bounding box; D is the loss value; sqrt(D) is a nonlinear function incorporating the loss value D to smooth the loss; τ is a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula:

[0109]

[0110] Where D refers to the KLD loss value, which measures the difference between two probability distributions; u p The mean of the predicted distribution; u t Tr is the mean of the true distribution; Tr is the trace of the covariance matrix of the true distribution. The covariance matrix of the true distribution; It is the covariance matrix of the predicted distribution.

[0111] The KLD loss function is scale-invariant, meaning the loss value remains the same across different scales. This reduces the impact of differences in predicted bounding box size on the loss, improving detection accuracy and robustness. Furthermore, as the aspect ratio of the target increases, the derivative of the KLD loss function focuses more on angle optimization, effectively reducing the impact of angle differences on targets with large aspect ratios. Therefore, the KLD loss function is particularly suitable for detecting objects with large aspect ratios and can provide higher accuracy results. For example, for detecting clumps and stems, the regression loss of the bounding box for abnormal tobacco shreds can be calculated using the following formula:

[0112]

[0113] Among them, L kld The regression loss is for the bounding box. To adjust the hyperparameters of the entire loss function.

[0114] Through steps one through four above, a rotating frame model based on YOLOv5 is constructed and generated, enabling accurate location of abnormal tobacco shreds using rotating frame technology. This significantly reduces interference from the background area and provides an online detection solution for the accurate counting of abnormal tobacco shreds and the precise grading control and optimization of shredded tobacco quality, thus contributing to the modernization of tobacco processing technology.

[0115] Specific Implementation Method 3 for Abnormal Tobacco Detection and Phenotypic Measurement Method Based on YOLOv5 Rotating Frame Model:

[0116] The following section further explains the abnormal tobacco detection and phenotypic measurement method based on the YOLOv5-based rotating frame model, focusing on its application in model training, evaluation, and abnormal tobacco detection when constructing a YOLOv5-based rotating frame model, as well as the conclusions drawn from these findings:

[0117] 1. Regarding materials and equipment: The samples used for model training are images of tobacco shreds after being shredded by a shredder. The brand is Grey Wolf, and the width of the shreds ranges from 0.41 to 1.53 mm, used to create the tobacco shred dataset. The tobacco shreds are categorized into flakes (shredded tobacco leaves that remain in sheet form after the shredding process and are larger than the required processing standards), strips (shredded tobacco that is stuck together in the cutting direction and does not easily separate), and stems (tobacco stems remaining after tobacco processing). See [link to documentation]. Figure 10 This is a schematic diagram of the acquisition device and tobacco image sample in a practical application scenario of the abnormal tobacco detection and phenotypic measurement method based on the YOLOv5 rotating frame model proposed in this invention.

[0118] Because the abnormal tobacco shreds are long and narrow and densely distributed, using a rotating frame annotation can more accurately represent the area of ​​abnormal tobacco shreds, avoiding the inclusion of more background within the frame. The effects of the two annotation methods are compared below. Figure 11As shown, (a), (b), (c), and (d) are included; (a) is the original image, (b) is the original image after horizontal bounding box annotation, (c) is the original image after rotating bounding box annotation, and (d) is a comparison of the two annotation methods. Table 1 below shows the average pixel area of ​​abnormal tobacco shreds annotated with horizontal bounding boxes (HBB) and rotating bounding boxes (OBB). In the abnormal tobacco shred images, rotating the bounding box reduces the pixel area of ​​the strip annotation by 61.9%, and the area of ​​the stem annotation box is reduced by 54.9%. Therefore, rotating the bounding box annotation can more accurately obtain the features of abnormal tobacco shreds and reduce the interference of the background and adjacent targets. Table 1 below shows the difference in average area between horizontal bounding boxes and rotating bounding boxes:

[0119]

[0120] Table 1

[0121] 2. Dataset Construction: During model training and learning, `labelImg` and `roLabelImg` were used to accurately label the bars, tags, and slices in the original images. In `roLabelImg`, data was saved in PASCAL VOC (PASCAL VisualObject Classes) format, and the generated XML file contained the center coordinates, length, width, and rotation angle information of the rectangles. Augmentation techniques were applied to the 2000 original images to randomly augment the data, including rotation, scaling, and flipping, to expand the dataset to 5000 images. The dataset was randomly divided into training, validation, and test sets in a 7:2:1 ratio, with 3500 samples in the training set, 1000 samples in the validation set, and 500 samples in the test set. Subsequently, a Python program was used to convert the XML file into a TXT format file required by the YOLO algorithm for recognition and computation by the YOLO model.

[0122] 3. Rotation Target Detection Model Construction: To further achieve accurate detection of rotating targets, a Rot-YOLOv5 rotation target detection model (i.e., a rotation bounding box model based on YOLOv5) was constructed, improving the recognition performance of strips, clumps, and stems. Rot-YOLOv5 consists of three parts: Backbone, Neck, and Head. The Backbone is responsible for extracting features from the input image. By combining the EMA attention mechanism with the DSConv module, a C3-DEblock module is constructed, allowing the network to more effectively focus on the region of interest, thereby improving detection accuracy. The Neck fuses features at different scales to cope with complex scene changes and introduces the CAA-BiFPN module to enhance the ability of multi-scale feature fusion, alleviating the problem of inaccurate recognition caused by occlusion and overlap, and improving the model's performance in densely stacked scenes. The Head performs classification and regression, using the KLD loss function to regress the five parameters of the rotated bounding box. The KLD loss function treats the rotation angle as a probability distribution, solving the problem of periodic angles, and is particularly suitable for detecting abnormal tobacco shreds with large aspect ratios, thereby improving localization accuracy.

[0123] 4. Experimental Parameters and Evaluation Metrics: All model training and learning experiments were conducted using the Python version of the PyTorch deep learning framework, and validated and evaluated on a self-built abnormal tobacco shred dataset. The experiments ran on a Windows 10 operating system with the following hardware configuration: CPU i9-12900KF 3.2GHz, RAM 64GB, GPU NVIDIA GeForce RTX 4090, CUDA version 11.3. The specific training strategy was as follows: batch size was set to 16, model training consisted of 300 epochs, the initial learning rate was 0.01, and a momentum coefficient of 0.937 was used to accelerate gradient descent and prevent the model from getting trapped in local optima. The input image size was 640×640 pixels to ensure uniform image size. To verify the effectiveness of the abnormal tobacco shred detection algorithm, precision, recall, mean average precision (mAP), parameter count, and detection speed (framesper second, FPS) were used as evaluation metrics.

[0124] Regarding the accuracy of dimensional calculation, the detection results obtained by the rotation detection algorithm were compared with the actual dimensions of abnormal tobacco shreds measured by technicians using vernier calipers (accuracy 0.1 mm). Experimental data were obtained from a test set, including 100 samples of combined tobacco strips, 100 samples of loose tobacco, and 100 samples of stems. During filming, the width and length of the abnormal tobacco shreds were precisely measured using vernier calipers. To reduce measurement error, the length and width of each type of abnormal tobacco shred were measured three times independently, and the average of the measurements was taken as the actual size of that type of abnormal tobacco shred. Finally, the accuracy of the predicted dimensions of the abnormal tobacco shreds was evaluated using mean squared error (MSE) and mean absolute error (MAE). The specific calculation method is as follows:

[0125]

[0126]

[0127] Where n represents the number of abnormal tobacco leaves, Represents the actual truth value, and This represents the output value of the model.

[0128] 5. Results and Analysis: 5.1) Comparative Experiment and Analysis: The precision and recall curves of Rot-YOLOv5 and YOLOv5 were compared, and the results are as follows: Figure 12 As shown in the comparison results, Rot-YOLOv5 has a slightly higher precision than YOLOv5, and after 80 iterations, the precision curve of Rot-YOLOv5 remains higher than that of YOLOv5, even though their convergence speeds are similar. Regarding recall, the difference between Rot-YOLOv5 and YOLOv5 is small, with almost identical final recall rates. Therefore, the improved Rot-YOLOv5 achieves a higher precision while maintaining a recall rate similar to YOLOv5.

[0129] 5.2) Backbone Network Improvement Module Experiment: To analyze the performance of the proposed C3-DEblock, a comparative experiment was conducted on BottleNeck integrating DSConv and attention mechanisms. The EMA attention mechanism in the Rot-YOLOv5 model was compared with the currently popular CBAM (convolutional block attention module) and ECA (efficient channel attention) attention mechanisms. The results are shown in Table 2, which compares the detection accuracy of the models after embedding different modules.

[0130]

[0131] Table 2

[0132] Introducing dynamic snake-shaped convolution DSConv into the backbone network BottleNeck improved mAP by 0.16 percentage points. Building upon this, embedding CBAM, ECA, and EMA further improved mAP by 0.27, 0.23, and 0.5 percentage points, respectively, while increasing the number of parameters by 0.44, 0.38, and 0.41M, respectively, with virtually no change in computational cost. Compared to ECA, mAP improved by 0.27 percentage points with essentially the same computational cost.

[0133] 5.3) Comparison of Rotated and Horizontal Boxes: To compare the performance of the horizontal detection algorithm YOLOv5 and the rotation detection algorithm Rot-YOLOv5 in detecting abnormal tobacco shreds, a comparative experiment was conducted. The detection results are shown below. Figure 13 As shown in the figure. Experimental results show that the Rot-YOLOv5 algorithm can more accurately fit the true contours of the tobacco strips and stems in the output of the predicted bounding box, and can better reflect the orientation of abnormal tobacco strips in complex backgrounds. Especially when the tobacco strips are densely arranged at an inclined angle, the detection effect of Rot-YOLOv5 is more outstanding, and there are fewer false negatives, while the detection using ordinary rectangular boxes will cause visual interference and have obvious false negatives.

[0134] 5.4) Model Performance Comparison: The detection performance of different models on the test set is shown in Table 3. As can be seen from the table, the two-stage object detection model has the highest number of parameters, resulting in the slowest computation speed and the highest GPU resource consumption. Furthermore, Faster R-CNN performs poorly in detecting abnormal tobacco shavings, with an average accuracy of only 82.5%. This is mainly because the anchor boxes in Faster R-CNN need to be manually set, while the tobacco shavings are small targets. Manually set anchor boxes cannot accommodate such small targets, resulting in even the smallest anchor box being much larger than the tobacco shavings, thus affecting the detection accuracy of small targets.

[0135] YOLOv4-tiny had the lowest average detection accuracy because it used a lightweight CSPDarknet53-tiny backbone network. While this effectively reduced model complexity, it also limited its feature extraction capabilities for complex scenes and small targets. In contrast, Rot-YOLOv5 achieved an mAP of 96.51%, a 2.61 percentage point improvement over YOLOv5s, demonstrating higher recall and precision. Its inference time, number of parameters, and computational cost were similar to YOLOv5, and the introduction of the KLD loss function made it easier to optimize the regression loss for abnormal tobacco filaments, thus improving recognition accuracy. Further analysis revealed that the accuracy of horizontal bounding box detection was significantly lower than that of rotated bounding box detection. This is because stem targets are small, similar to the background, and easily occluded, making them difficult to detect accurately using traditional methods. Rot-YOLOv5, through the introduction of rotated bounding boxes, effectively avoided interference from redundant background information, improving its ability to identify difficult-to-detect targets.

[0136] The following table is Table 3:

[0137]

[0138] Table 3

[0139] 5.5) Improved model ablation experiment

[0140] Based on YOLOv5, a new C3-DEBlock was constructed by introducing a dynamic serpentine convolution module and an efficient multi-scale attention mechanism into the C3 module of the backbone network, enhancing the model's ability to detect slender targets. In the neck network, BiFPN was added to fuse feature maps of different scales, and a CAA module was embedded to make the model pay more attention to global contextual feature information, resulting in more accurate localization of targets with large-scale changes, such as stems, strips, and fragments. When selecting prediction boxes, the KLD loss function was used to improve the localization accuracy of abnormal tobacco shred detection boxes with rotation angle information. To verify the effect of the improved strategy on the model's detection performance, ablation experiments were conducted, and the results are shown in Table 4.

[0141]

[0142] Table 4

[0143] As shown in Table 4, the original YOLOv5 model achieved an mAP of 93.9% on the test set. The KLD loss function was used on the baseline model to improve the localization accuracy of the abnormal tobacco detection boxes with rotation angle information, effectively avoiding missed detections of overlapping and merging tobacco strands, thus improving the mAP by 0.66 percentage points. Based on the addition of KLD, the receptive field was expanded by using C3-DEBlock to generate the output feature layer in the backbone network, enabling the model to handle abnormal tobacco targets of different scales, thus improving the mAP by 0.29 percentage points. The introduction of BiFPN into the neck network, through the fusion of different... Feature maps of the same scale enrich the feature information they contain, improving mAP by 0.87 percentage points. Adding a CAA module to the original PANet feature fusion branch makes the model focus more on the target location region of abnormal tobacco shreds, improving the model's mAP by 0.24 percentage points. The effectiveness of the combination of improved modules was verified by combining them pairwise, showing improvements of 1.25, 0.66, and 0.9 percentage points respectively compared to the model with the KLD loss function. Embedding four improved modules into the baseline model resulted in the highest mAP, improving it by 2.61 percentage points compared to the original model. Ablation experiments show that all four improved modules significantly improve the model's detection accuracy, and when used simultaneously, the model's mAP reaches 96.51%.

[0144] 5.6) Feature Visualization Analysis and Performance Comparison of the Rot-YOLOv5 Model: Grad-CAM performed feature visualization analysis on the model's detection performance, such as... Figure 14 As shown in the figure, Rot-YOLOv5, based on rotation detection, concentrates its high-response region on the core contour of the target. Compared to YOLOv5, which focuses on horizontal detection, Rot-YOLOv5 has a significantly larger response region on stripes, while YOLOv5's response region on stripes is negligible. This is because the addition of dynamic serpentine convolution adaptively focuses the receptive field on slender targets. Simultaneously, Rot-YOLOv5's response region on stems is also more prominent, demonstrating good detection performance even for small targets at a distance. Therefore, Rot-YOLOv5 exhibits significantly higher attention and focus on the target object.

[0145] 5.7) Counting and Size Analysis in Different Scenarios: To further verify the counting performance of the Rot-YOLOV5 model proposed in this paper, such as... Figure 15As shown, this paper selects three complex scenarios—dense stacking, multiple targets, and small targets—for experiments to evaluate the model's counting accuracy and size adaptability. In dense stacking scenarios, targets are easily obscured by stripes, making them difficult to detect accurately and indistinguishable from the background. However, the introduction of a rotated bounding box effectively removes redundant background information, significantly improving target detection and counting accuracy. In multi-target scenarios, the network achieves efficient fusion of deep and shallow features by integrating a multi-scale attention mechanism and a bidirectional feature pyramid network, further improving the detection and counting accuracy of multi-scale targets. In small target scenarios, the network introduces a context anchor attention mechanism, which can mine long-range global contextual feature relationships, thereby enhancing the detection and counting capabilities of small targets. Building upon the powerful detection capabilities of the Rot-YOLOV5 model, not only is the counting accuracy improved, but the rotation of the bounding box to closely fit the target further improves the accuracy of measuring abnormal tobacco W / L pixel values.

[0146] To verify the performance of the improved model in the abnormal tobacco detection and counting task, 300 photos were selected from the test set, including 260 samples of combined tobacco strips, 180 samples of loose tobacco, and 96 samples of stems. In the counting task, the score threshold for the predicted bounding boxes was set to 0.7, effectively filtering out those predicted boxes with low confidence. By comparing the results of manual counting with the prediction results of the improved model, Table 5 clearly shows the significant advantages achieved by the improved model in the experiment. The improved Rot-YOLOv5 model achieved an accuracy of over 90% in the total counting of abnormal tobacco strips across all categories, providing more reliable results for practical applications. Particularly noteworthy is the significant advantage shown by the improved model in the stem counting task, highlighting its improvement in small object detection. This indicates that the improved model can provide more accurate abnormal tobacco strip counting results in complex environments.

[0147]

[0148] Table 5

[0149] 5.8) Measurement and Analysis of Abnormal Tobacco Phenotypes: Comparison of automatically extracted and measured values ​​of six phenotypic traits (length of bundled strips, width of bundled strips, length of stem, width of stem, length of leaf fragment, width of leaf fragment), such as... Figure 16 As shown.

[0150] One hundred samples of rolled tobacco, loose tobacco, and stem tobacco were tested, and phenotypic parameters such as rolled tobacco length, rolled tobacco width, stem tobacco length, stem tobacco width, loose tobacco length, and loose tobacco width were automatically extracted. The comparison between the extracted values ​​and the measured values ​​showed coefficients of determination of 0.98, 0.985, 0.99, 0.985, 0.995, and 0.98, respectively, indicating a high correlation between the calculated and measured values ​​of the phenotypic parameters for various abnormal tobacco shreds. Table 6 shows that the manually calculated dimensions are close to the model calculations. The calculated average lengths of rolled tobacco, loose tobacco, and stem tobacco were 44.89 mm, 15.03 mm, and 26.62 mm, respectively, and the calculated average widths were 5.03 mm, 3.90 mm, and 1.81 mm, respectively. Furthermore, the MSE (Mean Sequence Estimation) of the length and width of the three abnormal tobacco shreds using the Rot-YOLOv5 model was no greater than 0.11, indicating that this method performed best and closely approximated the actual measured values. This method is suitable for extracting phenotypic parameters of abnormal tobacco shreds under complex backgrounds and has good versatility, which can be further extended to the phenotypic measurement of more types of tobacco shreds.

[0151] Table 6

[0152] 6. Conclusion: Experiments have shown that this method has high detection accuracy, with a mean average precision (mAP) of 96.51%. This method can not only effectively detect rotating targets, but also minimize background redundancy in the abnormal tobacco detection area.

[0153] Compared to the YOLOv5 model, the improved Rot-YOLOv5 model achieves significant improvements in abnormal tobacco identification, with an average recognition accuracy increase of 2.61 percentage points and an F1 score increase of 3.1 percentage points, while reducing the model size by 0.19 MB. This improved model not only excels in accuracy and size but also effectively reduces model size, facilitating its transfer and application in different scenarios.

[0154] The improved Rot-YOLOv5 model can effectively predict the location of abnormal tobacco shreds and accurately count them, achieving a counting accuracy of over 90% for all categories of abnormal tobacco shreds. By employing a rotation detection region localization algorithm, the skeleton of abnormal tobacco shreds was successfully reconstructed, and phenotypic information such as fused tobacco, broken tobacco, and stems were accurately extracted. Test results show that the coefficients of determination for regression analysis with manually measured abnormal tobacco shred length and width are 0.98, 0.985, 0.99, 0.985, 0.995, and 0.98, respectively. The mean squared error (MSE) of the length and width estimates for the three types of abnormal tobacco shreds is less than 0.11, indicating that the errors of the above phenotypic parameters are within an acceptable range. Therefore, this method demonstrates that it can accurately extract the phenotypic information of abnormal tobacco shreds.

[0155] Specific implementation of an abnormal tobacco detection and phenotypic measurement device based on the YOLOv5 rotating frame model:

[0156] Another aspect of the present invention proposes an abnormal tobacco detection and phenotypic measurement device based on a rotating frame model of YOLOv5, including a processor, which is used to execute the above-mentioned abnormal tobacco detection and phenotypic measurement method based on a rotating frame model of YOLOv5. For specific implementation of the device, please refer to specific implementations 1-3 of the method for abnormal tobacco detection and phenotypic measurement based on a rotating frame model of YOLOv5, which will not be repeated here.

[0157] In summary, to accurately locate and count abnormal tobacco shreds in complex tobacco processing environments with interference and occlusion, this invention proposes a method and device for constructing a rotating bounding box model based on YOLOv5, detecting abnormal tobacco shreds, and measuring phenotypic characteristics. First, an efficient multi-scale attention (EMA) module, dynamic serpentine convolution, and a C3 module are combined in the backbone network to construct C3-DEBlock, which adaptively adjusts the receptive field, thereby enhancing feature extraction capabilities. Second, a context-anchored bidirectional feature pyramid network (CAA-BiFPN) is introduced into the neck network to capture long-distance contextual information and enhance multi-scale feature fusion capabilities. Finally, the Kullback-Leibler divergence between Gaussian distributions is used as the regression loss function, allowing the parameter gradient to be dynamically adjusted according to the target features, thus enabling more accurate bounding box regression. Experiments show that compared with mainstream object detection models Faster R-CNN, YOLOv4-tiny, and YOLOv5, the mAP is improved by 14.01, 25.21, and 2.61 percentage points, respectively. Regression analysis was performed with manually measured abnormal tobacco length and width, yielding coefficients of determination of 0.98, 0.985, 0.99, 0.985, 0.995, and 0.98, respectively. This method accurately locates abnormal tobacco using a rotating frame technique, significantly reducing interference from the background area. It provides an online detection solution for the precise counting of abnormal tobacco and the accurate grading and optimization of tobacco cutting quality, contributing to the modernization of tobacco processing technology.

Claims

1. A method for constructing a rotated frame model based on YOLOv5, characterized in that, include: Based on the YOLOv5 model architecture, a feature extraction module containing multi-scale attention mechanism and dynamic snake convolution is constructed in the backbone network; A context-anchor-based bidirectional feature pyramid network is designed in the neck network to capture long-range contextual information and extract features bidirectionally; The KLD loss function is introduced into the head network to generate a YOLOv5-based rotated box model for object detection using rotated boxes.

2. The method for constructing a YOLOv5-based rotated frame model according to claim 1, characterized in that, The feature extraction module is constructed using the following steps: 1) Combine multi-scale attention mechanisms and dynamic serpentine convolutions to generate DEblock modules; 2) The feature extraction module is obtained by following the order of the convolution module, the DEblock module and the convolution module.

3. The method for constructing a YOLOv5-based rotated frame model according to claim 2, characterized in that, The feature extraction module is composed of two convolutional modules, several deblocking modules, and one convolutional module.

4. The method for constructing a YOLOv5-based rotated frame model according to claim 2, characterized in that, The Deblock module is composed of one layer of dynamic serpentine convolution, two layers of standard convolutional layers, and one layer of multi-scale attention mechanism module.

5. The method for constructing a YOLOv5-based rotated frame model according to claim 1, characterized in that, A context-anchor-based bidirectional feature pyramid network for capturing long-range contextual information and extracting features bidirectionally is designed in the neck network using the following method: In the path aggregation network structure, the context anchor-based bidirectional feature pyramid network is obtained by bidirectional information fusion and the introduction of a context anchor attention mechanism to capture long-distance context information.

6. The method for constructing a YOLOv5-based rotated frame model according to claim 1, characterized in that, The KLD loss function is expressed by the following formula: ； Among them, L kld Let represent the regression loss of the bounding box; D be the loss value; sqrt(D) be a nonlinear function containing the loss value D; τ be a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula: ； Among them, u p The mean of the predicted distribution; u t Tr is the mean of the true distribution; Tr is the trace of the covariance matrix of the true distribution. The covariance matrix of the true distribution; It is the covariance matrix of the predicted distribution.

7. A method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a rotating frame model of YOLOv5, characterized in that, include: A pre-built YOLOv5-based rotating bounding box model is invoked to detect abnormal tobacco shreds from an original image containing several tobacco shreds, and the phenotypic parameters of the abnormal tobacco shreds are obtained. The YOLOv5-based rotating bounding box model is constructed through the following steps: 1) Based on the YOLOv5 model architecture, a feature extraction module containing multi-scale attention mechanism and dynamic snake convolution is built in the backbone network; 2) In the neck network, a bidirectional feature pyramid network based on context anchor is designed to capture long-distance context information and extract features bidirectionally; 3) In the head network, the KLD loss function is introduced to generate a YOLOv5-based rotated box model for object detection using rotated boxes.

8. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 7, characterized in that, The feature extraction module is constructed using the following steps: 1) Combine multi-scale attention mechanisms and dynamic serpentine convolutions to generate DEblock modules; 2) The feature extraction module is obtained by following the order of the convolution module, the DEblock module and the convolution module.

9. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 8, characterized in that, The feature extraction module is composed of two convolutional modules, several deblocking modules, and one convolutional module.

10. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 8, characterized in that, The Deblock module is composed of one layer of dynamic serpentine convolution, two layers of standard convolutional layers, and one layer of multi-scale attention mechanism module.

11. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 7, characterized in that, A context-anchor-based bidirectional feature pyramid network for capturing long-range contextual information and extracting features bidirectionally is designed in the neck network using the following method: In the path aggregation network structure, the context anchor-based bidirectional feature pyramid network is obtained by bidirectional information fusion and the introduction of a context anchor attention mechanism to capture long-distance context information.

12. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 7, characterized in that, The KLD loss function is expressed by the following formula: ； Among them, L kld Let represent the regression loss of the bounding box; D be the loss value; sqrt(D) be a nonlinear function containing the loss value D; τ be a hyperparameter that adjusts the entire loss function; where D is expressed by the following formula: ； Among them, u p The mean of the predicted distribution; u t Let be the mean of the true distribution, and Tr be the trace of the covariance matrix of the true distribution; The covariance matrix of the true distribution, It is the covariance matrix of the predicted distribution.

13. The method for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a YOLOv5 rotating frame model according to claim 7, characterized in that, The phenotypic parameters include the length and / or width of the abnormal tobacco shreds.

14. A device for detecting abnormal tobacco shreds and measuring phenotypic characteristics based on a rotating frame model of YOLOv5, characterized in that, Includes a processor for executing the abnormal tobacco detection and phenotypic measurement method based on the YOLOv5 rotating frame model as described in any one of claims 7-13.

Images