Abnormal tobacco shred identification and detection method and device based on YOLO model

Abstract

The invention provides an abnormal cut tobacco identification and detection method and device based on a YOLO model, and belongs to the technical field of cigarette manufacturing. The abnormal tobacco shred identification and detection method comprises the following steps: acquiring an abnormal tobacco shred image and constructing a data set of the abnormal tobacco shred image, wherein the abnormal tobacco shred image takes densely stacked normal tobacco shred as a background; an abnormal tobacco shred recognition model based on a YOLO model is constructed, the abnormal tobacco shred recognition model at least comprises a backbone network, a neck network and a head network, and a BiFormer attention mechanism based on a Transform architecture is introduced into the neck network; training the abnormal tobacco shred identification model through the data set; and identifying abnormal tobacco shreds in the tobacco shred pile by using the trained abnormal tobacco shred identification model. Through a deep learning algorithm, the identification precision is remarkably improved, and abnormal tobacco shreds with the density similar to that of normal tobacco shreds are effectively distinguished.

Description

Technical Field

[0001] The present invention belongs to the technical field of cigarette manufacturing, and in particular relates to a method and a device for detecting abnormal shredded tobacco based on a YOLO model. Background Art

[0002] In the process of cigarette making, abnormal tobacco needs to be removed to ensure the quality of the final cigarette product. Therefore, abnormal tobacco detection and removal is one of the important processes in the production process of silk. At present, the tobacco industry often uses wind selection equipment or adjusts cigarette units in the silk production line to reduce the content of abnormal tobacco in cigarettes and improve the quality of cigarette production. Since the wind selection equipment uses the difference in density between abnormal tobacco and normal tobacco to remove heavier abnormal tobacco, it also makes it difficult for the wind selection equipment to effectively remove stems, runaway pieces, and strips of similar specific gravity. However, the presence of runaway pieces, strips, and stems in cigarette products will not only affect the combustion performance and sensory quality of cigarettes to a certain extent, increasing product quality risks; it will also cause blockages in the subsequent links of the production line, affecting the efficiency and progress of the entire production line.

[0003] In recent years, deep learning technology has been used to identify and detect tobacco in various fields of tobacco production and processing. For example, a cigar cigarette quality detection device detects defects on the surface of tobacco pillars, with an average accuracy rate MAP of 87.7%; a tobacco stem classification and recognition model was established after improving the YOLOv3 model, and the recognition accuracy reached 95.01%; a method for tobacco package defect detection was established based on the improved YOLOv4-tiny model, with an average detection time of 17ms; the generated tobacco insect images were trained with the YOLOv3 model using data enhancement technology and k-means++ clustering method, which improved the recognition accuracy of tobacco insect position, tobacco and tobacco end; the YOLOv5 model was used to train five types of tobacco leaves with different maturity levels, and detection models of different maturity levels were constructed to achieve accurate detection of fresh leaf grades; Ghost was used to train the generated tobacco insect images ... Ghost was used to train five types of tobacco leaves with different maturity levels, and detection models of different maturity levels were constructed to achieve accurate detection of fresh leaf grades; Ghost was used to train the generated tobacco insect images with the YOLOv3 model using data enhancement technology and k-means++ clustering method, which improved the recognition accuracy of tobacco insect position, tobacco and tobacco end; Ghost was used to train five types of tobacco leaves with different maturity levels, and detection models of different maturity levels were constructed to achieve accurate detection of fresh leaf grades; Ghost was used to train the generated tobacco insect images with the YOLOv3 model using data enhancement technology and k-means++ clustering method to improve the recognition accuracy of tobacco insect position, tobacco and tobacco end; Ghost was used to train five types of tobacco Net combined with YOLOv5, the lightweight YOLOv5 algorithm accurately detects tea buds in similar backgrounds, which is 7.84% higher than the original algorithm; a lightweight YOLOv8 algorithm introduces FasterNet blocks in the backbone part and adds an asymptotic feature pyramid network structure to improve the detection accuracy of pests and diseases, reaching 92.6%. However, most of the above technical solutions focus on the detection of targets with distinct features, while the detection of abnormal tobacco with similar backgrounds and shapes in dense states is rarely reported.

[0004] The detection of abnormal tobacco using deep learning technology, especially the detection of abnormal tobacco with a density similar to that of normal tobacco, has the following difficulties: first, the color of the tobacco being detected is similar to the background, and the size of the runners and stems is small, which will reduce the detection accuracy and cause errors, and the robustness is not high; second, the existing target detection algorithm cannot meet the requirements of detection on industrial production lines in terms of detection speed; third, in the case of densely stacked state, the existing algorithm still has a certain deviation in the positioning of the detection target.

[0005] In view of this, the inventor, based on many years of production design experience in this field and related fields, has designed a method and device for identifying and detecting abnormal tobacco based on the YOLO model after repeated experiments, in order to solve the problems existing in the prior art. Summary of the invention

[0006] The object of the present invention is to provide a method and device for identifying and detecting abnormal shredded tobacco based on the YOLO model, which can effectively identify abnormal shredded tobacco with a density similar to that of normal shredded tobacco.

[0007] To achieve the above-mentioned object of the invention, the present invention proposes an abnormal tobacco shreds recognition and detection method based on the YOLO model, wherein abnormal tobacco shreds images are collected and a data set of abnormal tobacco shreds images is constructed, wherein the abnormal tobacco shreds images are based on densely stacked normal tobacco shreds as the background;

[0008] Constructing an abnormal tobacco recognition model based on the YOLO model, wherein the abnormal tobacco recognition model includes at least a backbone network, a neck network, and a head network, and introducing a BiFormer attention mechanism based on a Transformer architecture into the neck network;

[0009] Training the abnormal tobacco recognition model using the data set;

[0010] The trained abnormal tobacco cuts identification model is used to identify abnormal tobacco cuts in the tobacco cuts pile.

[0011] Compared with the prior art, the present invention has the following characteristics and advantages:

[0012] The present invention proposes a method and device for detecting abnormal tobacco shreds based on the YOLO model. The improved deep learning algorithm significantly improves the recognition accuracy and effectively distinguishes abnormal tobacco shreds with a density similar to that of normal tobacco shreds. The method and device for detecting abnormal tobacco shreds proposed by the present invention can operate stably under complex backgrounds and different lighting conditions, meeting the real-time detection requirements on high-speed production lines, while reducing the demand for computing resources through lightweight model design. This not only reduces the cost of manual inspection and avoids production interruptions, but also improves production efficiency and product quality.

[0013] Another effect of the present invention is that the abnormal tobacco identification and detection device based on the YOLO model is easy to integrate with the existing production line system, showing good flexibility and scalability, providing data support and analysis tools for production decisions, bringing innovative quality control solutions to the tobacco processing industry, and promoting industry technological progress and optimization of production processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings described herein are only for explanation purposes and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the various components in the figures are only schematic, used to help understand the present invention, and are not specifically limited to the shapes and proportional dimensions of the various components of the present invention. Those skilled in the art can select various possible shapes and proportional dimensions to implement the present invention according to the teachings of the present invention.

[0015] Figure 1 A schematic diagram of a tobacco data image in an embodiment of the present invention;

[0016] Figure 2 Schematic diagram of an industrial camera acquisition device in an embodiment of the present invention;

[0017] Figure 3 Schematic diagram of data enhancement of tobacco cut image in an embodiment of the present invention;

[0018] Figure 4 Schematic diagram of abnormal tobacco marking in a tobacco image in an embodiment of the present invention;

[0019] Figure 5 It is a schematic diagram of the network structure of YOLOv5 of the present invention;

[0020] Figure 6 Schematic diagram of the Biformer attention mechanism of the present invention;

[0021] Figure 7 This is a schematic diagram of the C3-Faster module of the present invention;

[0022] Figure 8 Schematic diagram of the predicted frame and the real frame of the present invention;

[0023] Fig. 9 It is a schematic diagram of the network structure of CBS-YOLOv5 of the present invention;

[0024] Fig.10 A schematic diagram of the number of abnormal tobacco categories in an embodiment of the present invention;

[0025] Fig.11 It is a schematic diagram of a PR curve diagram in an embodiment of the present invention;

[0026] Fig.12Schematic diagram of Loss curve in an embodiment of the present invention;

[0027] Fig.13 Schematic diagram of visualization results of different models in the embodiments of the present invention;

[0028] Fig.14 This is a schematic diagram of the change of abnormal tobacco shreds over time in an embodiment of the present invention;

[0029] Fig.15 The figure is a flow chart of the abnormal tobacco recognition and detection method based on the YOLO model of the present invention. DETAILED DESCRIPTION

[0030] The details of the present invention can be more clearly understood with reference to the accompanying drawings and the description of the specific embodiments of the present invention. However, the specific embodiments of the present invention described herein are only used for the purpose of explaining the present invention and cannot be understood as limiting the present invention in any way. Under the guidance of the present invention, technicians can conceive of any possible variations based on the present invention, which should all be considered to belong to the scope of the present invention.

[0031] The present invention proposes an abnormal tobacco recognition and detection method based on the YOLO model, which includes collecting abnormal tobacco images and constructing a data set of abnormal tobacco images, wherein the abnormal tobacco images are based on densely stacked normal tobacco as the background; constructing an abnormal tobacco recognition model based on the YOLO model, wherein the abnormal tobacco recognition model at least includes a backbone network, a neck network and a head network, and a BiFormer attention mechanism based on a Transformer architecture is introduced into the neck network; the abnormal tobacco recognition model is trained through the data set; and the abnormal tobacco in the tobacco pile is recognized using the trained abnormal tobacco recognition model.

[0032] The abnormal tobacco shreds identification and detection method based on the YOLO model proposed in the present invention significantly improves the recognition accuracy through its improved deep learning algorithm, and effectively distinguishes abnormal tobacco shreds with a density similar to that of normal tobacco shreds. The abnormal tobacco shreds identification and detection method can operate stably under complex backgrounds and different lighting conditions, realizing the real-time detection requirements on high-speed production lines, while reducing the computing resource requirements through lightweight model design. This not only reduces the cost of manual inspection and avoids production interruptions, but also improves production efficiency and product quality.

[0033] In an optional embodiment of the present invention, Figure 1As shown, abnormal tobacco shreds include at least running sheets, drawing strips and stems. For the three common abnormal tobacco shreds in the process of making tobacco: running sheets, drawing strips and stems. The tobacco shreds image is obtained by an image acquisition device arranged above the conveyor belt in the making workshop, and the image is analyzed and processed by a deep learning model. Specifically, the model first learns the image features of normal tobacco shreds through the pre-training stage, and then further learns the abnormal features of running sheets, drawing strips and stems in the post-training stage. In this way, the model can accurately identify and classify the three abnormal tobacco shreds, and realize the rapid detection of abnormal tobacco shreds. This embodiment realizes the efficient identification and detection of abnormal tobacco shreds, reduces the cost of manual inspection, and avoids product quality problems caused by missed inspections. Its automation characteristics improve production efficiency and ensure the consistency and stability of product quality. The real-time response capability of the model can timely eliminate abnormal tobacco shreds. At the same time, data enhancement processing enhances the generalization of the model, providing an innovative quality control solution for the tobacco processing industry, meeting the production needs of automation and intelligence.

[0034] In an optional embodiment of the present invention, a data set is expanded by a data enhancement method, and the abnormal tobacco recognition model is trained using the expanded data set. In order to improve the generalization ability and detection accuracy of the abnormal tobacco recognition model, the original data set is expanded by a data enhancement method. First, tobacco images containing abnormal tobacco types such as run-off, drawing, and stems are collected, and then new images are generated by data enhancement techniques such as random brightness adjustment, Gaussian noise addition, random flipping, translation, and random cropping. These enhancement operations not only increase the diversity of the data set, but also simulate various situations that may occur in the actual production environment, so that the model can better learn and adapt to different detection conditions. The data set expanded by this method is used to train the abnormal tobacco recognition model, ensuring the robustness and accuracy of the model in practical applications. The application of this enhancement technology effectively avoids the overfitting of the model to the original data set and improves the generalization ability of the model in actual production. The results show that the model trained with the expanded data set can more accurately identify abnormal types such as run-off, drawing, and stems when detecting abnormal tobacco, reducing the situation of false detection and missed detection, thereby improving the quality control level in the tobacco making process.

[0035] In an optional embodiment of the present invention, the LabelImg tool is used to mark the abnormal tobacco in the abnormal tobacco image with the minimum circumscribed rectangle, and an XML type label file is corresponding to each abnormal tobacco image. LabelImg is a commonly used graphical interface annotation tool, which allows the user to draw the minimum circumscribed rectangle on the image by dragging the mouse to accurately cover the abnormal tobacco in the image. The circumscribed rectangle annotation of each abnormal tobacco not only defines its position in the image, but also clarifies its category. After the annotation is completed, the LabelImg tool automatically generates a corresponding XML format label file for each abnormal tobacco image. These label files record the position information and category information of the abnormal tobacco, and provide necessary structured data for subsequent model training. The precise annotation and XML label file generation using the LabelImg tool provide high-quality annotation data for the training of the abnormal tobacco recognition model. This embodiment ensures that the model can learn the accurate features and boundaries of the abnormal tobacco, thereby improving the recognition and positioning accuracy of the model. In addition, the structured XML label file is easy to manage and maintain, and also facilitates the expansion and update of the data set. In this way, the technical solution of the present invention not only improves the training efficiency of the model, but also enhances the model's ability to detect abnormal tobacco, so that the final detection system can be more reliably applied to actual tobacco production lines.

[0036] In an optional embodiment of the present invention, the convolutional layers in the three CBS blocks of the C3 module in the backbone network are replaced with partial convolutional layers and / or PWConv point convolutional layers. By replacing the convolutional layers of three of the CBS blocks with partial convolutional layers (PConv) and / or point convolutional layers (PWConv), a significant improvement in model efficiency and performance is achieved. The improvement takes advantage of the advantages of PConv in reducing the amount of calculation and the characteristics of PWConv in reducing the number of parameters while maintaining the ability to represent features. This embodiment reduces the number of parameters and computational complexity of the model through the designed network structure adjustment without sacrificing the efficiency of feature extraction. This improvement not only makes the model more suitable for environments with limited computing resources, but also improves the running speed of the model in real-time detection, meeting the requirements of industrial production lines for high efficiency. At the same time, since the optimized C3 module maintains efficient feature extraction capabilities, the model's recognition accuracy for abnormal tobacco is guaranteed, thereby improving the overall performance, making the technical solution of the present invention more efficient and practical in actual industrial applications.

[0037] In an optional embodiment of the present invention, the Shape-IoU Loss function is used as the bounding box regression loss function of the abnormal tobacco recognition model. The Shape-IoU Loss function is used as the loss function of the bounding box regression of the abnormal tobacco recognition model. The loss function pays special attention to the shape and scale of the bounding box, thereby optimizing the positioning accuracy of the model for abnormal tobacco. Through this embodiment, the model uses the Shape-IoU Loss function to carefully adjust the bounding box prediction during the training process to ensure the minimized difference between the predicted box and the real box. This method not only speeds up the convergence of the model, but also improves the training efficiency. Its Shape-IoU Loss function significantly improves the accuracy of bounding box positioning and reduces false detections and missed detections, especially in the detection of small targets and complex backgrounds; secondly, due to the sensitivity of the loss function to shape information, the model's generalization ability for abnormal tobacco of different shapes is enhanced, and the applicability and robustness of the model in an actual production environment are improved.

[0038] In an optional embodiment of the present invention, the data set is divided into a training set, a validation set and a test set, and the training set is transmitted to the abnormal tobacco recognition model for training. The tobacco image data set is divided into a training set, a validation set and a test set in a ratio of 7:2:1. The training set is used for model learning and occupies the main part of the data set, while a smaller part of the data is used as a validation set and a test set, respectively. The former is used for model tuning to avoid overfitting, and the latter is used to evaluate the final performance of the model. The training set helps the model capture the characteristics of abnormal tobacco, the validation set ensures that the model does not overfit the training data, and the independent evaluation of the test set proves the generalization ability of the model.

[0039] In an optional example of this implementation, the initial learning rate of the abnormal tobacco identification model is set to 0.01, and the Adam adaptive gradient descent algorithm is used. In order to optimize the training process of the abnormal tobacco identification model, the initial learning rate of the model is set to 0.01, and the Adam adaptive gradient descent algorithm is selected as the optimizer. The Adam algorithm combines the advantages of momentum and adaptive learning rate adjustment (AdaGrad), and can automatically adjust the learning rate of each parameter. In the early stage of model training, the setting of the initial learning rate provides a moderate exploration space for the model, and as the training proceeds, the Adam algorithm will dynamically adjust the learning rate according to the first-order moment and second-order moment estimation of the gradient, thereby accelerating the convergence speed and improving the training efficiency. The implementation method of using an initial learning rate of 0.01 combined with the Adam adaptive gradient descent algorithm, a reasonable initial learning rate ensures that the model can converge quickly in the early stage of training, while avoiding the problem of unstable gradient estimation due to excessively high learning rate. The adaptive nature of the Adam algorithm enables the model to intelligently adjust the learning rate for different parameters, which helps to handle parameter updates of different scales and directions. This optimization strategy enables the model to find the minimum value of the loss function more quickly during training, thereby improving the model's recognition accuracy and generalization ability.

[0040] In an optional example of this embodiment, the abnormal tobacco recognition model is tested using a test set and the abnormal tobacco recognition model is evaluated based on the test results. This embodiment evaluates the abnormal tobacco recognition model by using an independently divided test set. The data in the test set is not used in the model training phase, which ensures the fairness of the evaluation and the generalization ability of the model. The model recognizes the images in the test set, and its output results are compared with the actual annotations to calculate the key performance indicators of the model, such as recognition accuracy and recall rate. This evaluation method ensures accurate measurement of model performance and reflects the performance of the model in practical applications.

[0041] In an optional example, the indicators used to evaluate the abnormal tobacco identification model include at least average detection accuracy, precision, recall, network parameter quantity and detection speed. The present invention uses a series of key indicators to evaluate the performance of the abnormal tobacco identification model, which comprehensively covers the accuracy, efficiency and complexity of the model, ensuring the multi-dimensionality and depth of the evaluation. Specifically, the average detection accuracy reflects the comprehensive recognition ability of the model in different categories, and the improvement of the average detection accuracy shows the high efficiency of the model in multi-category recognition; the precision and recall rate measure the correct recognition rate and coverage of the model respectively to ensure the accuracy and reliability of the model in practical applications; the network parameter quantity reflects the complexity of the model, and the control of the network parameter quantity helps to reduce the computational cost of the model, making it more suitable for deployment in a resource-constrained environment; the detection speed is directly related to the feasibility of the model in practical applications, meeting the real-time requirements of industrial-grade applications. Through the measurement and analysis of these indicators, the performance of the model can be comprehensively evaluated and further optimization can be guided.

[0042] The present invention also proposes an abnormal tobacco recognition and detection device based on the YOLO model, and the abnormal tobacco recognition and detection device includes:

[0043] An image acquisition module is disposed above the tobacco shreds transfer conveyor belt and is capable of acquiring images of densely stacked tobacco shreds on the tobacco shreds transfer conveyor belt;

[0044] The image recognition module inputs the image of the tobacco into a pre-trained abnormal tobacco recognition model, and recognizes the abnormal tobacco in the image of the tobacco through the normal tobacco recognition model;

[0045] The output module outputs the number of abnormal tobacco shreds in the image of each tobacco shred.

[0046] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the abnormal tobacco recognition and detection method based on the YOLO model are implemented as described above.

[0047] The present invention also proposes a computer device, including a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, wherein the processor implements the steps of the abnormal tobacco recognition and detection method based on the YOLO model when executing the computer program.

[0048] The specific implementation process of the present invention is now described in detail in conjunction with an embodiment:

[0049] 1 Materials and methods

[0050] 1.1 Materials

[0051] The tobacco samples were collected from the tobacco production workshop in Longyan, Fujian in 2023. Different types of abnormal tobacco were collected. The abnormal tobacco collected had great differences in length, width and shape. In order to obtain tobacco data faster and more efficiently, tobacco collected directly from the tobacco production site was used, which is more in line with the needs of actual industrial production. Figure 2 As shown, an image of densely stacked cut tobacco is displayed, including run pieces, drawn strips, stems and normal cut tobacco.

[0052] A tobacco image acquisition device is built above the running conveyor belt, which can easily obtain images, such as Figure 3 As shown. The industrial camera is a MARS-1231-46G5C-P Mars (MARS-G5-P) series digital camera (Beijing Daheng Image Co., Ltd.), the lens is HN-P-1224-20M-C1.1 / 1 (Beijing Daheng Image Co., Ltd.), the resolution is 20M, and the image acquisition software is GalaxyView. When shooting tobacco, the upper and lower adjustable shooting bracket fixes the industrial camera above the conveyor belt parallel to the conveyor belt, and the camera lens is placed horizontally downward above the center of the conveyor belt to prevent the camera's field of view from being too large and shooting the belt that is not covered by tobacco. The height of the bracket can be adjusted between 10 and 50 cm, and the industrial camera can be controlled by a computer.

[0053] 1.3 Dataset Construction

[0054] The dataset uses tobacco images taken by high-speed industrial cameras on the industrial assembly line of the Longyan Tobacco Factory, including three types of abnormal tobacco: runaway, drawn, and stemmed. A total of 2,000 images were collected. In order to avoid overfitting, enhance the generalization ability of the model and improve the accuracy of detection, the dataset was enhanced using data enhancement methods such as random brightness, Gaussian noise, random flipping, translation, and random cropping, and expanded to 5,000 images, which improved the robustness of the model and enabled it to better process tobacco images in various production environments. The enhanced image effect can be seen in the figure. Figure 3 The training set, validation set, and test set are allocated in a ratio of 7:2:1. The training set samples are 3500, the validation set samples are 1000, and the test set samples are 500. The LabelImg tool is used to mark the abnormal tobacco in the image with the minimum bounding rectangle. Each image corresponds to an XML-type label file to record the location and category of the abnormal tobacco. Figure 4 , except for abnormal tobacco, no other tobacco is labeled, including normal tobacco and thick and thin tobacco.

[0055] 1.4 Abnormal tobacco identification model

[0056] 1.4.1CBS-YOLOv5 algorithm construction

[0057] Due to the complex background of abnormal tobacco, it is impossible to use traditional image processing technology to identify abnormal tobacco. In this study, the YOLOv5s model based on deep learning is used to detect abnormal tobacco. Figure 5 As shown in the figure, the original 640×640×3 image is transformed into three feature maps of different scales for detecting different targets after a series of operations such as convolution. The feature map is extracted by C3, Conv and SPPF modules. The C3 module greatly increases the depth and receptive field of the network and improves the network's feature extraction capability. The FPN+PAN structure is used in the Neck part. This pyramid structure can integrate shallow position information with deep semantic information, thereby enhancing the network's detection capability at multiple scales. Based on the original YOLOv5s model, in the task of detecting abnormal tobacco in the assembly line, the color background of normal tobacco and abnormal tobacco are similar, the run-off and stem targets are small, and there is a dense distribution of occlusion, which results in low detection accuracy of the existing algorithm, frequent false detection and missed detection problems, and the existing model is improved to establish an abnormal tobacco recognition model.

[0058] ① The BiFormer attention mechanism based on the Transformer architecture is introduced at the neck of the network, which is conducive to increasing the weight of the network's allocation to abnormal tobacco features, improving the network's ability to understand the semantic information of image features, paying more attention to the target to be identified, and improving the accuracy of small target detection. Figure 6 As shown, the Biformer attention mechanism first divides the feature map obtained by the convolution operation of the input tobacco image into multiple grid areas, and then obtains Q, V, and K through linear mapping. Secondly, for each area, the average values ​​Qr and Kr of the area to be paid attention to are obtained. The adjacency matrix Ar between each area is obtained by transposing and multiplying Qr and Kr to indicate the degree of connection between the semantic information of each area. The routing index matrix is ​​obtained by pruning the affinity graph to obtain the first K most closely connected areas, and then the key-value pairs of the K areas are obtained, and the attention mechanism is applied to the key-value pairs. ② In order to reduce the number of parameters brought by BiFormer, after considering the trade-off between the number of parameters and accuracy, the lightweight idea of ​​the FasterNet model is adopted. The Conv layer in the three CBS blocks of the C3 module is replaced with PConv (partial convolution) and PWConv (point convolution) layers. In this way, a lightweight C3-Faster structure is designed for YOLOv5s, as shown in Figure 7 shown.

[0059] The loss function of bounding box regression is crucial to the target detection task. A good loss function will significantly improve the detection accuracy of the model. YOLOv5 performs bounding box regression based on CIoU. The definition of CIoU is shown in (1) to (5). Although the CIoU loss function takes into account the loss of the aspect ratio of the GT box and the predicted box, the center point distance, and the overlapping area, it improves the accuracy of regression positioning.

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] However, there are still limitations in the regression of small target boxes, and the influence of the inherent attributes of the box itself, such as its shape and scale, on the regression of the box is ignored, which sometimes hinders the effective optimization of the model. In addition, the abnormal tobacco has different aspect ratios, the target of the run-out and the stem is small. Therefore, this embodiment introduces Shape-IoU for bounding box regression, so that the box pays more attention to the influence of its own shape factors and scale factors on the detection results during the regression process, improves the detection accuracy, and reduces the regression deviation caused by the model's own shape during the training process.

[0066] The schematic diagram of the predicted box and the true box is as follows Figure 8 As shown, this embodiment uses Shape-IoU as the bounding box regression loss function, which is defined as shown in formula (6).

[0067] L Shape-IoU =1-IOU+distance shape +0.5×Ω shape (6)

[0068] The model structure of CBS-YOLOv5 in this embodiment is as follows Fig. 9 As shown:

[0069] 1.4.2 Model Evaluation Indicators

[0070] This embodiment uses a variety of evaluation indicators, mainly using Mean Average Precision (MAP), precision, recall, network parameters, detection speed (Frames Per Second, FPS), and F1-Score as evaluation indicators for abnormal tobacco identification. Precision indicates the proportion of correctly detected tobacco types to the total number of detected targets, and recall represents the ratio of correctly detected runaways, drawn strips, and stems. The F1 score reflects the balance between recall and precision. The detection speed is mainly measured in frames per second (FPS). The definition formulas of the above indicators are as follows.

[0071]

[0072]

[0073]

[0074]

[0075] Where P i (r) is the detection accuracy of each target; TP refers to the result of correctly identifying the tobacco category; FP refers to the result of incorrectly identifying the tobacco category; FN refers to the result of failing to detect the tobacco category; n is the detection category.

[0076] 1.4.3 Model Training

[0077] The model runs under Windows 10 operating system, with a CPU of i9-12900KF3.2GHz, 64G memory, and a GPU of NVIDIA GeForce RTX 4090 with 24G memory. Pytorch1.7.1 and Cuda11.3 frameworks are used for training and testing. The divided training set is sent to the network for training using the improved YOLOv5 model. During the training process, the model parameters are set as follows: the initial learning rate is 0.01, the Adam adaptive gradient descent algorithm is used, the training rounds (epoch) are 300, the image input size is 640 pixels × 640 pixels, the batch size of the image processing is 16, and the weight file is saved every 100 rounds, and the pre-trained weight YOLOv5s.pt is loaded for training.

[0078] 1.5 Abnormal tobacco quantity detection method

[0079] The number of abnormal tobacco shreds will directly affect the quality of the shredded tobacco in the production line. Due to the wide variety of tobacco shreds and complex background colors, this embodiment uses a deep learning-based method to complete the detection of abnormal tobacco shreds. In order to further enhance the value of the algorithm in practical engineering applications, the improved algorithm adds a target counting function. The basic principle is to calculate the number of each target category by calculating the number of anchor frames, and display the calculated results in the upper left corner of the picture. Finally, the number of abnormal tobacco shreds in the picture or video can be counted. The detection results are shown in Fig.10 .

[0080] 2 Results and analysis

[0081] 2.1 Model Training Results

[0082] 300 rounds of training were carried out in the experimental environment, and the model was tested using the test set. The results showed that the model had an accuracy of 96.7%, a recall rate of 95.6%, and an mAP value of 96.4%. The PR curve and Loss curve of the test results are shown in Figure 2. Fig.11 and Fig.12 shown.

[0083] 2.2 Application Effect

[0084] In order to verify the effectiveness of the algorithm, the data collected by the industrial camera was used to study and analyze the results of the shredder one hour after it was started. The trends of the running, drawing, and stem picking over time after the shredder was started are as follows: Fig.14 As shown. During the startup of the shredder, the number of the three types of abnormal tobacco was the largest, among which the number of strips accounted for a relatively high proportion, reaching 1,194. Secondly, the number of missing pieces was 576, and the number of stems was the lowest, at 192. Although the number of the three types of abnormal tobacco fluctuated slightly over time, they all showed a decreasing trend. From the perspective of practical application, at the moment the shredder is started, due to the inertia of the system and the transmission mechanism, the supply of tobacco may be uneven, the thickness of the material layer may be uneven, and the material layer may be too high. The knife door is unevenly stressed when compacting the material, which will cause an increase in the number of missing pieces and strips. As the material supply stabilizes, the number of abnormal tobacco also shows a significant decrease.

[0085] 3. Results

[0086] In order to further improve the model's ability to detect abnormal tobacco, this paper proposes a CBS-YOLOv5 algorithm for target detection in the tobacco production workshop assembly line based on the improved YOLOv5 algorithm. The algorithm uses the module in Faster Net as a lightweight network to solve the problem of large parameters and high computational complexity brought by the backbone network, and improves the detection speed of the model. The BiFormer attention mechanism is used to enhance the feature extraction ability of small targets and improve the network's understanding of semantic information. In addition, Shape-IoU is used for bounding box regression to improve positioning accuracy and accelerate the convergence speed of the model. The experimental results show that: ① Compared with Faster R-CNN, YOLOv4-tiny, YOLOv5, and YOLOv8 algorithms, the algorithm has significant advantages in speed and accuracy. Compared with the original model, the detection accuracy is increased by 3.09% and the F1 score is increased by 4.7%. ② Compared with the commonly used lightweight feature extraction network, it shows strong superiority, and the detection speed reaches 85.352 frames / s③ Based on the improved FCS-YOLOv5 model, it can accurately detect the number of abnormal tobacco shreds, the fluctuation status of abnormal tobacco shreds within a certain period of time and their positions. It provides technical support and reliable guarantee for application scenarios such as production line cutting quality detection, abnormal tobacco shreds early warning, feedback adjustment of previous processes, etc. The detailed explanation of the above-mentioned embodiments is only intended to explain the present invention so as to better understand the present invention. However, these descriptions cannot be interpreted as limitations on the present invention for any reason. In particular, the various features described in different embodiments can also be arbitrarily combined with each other to form other embodiments. Unless there is a clear description to the contrary, these features should be understood to be applicable to any embodiment, and are not limited to the described embodiments.

Claims

1. A method for detecting abnormal tobacco shreds based on the YOLO model, characterized in that: Collecting abnormal tobacco shreds images and constructing a data set of abnormal tobacco shreds images, wherein the abnormal tobacco shreds images are based on densely stacked normal tobacco shreds as a background; Constructing an abnormal tobacco recognition model based on the YOLO model, wherein the abnormal tobacco recognition model includes at least a backbone network, a neck network, and a head network, and introducing a BiFormer attention mechanism based on a Transformer architecture into the neck network; Training the abnormal tobacco recognition model using the data set; The trained abnormal tobacco cuts identification model is used to identify abnormal tobacco cuts in the tobacco cuts pile.

2. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1, characterized in that: The abnormal shredded tobacco at least includes loose pieces, drawn strips and stems.

3. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1, characterized in that: The data set is expanded by using a data enhancement method, and the abnormal tobacco recognition model is trained using the expanded data set.

4. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1 or 3, characterized in that: The LabelImg tool is used to mark the abnormal tobacco in the abnormal tobacco image with a minimum circumscribed rectangle, and an XML-type label file is corresponding to each abnormal tobacco image.

5. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1, characterized in that: The convolutional layers in the three CBS blocks of the C3 module in the backbone network are replaced with partial convolutional layers and / or PWConv point convolutional layers.

6. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1, characterized in that: The Shape-IoU Loss function is used as the bounding box regression loss function of the abnormal tobacco recognition model.

7. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 1, characterized in that: The data set is divided into a training set, a validation set and a test set, and the training set is transmitted to the abnormal tobacco recognition model for training.

8. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 7, characterized in that: The initial learning rate of the abnormal tobacco recognition model is set to 0.01, and the Adam adaptive gradient descent algorithm is used.

9. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 7, characterized in that: The abnormal tobacco recognition model is tested using the test set and the abnormal tobacco recognition model is evaluated based on the test results.

10. The abnormal tobacco shreds identification and detection method based on the YOLO model according to claim 9, characterized in that: The indicators used to evaluate the abnormal tobacco identification model include at least average detection accuracy, precision, recall rate, network parameter quantity and detection speed.

11. An abnormal tobacco identification and detection device based on the YOLO model, characterized in that: The abnormal tobacco identification and detection device instructions include: An image acquisition module is disposed above the tobacco shreds transfer conveyor belt and is capable of acquiring images of densely stacked tobacco shreds on the tobacco shreds transfer conveyor belt; An image recognition module inputs the image of the shredded tobacco into a pre-trained abnormal shredded tobacco recognition model, and recognizes the abnormal shredded tobacco in the image of the shredded tobacco by using the normal shredded tobacco recognition model; An output module outputs the number of abnormal tobacco shreds in each of the tobacco shred images.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

Images