A method for detecting missing strips in cigarette boxes based on an improved YOLOv5s

Abstract

This invention discloses a method for detecting missing strips in cigarette boxes based on an improved YOLOv5s, belonging to the field of image processing and detection technology. The invention specifically includes the following steps: acquiring a dataset of missing strips in cigarette boxes and preprocessing the data; classifying and labeling the data using data processing algorithms; constructing a multi-frame topology network structure based on an improved YOLOv5s; training and validating the processed data using the improved YOLOv5s multi-frame topology network structure; and evaluating the improved YOLOv5s multi-frame topology network structure. The multi-frame topology network structure proposed in this invention mainly enhances the model's robustness to complex images by introducing a dual-channel concurrent Neck module; and by introducing a multi-level feature fusion module, which receives the processed input feature map and simultaneously sends it to the attention convolutional layer and the ordinary convolutional layer, and couples the outputs of both using dual-channel convolution technology to obtain a feature map with more prominent target features.

Description

Technical Field

[0001] This invention belongs to the field of image processing and detection technology, specifically relating to a method for detecting missing strips in cigarette boxes based on an improved YOLOv5s. Background Technology

[0002] In the tobacco industry, after being packaged in cartons by the packing machine, cigarettes are conveyed to the finished product warehouse via a conveyor line. During the packing process, as well as during the warehousing and outgoing of finished cigarettes, various factors can lead to missing cartons. Once this occurs, it can cause unnecessary disputes between tobacco factories and retailers, severely impacting the tobacco factory's reputation. Missing cartons are considered a Class A serious quality defect for tobacco factories and are given high priority by all tobacco manufacturers.

[0003] Currently, cigarette factories primarily use a combination of machine vision and dynamic weighing for detecting missing cigarette packs. Machine vision detection, located inside the packing machine before sealing, involves photographing the arrangement of the cigarette packs inside the pack with an industrial camera and using image processing algorithms to determine if any packs are missing. However, because image acquisition is in a non-enclosed environment, it is susceptible to external interference such as dust, light sources, and lens vibration, resulting in poor image quality, numerous false positives and false negatives, and the possibility of missing packs during the process of finished cigarette packs entering and leaving the warehouse cannot be completely eliminated. Dynamic weighing detection, located on the outbound line, is affected by the upper and lower limits of the weight deviation during the production of individual cigarettes, leading to a significant cumulative deviation among the 10,000 cigarettes in a pack. Furthermore, the recycling of the cigarette pack shell also affects the weight of the entire pack. These accumulated deviations can cause the normal weight fluctuation of a single pack of cigarettes to exceed that of a whole pack, making false positives and false negatives even more likely. Additionally, dynamic weighing detection only records the weight of the pack, failing to provide a direct view of the cigarette pack arrangement inside the pack for product quality traceability. Finally, due to adjustments made by the tobacco factory itself, the output end of a production line often transports several different cigarette brands through the same production line. In this case, neither machine vision nor dynamic weighing detection methods can accurately adapt the cigarette boxes to the model and assess their weight.

[0004] With the rapid development of science and technology and artificial intelligence, since the rise of the AlexNet deep learning algorithm model in 2012, and the continuous iteration and upgrading of deep learning algorithms, after more than a decade of development, deep learning technology has gradually integrated into people's daily lives. Its superior performance in image processing and detection technology has been increasingly recognized by numerous research institutions and companies. Compared to traditional machine vision, deep learning technology has strong self-learning capabilities, significantly reducing the time required for manual preprocessing of raw data, strong filtering ability against external anomalies, and strong generalization ability. Through convolutional neural networks, target information is extracted layer by layer to achieve automatic identification and localization of target information. These advantages make deep learning technology highly promising in the field of cigarette box defect detection.

[0005] Deep learning-based object detection algorithms are a crucial component of computer vision, enabling accurate object localization and classification from images or videos. Some classic algorithms include two-stage algorithms like Mask R-CNN, Faster R-CNN, Fast R-CNN, and R-CNN, as well as single-stage algorithms like YOLO and SSD. Two-stage algorithms first extract candidate bounding boxes from the image, then refine these boxes to obtain the detection points. This method offers high accuracy, but its efficiency is slower due to the secondary data processing. In contrast, single-stage algorithms directly compute the data from the image, resulting in faster detection speeds, although accuracy is slightly lower due to the elimination of secondary data analysis. In the application of detecting missing cigarette pack strips, we aim to improve the accuracy of the YOLO algorithm while maintaining recognition speed. Summary of the Invention

[0006] To address the aforementioned technical problems in existing technologies, this invention proposes a cigarette box missing strip detection method based on an improved YOLOv5s (YouOnly Look Once, a single-stage target detection algorithm). While maintaining the original detection speed, it effectively improves the recognition accuracy of cigarette box missing strips, thereby reducing false detections and missed detections. The design is reasonable, overcomes the shortcomings of existing technologies, and has good results.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for detecting missing strips in cigarette boxes based on an improved YOLOv5s includes the following steps:

[0009] Step 1: Acquisition and creation of the tobacco box missing strip dataset;

[0010] The acquired images of cigarette boxes with missing strips are preprocessed, and the data is reclassified and labeled using data processing algorithms.

[0011] Step 2: Construct a multi-frame topology network structure based on the improved YOLOv5s;

[0012] Step 3: Train and validate the processed data using the improved YOLOv5s multi-frame topology network structure;

[0013] Step 4: Evaluate the improved YOLOv5s multi-frame topology network structure.

[0014] Preferably, in step 1, the specific operation method for preprocessing the acquired cigarette box missing strip image data is as follows: using data augmentation technology, the original data is expanded by performing transformations including flipping, translation, and light intensity changes, thereby enhancing the diversity of the data and preventing the model from overfitting to the data during training.

[0015] Preferably, in step 1, the data is reclassified and labeled using a data processing algorithm. The specific operation method is as follows: the original dataset is divided into a training set, a validation set, and a test set using a data processing algorithm, with a ratio of 8:1:1.

[0016] Preferably, step 2 specifically includes the following steps:

[0017] Step 2.1: In the YOLOv5s network structure, the robustness of the model to complex images is enhanced by introducing a dual-channel concurrent Neck module: by freezing different convolution depths, the shallow layer of the network extracts the contour information of the target, and the deep layer extracts the detailed information of the target. The outputs of the two are coupled by dual-channel convolution technology to obtain a feature map with more prominent target features.

[0018] Step 2.2: Introduce a multi-level feature fusion module in Neck to enhance the semantic and detail information in the image by integrating the hierarchical feature maps generated by the encoder, so that the features can integrate local spatial information and global channel information.

[0019] Preferably, the dual-channel concurrent Neck module in step 2.1 refers to adding a Neck module on the basis of the original Neck module. By concurrently passing the input feature map to two Neck network modules with different frozen convolution depths, the outputs of the two are coupled and calculated using dual-channel convolution technology to obtain a feature map with more prominent target features.

[0020] Preferably, step 2.2 specifically includes the following steps:

[0021] Step 2.2.1: The encoder generates multi-level features for the input image. By integrating the hierarchical feature maps generated by the encoder, the semantic and detail information in the image is enhanced, so that the features can integrate local spatial information and global channel information.

[0022] Step 2.2.2: For the feature map of each level, the multi-level feature fusion module fuses high-level features containing more semantic information and low-level features capturing finer details through the Hadamard product operation, thereby enhancing the semantics and details of each level of features.

[0023] Step 2.2.3: Finally, the refined features are passed to the decoder for parsing, reconstruction, and image segmentation.

[0024] Preferably, in step 4, the metrics used to evaluate the model include precision P, recall R, mean precision AP, and mean precision mAP, and the functional expressions for each metric are as follows:

[0025]

[0026]

[0027]

[0028]

[0029] Where TP represents the number of true values ​​that match the actual results; FP represents the number of true values ​​that differ from the actual results; and FN represents the number of false values ​​that differ from the actual results. In the AP expression, R(n) = 0, P(n) = 1, and n represents the threshold number. In the mAP expression, AP... k This represents the average accuracy for class k; n represents the number of classes.

[0030] The beneficial technical effects of this invention are as follows:

[0031] This invention proposes a cigarette box defect detection method based on an improved YOLOv5s. It adds a new Neck module to the existing Neck module, concurrently passing the input feature map to two Neck network modules with different convolutional depths. The outputs of these two modules are coupled using dual-channel convolution to obtain a feature map that more prominently displays the target features. Simultaneously, a multi-level feature fusion module (SDI) is introduced into the Neck. First, the encoder generates multi-level features for the input image. By integrating the layered feature maps generated by the encoder, the semantic and detail information in the image is enhanced, allowing the features to integrate local spatial and global channel information. Second, for each level of feature map, the SDI module fuses high-level features containing more semantic information with low-level features capturing finer details using a Hadamard product operation, thereby enhancing the semantics and details of each level of feature. Finally, the refined features are then passed to the decoder for parsing, reconstruction, and image segmentation, improving the model's detection accuracy. Attached Figure Description

[0032] Figure 1 This is the overall operation flowchart of the present invention;

[0033] Figure 2 This is the Neck module structure diagram from the original YOLOv5s;

[0034] Figure 3 This is a structural diagram of the dual-channel concurrent Neck module in the improved YOLOv5s of this invention;

[0035] Figure 4 This is a diagram of the SDI multi-level feature fusion module in this invention;

[0036] Figure 5 This is a graph showing the relationship between the confidence level and F1 score of the improved training experiment results in this invention.

[0037] Figure 6 This is a graph showing the relationship between the accuracy and confidence level of the improved training experiment results in this invention.

[0038] Figure 7 This is a graph showing the relationship between recall and confidence in the improved training experiment results of this invention.

[0039] Figure 8 This is a graph showing the precision and recall of the improved experimental results under different classification thresholds. Detailed Implementation

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0041] A method for detecting missing strips in cigarette boxes based on an improved YOLOv5s includes the following steps:

[0042] Step 1: Acquisition and creation of the tobacco box missing strip dataset.

[0043] Step 1.1: Save the obtained tobacco box missing strip data to the specified JPEGImages folder, use LabelImg annotation software to annotate, classify the missing strip types into vertical tobacco box missing strips (SN) and horizontal tobacco box missing strips (HN), and store the generated XML format file in the Annotations folder.

[0044] Step 1.2: Use format conversion and data classification software algorithms to convert the original XML data file into a YOLOv5 training TXT file, and randomly divide the original dataset into training, validation, and test sets in an 8:1:1 ratio. The training set is used to train the improved model, the validation set is used to verify the stability and accuracy of the trained model, and the final test set is used to test the model's adaptability and generality.

[0045] Step 2: Construct a multi-frame topology network structure based on the improved YOLOv5s.

[0046] Step 2.1: In the YOLOv5s network structure, the robustness of the model to complex images is enhanced by introducing a dual-channel concurrent Neck module, such as... Figure 3 As shown, by freezing different convolution depths, the shallow layers of the network extract the contour information of the target, while the deep layers extract detailed target information. The outputs of both are coupled using a dual-channel convolution technique to obtain a feature map that highlights the target features more prominently.

[0047] Step 2.2: Introduce a multi-level feature fusion module (SDI) into Neck, such as... Figure 4 As shown. First, the encoder generates multi-level features for the input image. By integrating the multi-level feature maps generated by the encoder, the semantic and detail information in the image is enhanced, enabling the features to integrate local spatial information and global channel information. The formula is as follows:

[0048]

[0049] In the above formula, f i 1 This represents the feature map after the i-th level of processing. and Let f represent the spatial and channel attention mechanisms at level i, respectively. Furthermore, 1×1 convolutions are used to reduce f. i 1 The channel to c, where c is a hyperparameter. The feature map generated by convolution is denoted as f. i2 The formula is as follows:

[0050]

[0051] Where H i H i c and f represent f respectively i 2 Width, height, and number of channels.

[0052] Secondly, for each level of feature map, the SDI module fuses high-level features containing more semantic information and low-level features capturing finer details, sends the feature map that needs optimization to the decoder, and uses f on the i-th level of each decoder. i 2 As a reference target, the Hadamard product operation is used to enhance the semantics and details of the features at each level. Then, the size of the feature map at each level j is adjusted to match f. i 2 For the same resolution, the formula is as follows:

[0053]

[0054] in and These represent adaptive average pooling, identity mapping, and bidirectional interpolation f, respectively. j 2 To H i ×W i The resolution of , where i≥1, M≥j.

[0055] Then, a 3×3 convolutional layer is used to smooth each feature map that needs to be resized by sliding, as shown in the following formula:

[0056]

[0057] Where θ ij This represents the parameters of the smooth convolution. This represents the j-th smooth feature map of level i.

[0058] Finally, the refined features are passed to the decoder for parsing, reconstruction, and image segmentation.

[0059] Step 3: Train and validate the processed data using the improved YOLOv5s multi-frame topology network structure.

[0060] Step 3.1: The experimental hardware configuration uses an NVIDIA GeForce RTX A4000 4G dedicated GPU, an Intel Core i9-9900T CPU@2.10GHz processor, 16GB of memory, Windows 11 Professional (64-bit) operating system, Visual Studio Code development platform, Python programming language, PyTorch deep learning framework, YOLOv5s deep learning model, and torch2.0.0+cu118 GPU acceleration library.

[0061] Step 3.2: Set training parameters: batch-size is set to 32, imgsz is set to 256×256, initial learning rate lr0 is set to 0.01, stochastic gradient descent (SGD) is used as the optimizer, final learning rate lrf is set to 0.1, SGD momentum parameter is set to 0.937, and optimizer weight decay coefficient is set to 0.0005.

[0062] Step 4: Evaluate the improved YOLOv5s multi-frame topology network structure.

[0063] Step 4.1: The experimental results of this invention are as follows Figures 5-8 As shown.

[0064] This invention primarily evaluates the model using metrics such as precision (P), recall (R), average precision (AP), and mean average precision (mAP). The functional expressions for each metric are as follows:

[0065]

[0066]

[0067]

[0068]

[0069] In the above formula, TP represents the number of predictions that are the same as the actual results and are predicted as true. FP represents the number of predictions that are different from the actual results and are predicted as true. FN represents the number of predictions that are different from the actual results and are predicted as false. In the AP expression, R(n) = 0, P(n) = 1, where n represents the threshold number. In the mAP expression, AP k Let be the average accuracy for k classes, where n represents the number of classes.

[0070] Step 4.2: This invention includes four sets of experiments for comparison and illustration. The first set tests the original YOLOv5s deep learning model; the second set tests the YOLOv5s model with the T-Neck module added; the third set tests the model with SDI introduced into the Neck module; and the last set tests the improved YOLOv5s model. The test results are shown in Table 1.

[0071] YOLOv5s 0.853 0.843 0.869 0.788 YOLOv5s+T-Neck 0.877 0.861 0.874 0.804 YOLOv5s+SDI 0.873 0.858 0.883 0.807 our 0.896 0.887 0.901 0.819

[0072] The data in the table above shows that the last group, the improved YOLOv5s network structure, improves accuracy by 4.3% compared to the original network model. Similarly, it can be seen that both improving the T-Neck module and introducing the SDI module can optimize the original YOLOv5s model to a certain extent. Therefore, we can conclude that the cigarette box missing strip detection method designed in this invention, based on improved YOLOv5s, significantly improves the accuracy of cigarette box missing strip detection and recognition compared to the original model. Furthermore, the generalization ability and robustness of the entire network are enhanced, and it has a good effect in model adaptation. While ensuring industrial detection speed, it effectively reduces the occurrence of false positives and false negatives in the cigarette box detection process.

[0073] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for detecting missing strips in cigarette boxes based on an improved YOLOv5s, characterized in that: Includes the following steps: Step 1: Acquisition and creation of the tobacco box missing strip dataset; The acquired images of cigarette boxes with missing strips are preprocessed, and the data is reclassified and labeled using data processing algorithms. Step 2: Construct a multi-frame topology network structure based on the improved YOLOv5s; Step 3: Train and validate the processed data using the improved YOLOv5s multi-frame topology network structure; Step 4: Evaluate the improved YOLOv5s multi-frame topology network structure; Step 2 specifically includes the following steps: Step 2.1: In the YOLOv5s network structure, the robustness of the model to complex images is enhanced by introducing a dual-channel concurrent Neck module: by freezing different convolution depths, the shallow layer of the network extracts the contour information of the target, and the deep layer extracts the detailed information of the target. The outputs of the two are coupled by dual-channel convolution technology to obtain a feature map with more prominent target features. Step 2.2: Introduce a multi-level feature fusion module in Neck to enhance the semantic and detail information in the image by integrating the hierarchical feature maps generated by the encoder, so that the features can integrate local spatial information and global channel information; The dual-channel concurrent Neck module in step 2.1 refers to adding a Neck module on the basis of the original Neck module. By concurrently passing the input feature map to two Neck network modules with different frozen convolution depths, the outputs of the two are coupled and calculated using dual-channel convolution technology to obtain a feature map with more prominent target features. Step 2.2 specifically includes the following steps: Step 2.2.1: The encoder generates multi-level features for the input image. By integrating the hierarchical feature maps generated by the encoder, the semantic and detail information in the image is enhanced, so that the features can integrate local spatial information and global channel information. Step 2.2.2: For the feature map of each level, the multi-level feature fusion module fuses high-level features containing more semantic information and low-level features capturing finer details through the Hadamard product operation, thereby enhancing the semantics and details of each level of features. Step 2.2.3: Finally, the refined features are passed to the decoder for parsing, reconstruction, and image segmentation.

2. The method for detecting missing strips in cigarette boxes based on the improved YOLOv5s according to claim 1, characterized in that: In step 1, the specific operation method for preprocessing the acquired cigarette box missing strip image data is as follows: using data augmentation technology, the original data is expanded by performing transformations including flipping, translation, and light intensity, thereby enhancing the diversity of the data and preventing the model from overfitting to the data training.

3. The method for detecting missing strips in cigarette boxes based on the improved YOLOv5s according to claim 1, characterized in that: In step 1, the data is reclassified and labeled using a data processing algorithm. The specific operation method is as follows: the original dataset is divided into a training set, a validation set, and a test set using a data processing algorithm, with a ratio of 8:1:

1.

4. The method for detecting missing strips in cigarette boxes based on the improved YOLOv5s according to claim 1, characterized in that: In step 4, the metrics used to evaluate the model include precision (P), recall (R), average precision (AP), and mean average precision (mAP). The functional expressions for each metric are as follows: Where TP represents the number of true predictions that match the actual results; FP represents the number of true predictions that differ from the actual results; and FN represents the number of false predictions that differ from the actual results. In the AP expression, , This represents the threshold number; in the mAP expression, express Average accuracy of the class; Indicates the number of categories.

Images