Blasting tobacco stem defect detection method based on improved YOLOv5s

Abstract

The invention relates to a blasting tobacco stem defect detection method based on improved YOLOv5s, and belongs to the technical field of target detection and defect identification. The method comprises the steps of firstly, collecting an image data set, performing preprocessing and crack labeling, and dividing the image data set into a training set and a test set in proportion; secondly, improving the original YOLOv5s model by using a plurality of improvement strategies; then, training the improved YOLOv5s model by using the training set; and finally, evaluating the trained improved YOLOv5s model by using the test set to realize detection and output a detection result. An ablation experiment and a contrast experiment are carried out by using a tobacco stem data set, and compared with an original YOLOv5s model, the experimental result realizes double improvement of the detection precision and the detection speed.

Description

Technical Field

[0001] The invention relates to a method for detecting defects in exploded tobacco stems based on improved YOLOv5s, and belongs to the technical field of target detection and defect recognition. Background Art

[0002] In recent years, under the background of cost control and lean manufacturing of cigarette enterprises, tobacco stems have been fully utilized as a means of regulating the cost of cigarette formulas due to the advantages of large tobacco stem inventory and low tobacco stem cost. On the one hand, tobacco stems, as one of the main raw materials of cigarettes, can be processed into stem shreds or stem particles with high filling value and good combustibility, which plays an important role in reducing harm and tar, reducing formula costs, and improving the physical quality of cigarettes; on the other hand, the cigarette manufacturing process has formed an important breakthrough in improving the processing quality of tobacco stems and increasing the proportion of filamentous stem shreds, which fully improves the use value of tobacco stems and broadens the raw material range of tobacco stems, and effectively regulates the cost of cigarette formulas. The degree of damage to tobacco stems after blasting is an important indicator for measuring the quality and processing level of cigarette products, and has a significant impact on the structure of stem shreds after cutting, the silk yield, the proportion of stems, and the physical quality of cigarettes. Therefore, the level of the damage ratio of blasted tobacco stems directly affects the quality of subsequent tobacco stem processing, and the quality inspection of tobacco stems is very important for the entire tobacco stem application industry.

[0003] At present, the tobacco industry has not yet established a quantitative detection method for stem compression defects or stem deformation, and the actual effect of steam explosion technology in treating tobacco stems is still unclear. Under different explosion pressures and different explosion times, the morphology of tobacco stems is significantly different, and the defects in tobacco stems after explosion are mainly measured by manual visual inspection. This method is restricted by subjective factors, work experience, labor intensity, etc., and is highly subjective and slow to detect, resulting in low efficiency, making it difficult to achieve the goal of efficient and rapid production. Therefore, it is of great significance to establish a method that can quickly and accurately detect and identify the defects of exploded tobacco stems and realize online detection of tobacco stem quality, which is of great significance to improving the level of silk processing technology. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for detecting defects in exploded tobacco stems based on improved YOLOv5s. When the shapes of the defects in exploded tobacco stems are irregular and there are many small targets, the original target detection method cannot take into account both the detection accuracy and the detection speed at the same time.

[0005] The technical solution of the present invention is: a method for detecting defects of exploded tobacco stems based on improved YOLOv5s, and the specific steps are:

[0006] Step 1: Collect image data sets, perform preprocessing and defect annotation, and divide them into training sets and test sets in proportion;

[0007] Step 2: Use a variety of improvement strategies to improve the original YOLOv5s model;

[0008] Step 3: Use the training set to train the improved YOLOv5s model;

[0009] Step 4: Use the test set to evaluate the trained improved YOLOv5s model, implement detection and output the detection results.

[0010] Specifically, the Step 1 is as follows:

[0011] The image data set is constructed from tobacco stem sample images collected on a conveyor belt;

[0012] The preprocessing includes adjusting image brightness, adding Gaussian noise and horizontal flipping;

[0013] The defect marking includes cracks, fractures and chipping;

[0014] The training set is used for model training;

[0015] The test set is used for model evaluation.

[0016] Specifically, the Step 2 is as follows:

[0017] Improvement strategy 1: Fuse the Swin Transformer module and the C3 module to replace the last layer of the C3 module in the YOLOv5s backbone extraction network, expand the receptive field area by continuously increasing the number of parameters, and enhance the network's feature extraction capability;

[0018] Improvement strategy 2: Introduce the CBAM mechanism to increase the YOLOv5s network's attention to important features, improve feature extraction capabilities and detection accuracy;

[0019] Improvement strategy three: Replace the detection head in the original YOLOv5s model with a decoupling head to separate the classification task and the regression task to avoid conflicts between the two tasks;

[0020] Improvement strategy 4: Use the NWD loss function instead of the original loss function in the original YOLOv5s model to speed up convergence and improve detection accuracy.

[0021] Specifically, the Step 3 is as follows:

[0022] Step 3.1: Input the training set into the improved YOLOv5s model;

[0023] Step 3.2: Extract image features through convolutional neural network;

[0024] Step 3.3: Use the decoupling head of the improved YOLOv5s model to perform target detection on the extracted features;

[0025] Step 3.4: Calculate the loss value between the detection result and the annotation;

[0026] Step 3.5: According to the loss value, the YOLOv5s model parameters are updated and improved through the back propagation algorithm;

[0027] Step 3.6: Repeat the above steps. If the improved YOLOv5s model converges or the number of repeated rounds reaches the preset number of training rounds, the improved YOLOv5s model training is completed.

[0028] Specifically, the Step 4 is as follows:

[0029] Step 4.1: Input the test set into the trained improved YOLOv5s model for target detection;

[0030] Step 4.2: Calculate the evaluation indicators based on the detection results and annotations, including precision, recall, average precision and detection speed;

[0031] Step 4.3: Mark the test results and display them visually

[0032] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention is based on the original YOLOv5s model, by introducing a sliding window Transformer module (Swin Transformer), a convolutional block attention mechanism (Convolutional Block Attention Module, CBAM), a decoupled head design (Decoupled Head, DH) and a normalized Wasserstein distance loss function (Normalized Wasserstein Distance, NWD), to obtain a YOLOv5s-SNCD model, that is, a model obtained by combining the above four improvement strategies, which improves the detection accuracy and speed of the model. The present invention can not only run efficiently on low-computing power devices and realize real-time detection, but also effectively reduce manual intervention and improve detection efficiency, thereby providing an efficient detection method for quality control in the tobacco industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the steps of the present invention;

[0034] Figure 2 It is the improved YOLOv5s structure diagram of the present invention;

[0035] Figure 3is an image example in the image dataset preprocessed by the present invention;

[0036] Figure 4 This is the Swin Transformer backbone network structure diagram;

[0037] Figure 5 It is the structure diagram of the channel attention module (CAM);

[0038] Figure 6 It is the structure diagram of the spatial attention module (SAM);

[0039] Figure 7 This is an example of a visualized heat map of the gradient-weighted class activation mapping (GradCAM) of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.

[0041] Example 1: Figure 1 As shown in the figure, a method for detecting defects of exploded tobacco stems based on improved YOLOv5s is shown, and the specific steps are as follows:

[0042] Step 1: Collect image data sets, perform preprocessing and defect annotation, and divide them into training sets and test sets in proportion;

[0043] Step 2: Use a variety of improvement strategies to improve the original YOLOv5s model;

[0044] Step 3: Use the training set to train the improved YOLOv5s model;

[0045] Step 4: Use the test set to evaluate the trained improved YOLOv5s model, implement detection and output the detection results.

[0046] Specifically, the Step 1 is as follows:

[0047] The image data set is constructed from tobacco stem sample images collected on a conveyor belt;

[0048] The preprocessing includes adjusting image brightness, adding Gaussian noise and horizontal flipping;

[0049] The defect marking includes cracks, fractures and chipping;

[0050] The training set is used for model training;

[0051] The test set is used for model evaluation.

[0052] Furthermore, after manually removing images with poor pixel quality and blur, the image dataset is preprocessed to avoid the impact of too few samples in the image dataset on the model training effect and generalization performance, and to reduce the impact of the original image on the experiment. The preprocessing includes randomly adjusting the image brightness, adding Gaussian noise and horizontal flipping. Finally, the preprocessed image dataset is divided into training set and test set in a ratio of 8:2.

[0053] The specific steps of the pretreatment are:

[0054] The random adjustment of image brightness is to simulate images under different lighting conditions by changing the brightness value of each pixel in the image. The image is brighter or darker by operating on the brightness channel of the image, increasing or decreasing its value. The adjustment amplitude is usually a random value to ensure that the model can adapt to different lighting scenarios.

[0055] Specifically, in an RGB image, the brightness can be changed by adjusting the RGB value of each pixel in the image. The formula is:

[0056] Vn=V o ×F b

[0057] Where Vn is the brightness of the adjusted image, V o is the brightness of the image before adjustment, F b It is a randomly selected coefficient used to control the increase or decrease of brightness; in the method of the present invention, it is set to 1.2 to simulate a slightly brighter environment.

[0058] Adding Gaussian noise is to randomly add Gaussian noise to the image. It is a commonly used noise type that can simulate the noise generated by image sensors in real environments. Gaussian noise conforms to the normal distribution, and each pixel value of the noise is randomly sampled from a normal distribution with a mean of 0 and a standard deviation of σ. The formula is:

[0059] I noisy (x,y)=I(x,y)+N(0,σ 2 )

[0060] Among them, I noisy (x, y) is the image pixel after adding Gaussian noise, I(x, y) is the original image pixel, N(0,σ 2 ) has a mean of 0 and a variance of σ 2 Gaussian noise.

[0061] Specifically, Gaussian noise is directly added to each pixel of the image. The specific implementation is to add a noise value randomly sampled from a normal distribution to the color value of each pixel in the image. The intensity of the noise is controlled by the standard deviation σ, and a suitable value is usually selected according to the specific scene. A smaller σ produces slighter noise, while a larger σ produces obvious noise.

[0062] The horizontal flipping is to flip the image symmetrically along the vertical axis, and generate a new image by flipping each row of pixels of the image left and right along the vertical axis.

[0063] Specifically, the horizontal flipping operation can be completed by directly changing the position of the image pixels, the width of the image is w, and the pixel position after the pixel I (x, y) is flipped is I (wx, y). In the method of the present invention, the flipping is performed with a probability of 50% to keep the balance of the data set.

[0064] The preprocessed images are as follows Figure 3 As shown. Under the guidance of professionals, defects are annotated on the pictures, including cracks, fractures, and chipping. When training the model, the picture size is uniformly scaled to a pixel size of 640px×640px.

[0065] The improved YOLOv5s structure is as follows Figure 2 As shown, the Step 2 is specifically as follows:

[0066] Improvement strategy 1: Merge the Swin Transformer module and the C3 module to replace the last layer of the C3 module in the backbone extraction network, expand the receptive field area by continuously increasing the number of parameters, and enhance the feature extraction capability of the network;

[0067] Specifically, the present invention integrates the C3 module and the Swin Transformer module to replace the last layer C3 module of the backbone extraction network. The structure of the Swin Transformer module is as follows: Figure 4As shown in the figure, it includes a normalization layer (LayerNorm, LN), a window multi-head self-attention module (Windows Multi-head Self-attention, W-MSA), a shifted window self-attention module (Shifted Windows Multi-Head Self-attention, SW-MSA) and a multilayer perceptron (MultilayerPerceptron, MLP). LN is applied before each MSA module and MLP, and a residual connection is used after each MSA module and MLP. LN normalizes the feature map, W-MSA divides the window and performs multi-head self-attention calculation, SW-MSA performs segmentation and shift operations to achieve information interaction between different windows, and MLP is used to divide the category information of the feature map.

[0068] Improvement strategy 2: Introduce the CBAM mechanism to increase the network's attention to important features, improve feature extraction capabilities and detection accuracy; CBAMAttention belongs to a mixed domain attention mechanism, which is an attention mechanism used to extract task-related areas. It contains two independent sub-modules: channel attention module (CAM) and spatial attention module (SAM). In the channel attention module CAM, the feature image is first subjected to a global average pooling operation based on the spatial size width W and spatial size height H to obtain two 1×1×C feature images. Next, the two feature images are sent to two layers of neural networks respectively, and then the output results are added, and then the channel attention weight coefficient Mc is obtained by the Sigmoid activation function. Finally, Mc is multiplied by the input feature image to obtain a new channel enhanced feature. The calculation formula is:

[0069] M c (F) = σ(MLP(A F )+MLP(M F ))

[0070] A F =AvgPood(F)

[0071] M F =MaxPood(F)

[0072] Where: MLP represents the neural network; σ is the Sigmoid activation function; A F is the average pooling feature; M F is the maximum pooling feature; F is the feature map; AvgPood is the average pooling operation; MaxPood is the maximum pooling operation. The CAM structure is as follows Figure 5 shown.

[0073] In the spatial attention module SAM, first, the calculated new channel-enhanced features are used as input, and then channel-based maximum pooling and average pooling operations are performed on them to generate two H×W×1 feature images. Next, the two feature images are concatenated in the channel dimension, and then a convolution layer is used to reduce the dimension and reduce the number of channels to 1. Finally, the spatial attention weight coefficient M is obtained through the Sigmoid function. S , multiply it with the input channel enhancement feature to obtain the final spatial enhancement feature M S (F), calculated as follows:

[0074] M S (F) = σ(f 7x7 ([A F ;M F ]))

[0075] Improvement strategy three: Replace the detection head in the original YOLOv5s model with a decoupling head to separate the classification task and the regression task to avoid conflicts between the two tasks;

[0076] First, a 1×1 convolution is used to reduce the feature channels to 256. Then, two 3×3 convolutions are used for classification and regression tasks respectively. The classification task uses a 1×1 convolution for classification operations, and the regression task uses a 1×1 convolution for positioning and confidence operations. Through the above operations, the classification, positioning, and confidence are detected separately.

[0077] Improvement strategy 4: Use NWD loss function instead of the loss function in the original YOLOv5s model to speed up convergence and improve detection accuracy.

[0078] Specifically, the calculation process of the NWD loss function is as follows:

[0079] First, define the distribution of predicted boxes and true boxes:

[0080] Define a prediction box (x p ,y p ,w p ,h p ) and a ground-truth box (x g ,y g ,w g ,h g ), where x and y represent the center coordinates of the box, w and h represent the width and height of the box respectively, and the predicted box and the true box are represented as two-dimensional Gaussian distributions, respectively:

[0081]

[0082] in, It is the scale parameter of the predicted box and the real box, which can be calculated by the width and height of the box.

[0083] Next, calculate the Wasserstein distance:

[0084] The calculation formula of Wasserstein distance is:

[0085]

[0086] Where: μ p =(x p ,y p ) T and μ g =(x g ,y g ) T Represents the mean of the predicted box and the true box, Σ p and Σ g are the covariance matrices of the predicted box and the true box, respectively, usually assumed to be diagonal matrices, ∥·∥ 2 represents the Euclidean distance, ∥·∥ F represents the Frobenius norm. Here, the covariance matrix is ​​a diagonal matrix, so the Frobenius norm can be simplified to:

[0087]

[0088] Then, normalization is performed:

[0089] In order to make the loss value consistent at different scales, the Wasserstein distance is normalized. The average of the width and height can be used as the normalization factor:

[0090]

[0091] Finally, calculate the final NWD loss:

[0092] The NWD loss value is the normalized Wasserstein distance:

[0093]

[0094] The Step 3 is specifically as follows:

[0095] Step 3.1: Input the training set into the improved YOLOv5s model;

[0096] Step 3.2: Extract image features through convolutional neural network;

[0097] Step 3.3: Use the decoupling head of the improved YOLOv5s model to perform target detection on the extracted features;

[0098] Step 3.4: Calculate the loss value between the detection result and the annotation;

[0099] Step 3.5: According to the loss value, the YOLOv5s model parameters are updated and improved through the back propagation algorithm;

[0100] Step 3.6: Repeat the above steps. If the improved YOLOv5s model converges or reaches the preset number of training rounds, the training of the improved YOLOv5s model is completed.

[0101] The Step 4 is specifically as follows:

[0102] Step 4.1: Input the test set into the trained improved YOLOv5s model for target detection;

[0103] Step 4.2: Calculate the evaluation indicators based on the detection results and annotations, including precision, recall, average precision and detection speed;

[0104] Specifically, the present invention uses precision (P), recall (R), average precision (AP) and detection speed (FPS), a total of four evaluation indicators for comparison. The calculation formula is as follows:

[0105]

[0106]

[0107] Among them, TP, FP and FN respectively represent the number of correctly detected targets, the number of incorrectly detected targets and the number of undetected targets; P(R) represents the precision under the current recall rate Recall; AP is the average detection accuracy of the target. The AP value is used to measure the precision P and recall R of the algorithm when detecting targets. The size of the AP value is proportional to the detection effect. AP represents the overall detection performance of the algorithm. The larger the AP, the better the overall detection performance of the model; FrameNum represents the total number of detected images, and ElaspedTime represents the total time spent on detection.

[0108] Step 4.3: Mark the test results and display them visually.

[0109] Specifically, the present invention uses GradCAM visualization heat map to visualize the output layer.

[0110] The effectiveness of the technical solution of the present invention is illustrated by way of experiments below.

[0111] The processor model used in this experiment is Intel(R) Corei5-12400F 2.50GHz CPU, the graphics card model is Nvidia GeForce RTX 3060, the video memory is 12GB, the system environment is Windows 10, and the programming language is Python3.11. The deep learning framework is Pytorch 1.11.0 and the CUDA 11.7 parallel computing framework is used. In this experiment, the number of samples (batch size) used in each iterative training is set to 16, and the number of training epochs is set to 200 to avoid overfitting and reduce training time. Subsequent comparative experiments remain unchanged. The hyperparameters during training are shown in Table 1.

[0112] Table 1 Experimental parameter configuration

[0113]

[0114] In order to more accurately evaluate the impact of different improved modules on the accuracy of tobacco stem surface defect detection, an ablation experiment was conducted. First, the original YOLOv5s model was fused with the Swin Transformer to form the STR model; then, the loss function was replaced with NWD, and the CBAM mechanism was introduced; finally, the decoupling head was replaced. During the training process, the same parameter settings were used, and five groups of comparative experiments were designed. The experimental results are shown in Table 2.

[0115] Table 2 Ablation experiment results

[0116]

[0117]

[0118] The results in Table 2 show that the precision of the original YOLOv5s algorithm reached 80.7% and the recall rate reached 72.9%. After adding the STR module, the feature extraction ability of the model was improved. Although the precision rate decreased slightly, the recall rate increased by 0.7%, which was still better than the original YOLOv5s algorithm. After replacing the loss function with NWD, the AP value was further improved by 1.2%. Subsequently, the CBAM mechanism was introduced to enhance the network's perception of the target position and channel information and the focus on important features, further improving the detection speed and AP value. Finally, by replacing the decoupling head, the conflict between the classification and regression tasks was weakened. The recall rate and AP value of the YOLOv5s-SNCD algorithm after the decoupling head were 78.3% and 85.2% respectively, and the FPS reached 38.9 frames / s, which was the best effect. In summary, the YOLOv5s-SNCD algorithm after the above improvements effectively improved the detection accuracy, so that the detection accuracy meets the requirements of the burst tobacco stem defect detection scenario and has good applicability in this scenario.

[0119] In order to more comprehensively verify the effectiveness of the YOLOv5s-SNCD model proposed in the present invention, the present invention conducted comparative experiments with other models using the same training configuration. In these experiments, the YOLOv5s-SNCD model proposed in the present invention was compared in detail with other models. The specific comparison results are shown in Table 3.

[0120] Table 3 Comparative experimental results

[0121]

[0122] As can be seen from Table 3, when the current popular algorithm model is used for training and prediction of the data set, compared with the mainstream Faster-RCNN (Fast Regional Convolutional Neural Network), RetinaNet (Retinal Network), SSD (Single-Stage Multi-Frame Detector) and YOLOv8 models, the AP and FPS of the improved YOLOv5s-SNCD are generally higher than those of the other popular algorithms, once again proving the effectiveness of the YOLOv5s-SNCD algorithm for burst tobacco stem defect detection.

[0123] Finally, the present invention uses GradCAM to visualize the output layer using heatmaps, such as Figure 7 As shown in the figure, the heat map uses the brightness of a certain area to indicate the weight of the area in the prediction output process. The larger the area of ​​the brightly colored area, the higher the attention of the prediction output. It can be seen from the heat map that the improved model is more sensitive to tobacco stem defects.

[0124] Figure 7 The red part in the figure indicates the area that the model pays more attention to. YOLOv5s-SNCD can better concentrate computing resources on the target area and suppress computing power occupied by non-target areas, making it pay more attention to the area where the defects are located, which intuitively verifies the effectiveness of the improved model. Figure 7 From the observation, it can be clearly seen that the red part forms an obvious high-attention area around the defect.

[0125] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for detecting tobacco stem defects by using improved YOLOv5s, characterized in that: Step 1: Collect image data sets, perform preprocessing and defect annotation, and divide them into training sets and test sets in proportion; Step 2: Use a variety of improvement strategies to improve the original YOLOv5s model; Step 3: Use the training set to train the improved YOLOv5s model; Step 4: Use the test set to evaluate the trained improved YOLOv5s model, implement detection and output the detection results.

2. The method for detecting tobacco stem defects by using an improved YOLOv5s according to claim 1, characterized in that: The Step 1 is specifically as follows: The image data set is constructed from tobacco stem sample images collected on a conveyor belt; The preprocessing includes adjusting image brightness, adding Gaussian noise and horizontal flipping; The defect marking includes cracks, fractures and chipping; The training set is used for model training; The test set is used for model evaluation.

3. The method for detecting tobacco stem defects by using an improved YOLOv5s according to claim 1, characterized in that: The Step 2 is specifically as follows: Improvement strategy 1: Fuse the Swin Transformer module and the C3 module to replace the last layer of the C3 module in the YOLOv5s backbone extraction network, expand the receptive field area by continuously increasing the number of parameters, and enhance the feature extraction capability of the network; Improvement strategy 2: Introduce the CBAM mechanism to increase the YOLOv5s network's attention to important features, improve feature extraction capabilities and detection accuracy; Improvement strategy three: Replace the detection head in the original YOLOv5s model with a decoupling head to separate the classification task and the regression task to avoid conflicts between the two tasks; Improvement strategy 4: Use the NWD loss function instead of the original loss function in the original YOLOv5s model to speed up convergence and improve detection accuracy.

4. The method for detecting tobacco stem defects by using an improved YOLOv5s according to claim 1, characterized in that: The Step 3 is specifically as follows: Step 3.1: Input the training set into the improved YOLOv5s model; Step 3.2: Extract image features through convolutional neural network; Step 3.3: Use the decoupling head of the improved YOLOv5s model to perform target detection on the extracted features; Step 3.4: Calculate the loss value between the detection result and the annotation; Step 3.5: According to the loss value, the YOLOv5s model parameters are updated and improved through the back propagation algorithm; Step 3.6: Repeat the above steps. If the improved YOLOv5s model converges or the number of repeated rounds reaches the preset number of training rounds, the improved YOLOv5s model training is completed.

5. The method for detecting tobacco stem defects by using an improved YOLOv5s according to claim 1, characterized in that: The Step 4 is specifically as follows: Step 4.1: Input the test set into the trained improved YOLOv5s model for target detection; Step 4.2: Calculate the evaluation indicators based on the detection results and annotations, including precision, recall, average precision and detection speed; Step 4.3: Mark the test results and display them visually.

Images