Cigarette pack ash crack detection method and detection system based on deep learning semantic segmentation

Abstract

The invention provides a deep learning semantic segmentation-based cigarette ash wrapping gap detection method and detection system, and belongs to the technical field of cigarette detection. The cigarette ash wrapping gap detection method comprises the following steps: acquiring an ash wrapping gap image in a cigarette combustion process, and preprocessing the ash wrapping gap image; using a convolutional neural network to perform multi-scale feature extraction on the preprocessed gray-wrapped crack image to obtain a multi-scale feature map; and performing up-sampling on the multi-scale feature map in combination with a space attention mechanism and a channel incentive mechanism to obtain a multi-level feature map. Through image preprocessing, multi-scale feature extraction and an up-sampling technology combining a space and a channel attention mechanism, high-precision detection of the cigarette ash packing crack is realized, and meanwhile, the calculation efficiency is optimized, so that the method is accurate and efficient, and is suitable for industrial-grade real-time detection requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of cigarette detection, and in particular to a cigarette package ash crack detection method and detection system based on deep learning semantic segmentation. Background Art

[0002] The degree of ash column shrinkage and the proportion of ash cracks are important indicators for the quality of cigarette ash. Among them, the degree of ash column shrinkage mainly reflects the filling rate of tobacco. The internal pores of cigarettes with high filling rates are small, and the ash column shrinks less after burning. The formation of ash cracks is caused by multiple factors such as the wrapping performance of cigarette paper, tobacco formula, filling performance, and combustion thermal performance. The proportion of cracks on the ash column after burning is positively correlated with the ash flakes scattered during smoking. The ash flakes scattered will cause environmental pollution on the one hand, and affect consumers' smoking experience on the other hand.

[0003] The main trend in the field of cigarette inspection is to use visual image detection methods to detect the shrinkage of ash columns and the proportion of ash cracks. For visual image detection methods, the detection of the shrinkage of ash columns is relatively easy to achieve. However, the accurate detection of the proportion of crack area has always been a challenging problem. On the one hand, there are many types of colors and textures of cigarette paper, and the surface morphology after burning is relatively complex. On the other hand, the inadequate burning of cigarettes will lead to large differences in the color of ash columns (yellow, black, gray, and white), and the contrast between cracks and background at different positions is different. Conventional image segmentation algorithms are difficult to accurately extract the crack area. Therefore, this chapter focuses on the accurate and efficient detection of ash crack areas.

[0004] In the field of computer vision research, ash crack detection belongs to the problem of detecting cracks, cracks and other abnormal areas in defect detection. Early classic image algorithms mainly rely on the pixel value characteristics or texture characteristics of cracks for analysis and detection. For example, methods based on HoG gradient histograms, methods based on the combination of filters and machine learning, and local threshold segmentation methods. However, the versatility of such algorithms is limited by environmental changes and fixed parameters of manual feature extraction. It is difficult to achieve good results in the task of ash crack detection with significant global color changes. Especially during the burning process of cigarettes, the ash cracks are often irregular in shape and may appear in a variety of shapes such as lines, meshes or blocks. The cracks may run continuously throughout the entire ash column or appear only in local areas. As the burning proceeds, the cracks may gradually expand, resulting in the loss of the integrity of the ash column. The color of the cracks is usually different from the color of the surrounding ash, and may appear darker black or dark gray due to incomplete combustion or different degrees of oxidation. In summary, the ash cracks present characteristics such as strip distribution, unequal areas and darker colors during the smoking process, which poses a challenge to existing detection methods.

[0005] In view of this, the inventor, based on many years of production design experience in this field and related fields, has designed a cigarette ash crack detection method and detection system based on deep learning semantic segmentation after repeated experiments, in order to solve the problems existing in the prior art. Summary of the invention

[0006] The purpose of the present invention is to provide a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, which can achieve high-precision segmentation under complex backgrounds and diversified crack morphologies.

[0007] In order to achieve the above-mentioned object of the invention, the present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, wherein the cigarette ash crack detection method comprises:

[0008] Acquire the ash crack image during cigarette burning and pre-process the ash crack image;

[0009] Use convolutional neural network to extract multi-scale features from the preprocessed ash crack image and obtain a multi-scale feature map;

[0010] The spatial attention mechanism and channel excitation mechanism are combined to upsample the multi-scale feature maps and obtain multi-level feature maps.

[0011] The present invention also proposes a cigarette crack detection system based on deep learning semantic segmentation, wherein the system implements the above method and includes the following modules:

[0012] An image acquisition module, used for obtaining crack image data in the cigarette burning process in real time;

[0013] A data preprocessing module is used to remove noise, calibrate grayscale and enhance contrast of image data;

[0014] The crack segmentation module includes an encoder-decoder structure, a squeeze-and-excitation module, and a joint excitation upsampling module, which combines the upsampling operation of multi-level feature maps to achieve high-precision segmentation of the crack area;

[0015] The output module is used to generate visual segmentation results and analysis reports of the crack area.

[0016] The present invention also provides a computer-readable storage medium on which a computer program is stored, characterized in that the computer program implements the steps of the above-mentioned calculation method when executed by a processor.

[0017] The present invention also proposes a computer device, comprising a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, and the device is characterized in that the steps of the above-mentioned calculation method are implemented when the processor executes the computer program.

[0018] Compared with the prior art, the present invention has the following characteristics and advantages:

[0019] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, which effectively improves the accuracy and efficiency of cigarette ash crack detection through deep learning semantic segmentation technology, especially CSegNet and JEUNet algorithms. Through the combination of convolutional neural network (CNN) and Transformer attention mechanism, high-precision segmentation of ash crack areas under complex backgrounds is achieved. In particular, in the JEUNet algorithm, the calculation efficiency is greatly improved by reducing the number of parameters, while maintaining high-precision segmentation results, which is very suitable for large-scale real-time detection industrial applications. The ability to accurately detect cracks under different backgrounds, colors and lighting conditions reduces the demand for computing resources in the upsampling process, enhances the clarity and coherence of the crack boundary, and adapts to the diversity of crack morphology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings described herein are only for explanation purposes and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the various components in the figures are only schematic, used to help understand the present invention, and are not specifically limited to the shapes and proportional dimensions of the various components of the present invention. Those skilled in the art can select various possible shapes and proportional dimensions to implement the present invention according to the teachings of the present invention.

[0021] Figure 1 A schematic diagram of a cigarette ash crack image of the present invention;

[0022] Figure 2 It is a schematic diagram of the cigarette ash crack detection method based on deep learning semantic segmentation of the present invention;

[0023] Figure 3 It is a schematic diagram of the ECBAM module structure of the present invention;

[0024] Figure 4 It is a schematic diagram of the ECAM module structure of the present invention;

[0025] Figure 5 It is a schematic diagram of the SAM module structure of the present invention;

[0026] Figure 6 It is a schematic diagram of the JEUNet network structure of the present invention;

[0027] Figure 7 This is a schematic diagram of the structure of the joint excitation upsampling (JEU) module of the present invention;

[0028] Figure 8A comparison chart of the accuracy and computational complexity of JEUNet of the present invention and other advanced segmentation algorithms;

[0029] Fig. 9 It is a comparison diagram of the cigarette ash crack image, ground truth and multi-method segmentation effects of the present invention. DETAILED DESCRIPTION

[0030] The details of the present invention can be more clearly understood with reference to the accompanying drawings and the description of the specific embodiments of the present invention. However, the specific embodiments of the present invention described herein are only used for the purpose of explaining the present invention and cannot be understood as limiting the present invention in any way. Under the guidance of the present invention, technicians can conceive of any possible variations based on the present invention, which should all be considered to belong to the scope of the present invention.

[0031] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation. The cigarette ash crack detection method includes:

[0032] Acquire the ash crack image during cigarette burning and pre-process the ash crack image;

[0033] Use convolutional neural network to extract multi-scale features from the preprocessed ash crack image and obtain a multi-scale feature map;

[0034] The spatial attention mechanism and channel excitation mechanism are combined to upsample the multi-scale feature maps and obtain multi-level feature maps.

[0035] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, which realizes high-precision detection of cigarette ash cracks through image preprocessing, multi-scale feature extraction, and upsampling technology of joint spatial and channel attention mechanisms, while optimizing the computational efficiency, making the method both accurate and efficient, and suitable for industrial-level real-time detection needs.

[0036] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, such as Figure 1 As shown in the figure, the shape, size, texture and contrast of the cracks in the gray package show significant diversity and complexity. In order to achieve accurate segmentation of the cracks in the gray package, this paper proposes a crack segmentation algorithm (Crack Segmentation Network, CSegNet) that combines convolutional neural network CNN and Transformer self-attention mechanism. The CSegNet network adopts an Encoder-Decoder encoding and decoding framework similar to DeepLab V3+, and includes the following steps:

[0037] 1) Obtain crack images during cigarette burning and preprocess the images, including noise removal, grayscale calibration, and contrast enhancement;

[0038] 2) The ResNeXt-Transformer (ResNeXTR) module is used in the encoder, which includes the ResNeXt50 convolutional network and the Swin-Transformer network to extract local and global features in the image;

[0039] 3) In the decoder, an efficient convolutional block attention module (ECBAM) is combined, including an efficient channel attention module (ECAM) and a spatial attention module (SAM) to enhance the salient features of the gap area;

[0040] 4) Use average pooling and point-wise convolution operations to reduce the dimension of the encoder output to reduce the cost of self-attention calculation;

[0041] 5) Using the combined loss function of Binary Cross-Entropy with Logits and Dice loss, the Binary Cross-Entropy with Logits loss and Dice loss functions are combined with a weight of 0.5 each to optimize the segmentation accuracy and edge detection effect.

[0042] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation. In the encoder part, a ResNeXt-Transformer (ResNeXTR) feature extraction module is designed as the backbone network. ResNeXTR enhances the feature extraction capability of the crack area in the ash image by combining the flexibility of Transformers in modeling long-term dependencies and the ability of CNN to capture local context information through local receptive fields, weight sharing and spatial subsampling. In addition, in order to reduce the computational cost of Transformers, an average pooling layer is introduced to downsample the dimension of the encoded features.

[0043] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation. In the decoder part, an efficient channel attention module (ECAM) and a spatial attention module (SAM) are combined to design an efficient convolutional block attention module (ECBAM) to further optimize the feature expression to focus on the key information of cracks under different background colors.

[0044] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation, which adopts ReLU activation function, SGD gradient descent and Binary Cross Entropy with Logits mixed loss function to accelerate the convergence speed and improve the segmentation accuracy.

[0045] In an optional embodiment of the present invention, the preprocessing includes grayscale calibration, noise removal and contrast enhancement. Grayscale calibration ensures the authenticity of the image color and provides accurate color information for feature extraction; noise removal purifies the image, reduces the interference of irrelevant information, and improves the image quality; contrast enhancement further highlights the crack features and enhances the model's ability to recognize the crack.

[0046] Specifically, the grayscale calibration is first performed on the image of the ash crack obtained during the cigarette burning process to ensure that the color information of the image is accurate, thereby providing a reliable color basis for subsequent feature extraction. Next, the random noise in the image is eliminated through the noise removal step, which helps to reduce interference information and make the features of the crack clearer. Finally, the contrast enhancement technology is used to enhance the visual difference between the crack and the background in the image, making the crack area more prominent, which is convenient for the subsequent segmentation network to accurately identify and segment. These preprocessing steps work together to provide a solid image foundation for high-precision crack detection.

[0047] In an optional embodiment of the present invention, a convolutional neural network is used to perform multi-scale feature extraction on the preprocessed ash package crack image, including encoding in 4 stages through continuous convolutional layers, and downsampling and transmission between each two stages through 2×2 maximum pooling. The encoding process of the continuous convolutional layer enables the model to gradually abstract and extract local features of the image, while the 2×2 maximum pooling operation retains the most discriminative features while reducing the dimension of the feature map, thereby enhancing the model's ability to recognize cracks of different scales. The application of this multi-scale feature extraction technology significantly improves the accuracy and robustness of crack detection, allowing the model to operate stably in complex and changeable industrial environments and effectively identify and segment the cigarette ash package crack area.

[0048] Specifically, packet crack detection can be described as a pixel-level anomaly prediction task. The proposed joint excitation upsampling network JEUNet has a classic encoder-decoder symmetric structure. Its network framework is as follows: Figure 6 As shown. The yellow arrow indicates direct input; the black arrow indicates a 3×3 convolution operation with a stride of 1, which uses the zero-padding strategy to keep the feature maps of the same level at the same size; the gray arrow indicates a cut and connect operation, which cuts the left feature map to the size corresponding to the right and then connects it; the blue arrow indicates a 1×1 operation for final classification, and the last two layers of output are the result and the background; the red arrow indicates downsampling the feature map by 2×2 maximum pooling; the green arrow indicates upsampling the feature map by a factor of 2 using a 2×2 deconvolution operation. k indicates the number of base channels of the convolution feature map (in this article, k = 32).

[0049] In an optional embodiment of the present invention, during the decoding process, the feature maps in the second, third and fourth stages are adjusted to the same size and then cascaded, and then the output is adjusted to the original image size through 1×1 convolution. Through the encoding process of continuous convolutional layers, the model can deeply analyze the image and extract rich local features. Secondly, the 2×2 maximum pooling operation effectively reduces the spatial dimension of the feature map while retaining the most representative features, which helps to reduce the amount of calculation and improve the generalization ability of the model.

[0050] Specifically, the encoder part of JEUNet is consistent with the backbone network of UNet. The original input image is encoded in 4 stages through continuous convolutional layers, and is transmitted through 2× maximum pooling downsampling between every 2 stages. Therefore, there are 5 scales including the original image. The feature map of each scale contains information with different receptive fields. The shallow feature map is mainly the detailed texture information of local pixels, while the deep feature map contains the local semantic information in the image. In the decoding process of JEUNet, the outputs of the 2nd, 3rd and 4th stages are input to the designed JEU (Joint Excitation Up-sampling block) module, which replaces the continuous upsampling of convolutions at three scales in the original UNet. The output of the JEU module is upsampled once and cascaded with the output of the first stage of the encoder, and then two 3×3 convolution operations are performed in the same layer. Finally, it is upsampled to the original image size and connected with the initial convolution result of the original image. After two convolution operations, the prediction result is obtained. In the decoder part, the upsampled feature map is cascaded with the shallow features of the encoder part to improve the local prediction accuracy of the pixel.

[0051] The present invention proposes a cigarette ash crack detection method and detection system based on deep learning semantic segmentation. The research motivation of CSegNet is to achieve accurate segmentation of ash crack area. However, since it is embedded with ResNeXt50, Swin-Transformer V2, ECBAM and other modules, the parallel operation of these modules consumes a lot of computing resources, which is not conducive to the engineering of the algorithm. Therefore, it is necessary to study an efficient semantic segmentation algorithm in order to provide a fast and accurate algorithm for the engineering realization of ash crack detection.

[0052] In order to solve the problem that existing semantic segmentation algorithms need to consume large computing resources while achieving accurate segmentation, this section designs a Joint Excitation Up-sampling Network (JEUNet) for cleft segmentation. JEUNet is a network with an encoder-decoder symmetric structure. In the decoder stage, spatial attention and joint upsampling are integrated to replace the general layer-by-layer upsampling method. At the same time, feature information at multiple scales and the correlation of channel directions are considered, which improves the upsampling speed and enhances the accuracy of semantic segmentation.

[0053] In an optional example of this embodiment, the feature maps of the 4th and 3rd stages are upsampled to the same size as the feature map of the 2nd stage using embedded vector extraction, and the three upsampled feature maps are cascaded and merged into a multi-level feature map. Through embedded vector extraction and feature map fusion, multi-scale feature fusion of cigarette ash crack images can be achieved, greatly improving the accuracy and robustness of segmentation. Upsampling and cascading operations enable the model to make full use of feature information at different levels, improving the ability to recognize crack areas, especially in terms of details and contextual information in the crack area. The use of this multi-level feature map makes the network more effective in dealing with complex backgrounds and cracks of various shapes.

[0054] Specifically, the Jointly Excited Upsampling (JEU) module is a computational unit that integrates spatial attention into the joint upsampling network. On the one hand, three feature maps with different receptive fields are adjusted to the same size and connected as a joint feature map T; on the other hand, a transformation vector W is constructed to excite each channel in the tensor T. During the upsampling process, the JEU module simultaneously considers the correlation between multi-scale feature information and channel direction, improves the accuracy of semantic segmentation, and reduces the complexity of upsampling calculations. Figure 7 As shown, the structure diagram of the JEU block, the purple arrow indicates the joint upsampling path, and the red arrow indicates the feature map excitation path for multi-level feature map joint upsampling.

[0055] The JEUNet of the present invention uses joint upsampling to replace the initial two layer-by-layer upsampling in the decoder link of UNet. Through embedding vector extraction, the multi-level feature maps output by the 2nd, 3rd and 4th stages are respectively input into the designed JEU module, and the feature maps of stage 4 (w×w) and stage 3 (2w×2w) are upsampled to 4w×4w of the same size as stage 2. The feature maps after joint upsampling are concat-operated to obtain a multi-level feature map T of size 4w×4w×X (X=4k+8k+16k=28k=896). After joint upsampling, the new three feature maps are concatenated and merged into a multi-level feature map T of size 4w×4w×X, where X=4k+8k+16k=28k=896. The multi-level feature map T contains 4w×4w embedding vectors of size 1×X, each of which contains semantic information of three different receptive fields and corresponds one-to-one to each position (x, y) in the original image.

[0056] In an optional embodiment of the present invention, the spatial attention mechanism processes multi-level feature maps through maximum pooling and average pooling operations, and the pooling results are spliced ​​and globally weighted. The use of the spatial attention mechanism to process multi-level feature maps can significantly improve the accuracy and efficiency of cigarette ash crack detection. The combined use of maximum pooling and average pooling enables the model to capture both local significant features and global statistical features, enhancing the model's ability to identify crack areas in images. Through global weighting, the model can focus more on features that are critical to crack determination, thereby accurately segmenting the crack area in a complex background.

[0057] Specifically, in order to enable the network to effectively focus on the key information of the crack in a complex background while suppressing irrelevant information from different channels, CSegNet designs an Efficient Convolutional Block Attention Module (ECBAM) at the output of the decoder. The ECBAM structure is as follows: Figure 3 In order to reduce the problem of increasing model parameter count caused by adding the hybrid attention mechanism module, ECBAM first inputs the feature map into the ECAM (Efficient Channel Attention Module) module, as shown in Figure 4 As shown in Figure 2, ECAM uses 1D convolution to effectively realize the cross-channel interaction of local information; then, the feature map is input into the SAM (Spatial Attention Module) module, as shown in Figure 2. Figure 5 As shown in Figure 1, SAM performs maximum pooling and average pooling on the feature map at the same time, concatenates the results and performs an overall weighted operation on the feature map. This design enables ECBAM to focus on the global semantic information of the defect area while introducing the channel correlation of the feature map.

[0058] In an optional embodiment of the present invention, the channel excitation mechanism adjusts the weights of the feature responses of each channel through global average pooling.

[0059] Specifically, after joint upsampling, the JPU block then uses 4 dilated convolutions with different expansion rates to extract features and concatenate them. The purpose is to capture feature information at different scales in the multi-level feature map to ensure the accuracy of the semantic segmentation upsampling decoder. However, each scale of the feature map in the multi-level feature map itself contains semantic information of different receptive fields. The use of multiple dilated convolutions may be repeated with the multi-level embedding vector itself, resulting in no significant improvement in the accuracy of semantic segmentation. In addition, the multi-level feature map T contains X channels, each of which is obtained by convolution kernels with different parameters. The correlation between these channels has not been paid attention to, so the accuracy improvement of the semantic segmentation prediction results is limited.

[0060] In order to adaptively recalibrate the feature response in the channel direction and model the interdependence between different channels. We introduced the SE (Squeeze-and-Excitation) module in the JEU block to integrate spatial attention into the structure of the upsampling network, which is also a significant difference between the JEU and JPU modules. As shown in Formula 13, for a 4w×4w×X Tensor T, a global average pooling function is used to compress it into a 1×X feature vector F, and the xth element F in F x The calculation method is as follows:

[0061]

[0062] In the formula, F(x) corresponds to the x-th channel in Tensor T, and I and J are the x-th channel feature map T respectively. x The length and width of (I = J = 4w), i and j represent the coordinates of each point on the feature map. In order to establish the correlation between the features of different channels, F is connected to two fully connected layers to calculate the contribution weights of different channels during the learning process and to activate the corresponding channels in the feature map during prediction, as shown in formula (14).

[0063]

[0064] Where δ() represents the ReLU activation function and σ() represents the Sigmoid activation function.

[0065] During the learning process, the feature vector F is mapped through two fully connected layers to form the final weight vector W, where the weight of each channel in W is W xThe importance of the corresponding channel x in the multi-level feature map T is predicted to model the correlation between feature channels. In the two fully connected layer operations, the vector F is scaled (1 / 2) and restored once to reduce the amount of calculation. w1 is the mapping weight vector of the first fully connected layer, and w2 is the mapping weight vector of the second fully connected layer. Represents the feature map T corresponding to the xth channel x With the weight vector W x Weighted multiplication of the excitation feature map, the JEU block assigns a new response weight to each channel c in the multi-level feature map T obtained by joint upsampling through the weight vector W, and finally obtains the multi-level feature map after spatial channel excitation

[0066] In an optional embodiment of the present invention, the cigarette ash crack detection method further includes: constructing a cigarette ash crack detection model based on a multi-level feature map and training the cigarette ash crack detection model; and performing ash crack detection on cigarettes using the trained cigarette ash crack detection model.

[0067] Specifically, the computing platform settings are: JEUNet is implemented on the pytorch library, and the image reading and adjustment parts call some functions in Opencv 5.0. The CPU is Core(R) i7-8700K CPU@3.70GHz×12 (Intel Corporation, USA), graphics card is GeForce RTX 3080 (Nvidia Corporation, USA). The Python version of the deep learning environment is 3.6, the PyTorch version is 1.10.1, the OpenCV version is 4.6, the CUDA version is 11.3, and the CUDNN version is 8.2.

[0068] On the constructed cigarette package ash crack dataset, JEUNet randomly selected 90 images with ground truth and divided them into training set and validation set according to 10-fold Cross Validation for training, and tested on the remaining 10 images with ground truth. The images input to the network were uniformly resized to 448×448 pixels. The SGD optimizer was used during training, with momentum of 0.9 and weight_decay of 0.0001. The Binary Cross-Entropy loss function was used, the learning rate was set to 0.001, the batch_size was set to 1, and the training epochs was 100 rounds.

[0069] In an optional embodiment of the present invention, the cigarette ash crack detection model is trained by using the Binary Cross-Entropy loss function in combination with the Dice loss function, and the robustness of the cigarette ash crack detection model is verified by using 10-fold Cross Validation.

[0070] Specifically, in the training and validation stages of the model, a joint combined loss function consisting of the Binary Cross-Entropy with Logits (BCE With Logits Loss) loss function and the Dice loss function was designed.

[0071] (1) The Binary Cross-Entropy with Logits loss function is a loss function commonly used for binary classification problems, which combines the Sigmoid activation function and the binary cross entropy loss. In the crack segmentation task, the BCEWithLogits function provides clear feedback to the model, and its definition is shown in Equation 9:

[0072]

[0073] Where N represents the total number of samples; y i represents the true label of the i-th sample, 1 means that the pixel is a crack, and 0 means that the pixel does not belong to a crack; p i is the original output predicted by the model for the i-th pixel. i ) represents the result obtained by processing the original output of the model's prediction for the i-th pixel through the sigmoid function.

[0074] (2) The Dice loss function is different from the Binary Cross-Entropy function. The Dice loss function measures the overlap between the label and the prediction. Assuming that X represents the true label of the true crack segmentation image and Y represents the pixel category of the model predicted segmentation image, the definition of the Dice loss function is shown in Equation 10 and Equation 11:

[0075] L Dice =1-Dice Coefficient (10)

[0076]

[0077] In the formula, |X∩Y| represents the number of elements in the intersection of X and Y, and |X| and |Y| represent the number of elements in X and Y respectively. The final loss function used by the model is shown in formula 12:

[0078] L=0.5L BCEwtihlogits +0.5L Dice (12)

[0079] This formula combines the BCE With Logits loss and the Dice loss function with a weight of 0.5 each. The combination of these two loss functions helps to balance the accuracy of crack detection and boundary delineation in the segmentation task.

[0080] The present invention also proposes a cigarette crack detection system based on deep learning semantic segmentation, characterized in that the system implements the above method and includes the following modules:

[0081] An image acquisition module, used for obtaining crack image data in the cigarette burning process in real time;

[0082] A data preprocessing module is used to remove noise, calibrate grayscale and enhance contrast of image data;

[0083] The crack segmentation module includes an encoder-decoder structure, a squeeze-and-excitation module, and a joint excitation upsampling module, which combines the upsampling operation of multi-level feature maps to achieve high-precision segmentation of the crack area;

[0084] The output module is used to generate visual segmentation results and analysis reports of the crack area.

[0085] The present invention also provides a computer-readable storage medium on which a computer program is stored, characterized in that the computer program implements the steps of the above-mentioned calculation method when executed by a processor.

[0086] The present invention also proposes a computer device, comprising a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, and the device is characterized in that the steps of the above-mentioned calculation method are implemented when the processor executes the computer program.

[0087] The specific implementation process of the present invention is now described in detail with reference to the embodiments:

[0088] In order to evaluate the performance of the algorithm in the segmentation task, the paper comprehensively considered three commonly used evaluation indicators: IoU (Intersection over Union), Dice coefficient and Area Under the Receiver Operating Characteristic Curve (AUROC). These evaluation indicators are widely used in the evaluation of image segmentation performance. The calculation formulas of these evaluation indicators are shown in Figures 15 to 17.

[0089]

[0090]

[0091] In Equations 15 to 17: A represents the pixel area predicted to be a crack; B represents the pixel area that is actually a crack; ∩ represents the number of pixels that overlap with A and B; ∪ represents the total number of pixels in A and B; TPR(t) is the True Positive Rate at threshold t, and t ranges from 0 to 1. IoU, Dice, and AUROC are all used to evaluate the consistency of the predicted results with the ground truth in terms of spatial coverage. IoU (Jaccard coefficient) represents the ratio of the intersection and union of the predicted crack pixel area to the actual crack pixel area. The closer the IoU value is to 1, the higher the overlap between the predicted result and the actual situation, and the better the model performance. The Dice coefficient is an indicator commonly used to evaluate binary segmentation problems. Similar to IoU, the Dice coefficient also measures the similarity between the predicted result and the actual situation. The closer the Dice coefficient is to 1, the higher the prediction accuracy. AUROC is a statistical measure for evaluating the performance of a classifier at different decision thresholds. It is based on the ROC curve, which is a graphical representation of the true positive rate TPR (the proportion of positive samples correctly predicted as positive by the classifier) ​​and the false positive rate FPR (the proportion of negative samples incorrectly predicted as positive by the classifier). The closer the AUROC value is to 1, the better the segmentation performance.

[0092] Optimization Experiment

[0093] CSegNet optimization experiment

[0094] In order to investigate the effect of adding different modules to the benchmark model on the crack segmentation results, the CSegNet benchmark model uses the pre-trained ResNeXt50 network as the encoder and the decoder part of DeepLab V3+ as the decoder. An ablation experiment is performed on a comprehensive large dataset, and the experimental parameters are set according to 1.1.5. The experimental results are shown in Table 1. It can be seen that when the ResNeXt50 network as the encoder is replaced with the ResNeXt101 network with a deeper network layer, the model's crack segmentation results are worse; the shallow features of the ResNeXt50 network are combined with the output of SwinTransformer V2, combined with the deep features of ResNext50, and the ECA attention mechanism is added. The model has the best detection effect, with IOU and AUROC reaching 66.01% and 89.16% respectively.

[0095] Table 1:

[0096]

[0097] JEUNet optimization experiment

[0098] The JEUNet encode part is defined as Stage 1-Stage 4 according to the size of the feature map. In order to obtain the best algorithm combination, the paper selects the feature map combination of different stages during upsampling and inputs it into the JEU block and combines it with the feature map excitation path for joint upsampling to obtain different semantic segmentation models. Each model is trained on 90 panoramic gray images for 100 epochs, and then tested on 10 panoramic gray images. The results of the optimization experiment are shown in Table 2. It can be seen that the best upsampling combination is Stage 4+Stage 3+Stage 2, and the IOU and AUROC reach 57.89% and 81.30% respectively. In addition, after combining the excitation path, the IOU and AUROC of JEUNet are increased to 65.73% and 88.07% respectively. The average increase ratio of nearly 8% shows that the channel attention mechanism of the JEU block plays a very important role in the upsampling process.

[0099] Table 2:

[0100]

[0101] Comprehensive comparison of results with SOTA methods

[0102] To evaluate the performance of CSegNet and JEUNet, a comparative experiment was conducted on the cigarette ash crack image dataset with classic models such as Otsu, FCN, FastFCN, U-Net, U-Net++, and DeepLab V3+. Each method was trained on 90 images and tested on 10 images. The Otsu method is an automatic threshold segmentation technique that selects the threshold by maximizing the variance between the foreground and the background. In the case of crack segmentation, the Otsu method can distinguish between cracks (foreground) and non-crack areas (background). FCN is a pioneering work in the field of semantic segmentation in deep learning. It uses convolutional layers instead of the fully connected layers behind traditional CNN to output heat maps and uses upsampling to restore the image size. FastFCN and the classic joint pyramid upsampling block (JPU) greatly improve the upsampling performance of FCN. U-Net, U-Net++, and DeepLab V3+ are Encoder-Decoder architectures with symmetric structures.

[0103] For testing on 10 randomly selected images from the constructed dataset evaluated by IoU, Dice, and AUROC metrics, the Ostu method was used directly, while the other models were trained on 90 ground-truth images, with each model trained 50 to 100 times to achieve convergence. In addition, a complexity analysis of the deep learning based methods was performed, including measurements of the total test time Times (in seconds (s)), the number of floating point operations per second (FLOPs, Floating Point Operations, in gigabytes (G)), and the number of parameters Params (in millions (M)). For all previous deep learning methods and the proposed method, the measured tensor input size was 10×448×448. The comprehensive evaluation results of different methods are shown in Tables 3 and Figure 8 As shown, the following conclusions can be drawn:

[0104] The proposed CSegNet and JEUNet outperform the comparison methods in terms of crack segmentation performance, among which CSegNet achieves the best pixel-level prediction accuracy (IOU and AUROC reach 66.01% and 89.16% respectively), followed by JEUNet (IOU and AUROC reach 65.73% and 88.07% respectively);

[0105] Compared with FastFCN, JEUNet's IoU and AUROC indicators are improved by nearly 30% and 15% respectively, confirming the view that "modeling the interdependencies between different channels can further improve the performance of semantic segmentation networks";

[0106] Compared with the advanced DeepLabV3+ network, CSegNet and JEUNet perform better on the dataset constructed in this paper, with both IoU and AUROC increasing by about 1% to 2%;

[0107] Although CSegNet has achieved the best detection accuracy, it has the highest computational consumption, with Times, FLOPS, and Params all higher than other methods, and is not cost-effective;

[0108] While achieving detection results similar to those of CSegNet, JEUNet has a computational parameter count of 5.93, which is 1 / 6 of DeepLabV3+ and 1 / 7 of CSegNet. It has the shortest training and testing time, and the lowest computational complexity and time cost among all algorithms. From a cost-effectiveness analysis perspective, it is the most suitable method for detecting cracks in ash packaging.

[0109] Table 3:

[0110]

[0111] like Fig. 9 As shown in Figure 2, some challenging segmentation results of different methods are shown compared with manual annotation. It can be observed that the elongated burning lines extracted by CSegNet and JEUNet have the best continuity, and the segmentation results of the cracks are also closest to the ground-truth.

[0112] In order to realize the detection of ash crack rate in the apparent combustion quality index of cigarettes, the present invention uses the panoramic image data set of ash cracks in cigarette burning to carry out crack detection algorithm research, and proposes a cigarette ash crack detection algorithm solution suitable for engineering application requirements.

[0113] First, in order to achieve accurate segmentation of gray-encapsulated cracks under complex backgrounds, a crack precision segmentation algorithm CSegNet was studied and proposed. CSegNet designed a feature extraction module that combined ResNeXt50 and Swin-TransformerV2 as the ResNeXTR backbone network, which effectively enhanced the ability to extract meaningful features from gray-encapsulated images; in addition, ECAM and SAM were combined to construct the ECBAM attention mechanism to further optimize feature representation. CSegNet achieved 66.01% IoU, 80.24% Dice coefficient, and 89.16% AUROC on the constructed dataset.

[0114] Secondly, in order to further reduce the computational cost, an accurate and efficient crack segmentation algorithm JEUNet is studied and proposed. JEUNet integrates spatial attention into the joint upsampling block of the decoder part of the symmetric architecture, and models the correlation between semantic information at multiple scales and joint feature map channels. On the constructed dataset, JEUNet achieved 65.73% IoU, 79.22% Dice coefficient and 88.07% AUROC with a resource occupancy of 5.93M Params, 75.26G floating-point operations per second and a total test time of 106s, providing a cost-effective and engineerable algorithm for packet crack detection.

[0115] The detailed explanation of the above-mentioned embodiments is only intended to explain the present invention so as to facilitate a better understanding of the present invention. However, these descriptions cannot be interpreted as limitations on the present invention for any reason. In particular, the various features described in different embodiments may also be arbitrarily combined with each other to form other embodiments. Unless there is a clear description to the contrary, these features should be understood to be applicable to any embodiment and are not limited to the described embodiments.

Claims

1. A cigarette ash crack detection method based on deep learning semantic segmentation, characterized in that: The cigarette pack ash crack detection method comprises: Acquire an ash crack image during cigarette burning, and pre-process the ash crack image; Using a convolutional neural network to extract multi-scale features from the pre-processed ash crack image and obtain a multi-scale feature map; The spatial attention mechanism and channel excitation mechanism are combined to upsample the multi-scale feature maps and obtain multi-level feature maps.

2. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: The preprocessing includes grayscale calibration, noise removal and contrast enhancement.

3. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: A convolutional neural network is used to perform multi-scale feature extraction on the preprocessed gray crack image, including encoding in 4 stages through continuous convolutional layers, and downsampling and transmitting through 2×2 maximum pooling between every two stages.

4. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: During decoding, the feature maps in stages 2, 3, and 4 are resized to the same size and then concatenated, and then the output is resized to the original image size through 1×1 convolution.

5. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 4, characterized in that: Using embedding vector extraction, the feature maps of the 4th and 3rd stages are upsampled to the same size as the feature map of the 2nd stage, and the 3 upsampled feature maps are cascaded and merged into a multi-level feature map.

6. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: The spatial attention mechanism processes multi-level feature maps through maximum pooling and average pooling operations, and concatenates the pooling results and performs global weighting.

7. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: The channel excitation mechanism adjusts the weights of the feature responses of each channel through global average pooling.

8. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: The cigarette pack ash crack detection method also includes: Constructing a cigarette ash crack detection model based on the multi-level feature graph and training the cigarette ash crack detection model; The trained cigarette ash crack detection model is used to perform ash crack detection on cigarettes.

9. The cigarette ash crack detection method based on deep learning semantic segmentation according to claim 1, characterized in that: The Binary Cross-Entropy loss function combined with the Dice loss function was used to train the cigarette ash crack detection model, and the 10-fold Cross Validation was used to verify the robustness of the cigarette ash crack detection model.

10. A cigarette crack detection system based on deep learning semantic segmentation, characterized in that: The system implements the method described in any one of claims 1 to 9, and includes the following modules: An image acquisition module, used for obtaining crack image data in the cigarette burning process in real time; A data preprocessing module is used to remove noise, calibrate grayscale and enhance contrast of image data; The crack segmentation module includes an encoder-decoder structure, a squeeze-and-excitation module, and a joint excitation upsampling module, which combines the upsampling operation of multi-level feature maps to achieve high-precision segmentation of the crack area; The output module is used to generate visual segmentation results and analysis reports of the crack area.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the computing method according to any one of claims 1 to 9 are implemented.

12. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the computing method according to any one of claims 1 to 9 are implemented.

Images