A dense tobacco shred segmentation method and device based on an improved DeepLabv3+ network model

Abstract

The application provides a dense cut tobacco segmentation method and device based on an improved DeepLabv3+ network model, and belongs to the technical field of cigarette detection. A cut tobacco image on a conveying belt is collected, and a cut tobacco image dataset is constructed, wherein cut tobacco in the cut tobacco image is densely stacked; a dense cut tobacco segmentation model based on the improved DeepLabv3+ network model is constructed, the dense cut tobacco segmentation model comprises an encoding module and a decoding module, and the encoding module takes a MobileNetv2 network as a backbone network; the dense cut tobacco segmentation model is trained through the dataset; and the trained dense cut tobacco segmentation model is used to identify cut tobacco in a cut tobacco stack. The application realizes efficient and accurate segmentation of dense cut tobacco under a complex background, thereby providing technical support for online measurement of the width of cut tobacco on a cut tobacco production line, and improving the monitoring efficiency and accuracy of cut tobacco processing quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cigarette detection, in particular to a dense cut tobacco segmentation method and device based on an improved DeepLabv3+ network model. BACKGROUND

[0002] Tobacco is one of the important economic crops. The finished cigarette made of tobacco needs to go through a strict processing technology, and the tobacco cutting is one of the important links. The tobacco width is the main quality index of the tobacco cutting production process. However, the current tobacco cutting process cannot realize online detection of the tobacco width, and is mainly based on manual measurement. In the tobacco cutting process, the technician cannot master the tobacco cutting quality information in real time, cannot find the problems existing in the tobacco cutting process in time, and is easy to cause the tobacco cutting quality to deviate from the industry standard, thereby affecting the subsequent processing technology. In addition, the difficulty in detecting the tobacco width in the dense state also restricts the development of the tobacco cutting technology to intelligence, thereby affecting the production efficiency.

[0003] At present, in the research on the detection of the tobacco width, researchers usually take discrete single tobacco as the research object, use the digital image processing technology to extract the tobacco by using the color, shape, texture and other characteristics of the tobacco, and detect the tobacco width by using the Hough transform and matrix inversion method. At present, there are researches on the measurement of the tobacco width in the discrete and artificial placement states. However, in the complex environment of the dense stacking of the tobacco cutting production line, the image processing algorithm is easy to be affected by the shielding and noise. Moreover, the tobacco cutting production line belongs to the industrial production application scene, and has a high real-time requirement. The traditional image processing detection cannot completely meet the actual production.

[0004] In recent years, with the rapid development of deep learning in various fields, more and more researchers apply deep learning technology to the image recognition research in the agricultural field. Compared with the traditional semantic segmentation method, the semantic segmentation method based on deep learning can learn the feature information of the image through the neural network, and obtain the semantic information at different levels, and has better generalization ability. For example, a method based on Mask RCNN replaces the backbone network and the feature pyramid structure by using Densenet121 and U-FPN, and the recognition accuracy of the network to the overlapping tobacco is 89.1%. A two-level Unet tobacco impurity detection model has an average detection accuracy of 96.6% to the impurities, and the picture processing speed is 21ms / picture.

[0005] However, in the dense stacking state recognition task, the tobaccos are intertwined with each other and there is shielding, so that the boundary information of the tobacco is blurred, and the recognition difficulty of the tobacco suitable for measuring the width is increased. Moreover, most of the researches pay less attention to the lightweight problem of the model, and the network structure constructed is too complex, which leads to too large calculation amount and is not conducive to deployment to the tobacco cutting production line with limited computing power.

[0006] In view of this, the present inventors, based on many years of production design experience in this field and related fields, have designed a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model through repeated experiments, in order to solve the problems existing in the prior art. SUMMARY

[0007] The purpose of the present application is to provide a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, which can accurately measure the width of tobacco online.

[0008] To achieve the above-mentioned purposes, the present application provides a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, wherein a tobacco image on a conveying belt is collected and a data set of the tobacco image is constructed, and the tobacco in the tobacco image is densely stacked;

[0009] An improved DeepLabv3+ network model-based dense tobacco segmentation model is constructed, and the dense tobacco segmentation model includes an encoding module and a decoding module, and the encoding module takes a MobileNetv2 network as a backbone network;

[0010] The dense tobacco segmentation model is trained through the data set;

[0011] The trained dense tobacco segmentation model is used to identify the tobacco in the tobacco stack.

[0012] The present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0013] The present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and wherein the processor implements the steps of the above method when executing the computer program.

[0014] Compared with the prior art, the present application has the following characteristics and advantages:

[0015] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model solve the problems of complex image background, easy occlusion and difficult model deployment. BRIEF DESCRIPTION OF DRAWINGS

[0016] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure in any way. In addition, the shapes and proportions of the components in the drawings are only illustrative and are used to help understand the present application, and are not specific limitations on the shapes and proportions of the components of the present application. Those skilled in the art can select various possible shapes and proportions according to specific circumstances to implement the present application under the teaching of the present application.

[0017] Figure 1 The dense tobacco segmentation method based on the improved DeepLabv3+ network model of the present application is a flowchart;

[0018] Figure 2 The specific steps of the efficient channel attention mechanism of the present application are shown in the figure, which obtains the weight distribution by calculating the information interaction in each channel;

[0019] Figure 3a The original image of the dense tobacco recognition by the DeepLabv3+ of the present application;

[0020] Figure 3b The label image of the dense tobacco recognition by the DeepLabv3+ of the present application;

[0021] Figure 3c The segmentation image of the dense tobacco recognition by the DeepLabv3+ of the present application;

[0022] Figure 4 The network structure diagram of the improved DeepLabv3+ model of the present application;

[0023] Figure 5 Structure diagram of ECA attention mechanism of the application;

[0024] Figure 6 Fusion diagram of multi-expansion convolution receptive field cascade of the application;

[0025] Figure 7a Tobacco image example of Qipilang white of the application;

[0026] Figure 7b Tobacco image example of Qipilang gray of the application;

[0027] Figure 7c Tobacco image example of Qipilang sharp of the application;

[0028] Figure 8 Loss function curve diagram of improved model training process of the application;

[0029] Figure 9 Segmentation result diagram of different models on tobacco image of the application;

[0030] Figure 10 Tobacco segmentation result diagram under different brands of the application. DETAILED DESCRIPTION

[0031] The details of the application can be more clearly understood in conjunction with the description of the specific embodiments of the application and the accompanying drawings. However, the specific embodiments of the application described herein are only for the purpose of explaining the application, and should not be understood in any way as limiting the application. Based on the teachings of the application, a skilled person can conceive any possible modification based on the application, which should be considered as falling within the scope of the application.

[0032] The application provides a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, wherein a tobacco image on a conveying belt is collected and a data set of the tobacco image is constructed, and the tobacco in the tobacco image is densely stacked;

[0033] A dense tobacco segmentation model based on the improved DeepLabv3+ network model is constructed, the dense tobacco segmentation model comprises an encoding module and a decoding module, and the encoding module takes a MobileNetv2 network as a backbone network;

[0034] The dense tobacco segmentation model is trained through the data set;

[0035] The trained dense tobacco segmentation model is used to identify the tobacco in the tobacco stack.

[0036] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model provided by the application realize efficient and accurate segmentation of dense tobacco under complex background by collecting and constructing an image data set of densely stacked tobacco, training and identifying tobacco images by using a segmentation model composed of an encoding module with a MobileNetv2 network as the backbone combined with a decoding module, thereby providing technical support for online measurement of tobacco width on a tobacco processing line and improving the monitoring efficiency and accuracy of tobacco processing quality.

[0037] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model provided by the application, the image data is obtained from tobacco image data collected in Fujian Longyan Tobacco Industry Co., Ltd. in November 2023. The shooting device is a MARS-1231-46G5C-P Mars series camera produced by Daheng Company, the image data is transmitted through a capture card and saved in a computer, automatic white balance and exposure; the image resolution is 4096 pixels x 3000 pixels, stored in JPG format, the frame rate is 5Hz. During data acquisition, the digital camera is fixed on a support, and the shooting angle is parallel to the direction of the conveyor belt. According to the research target, only the freshly cut tobacco of the tobacco cutting machine is collected in this paper, which is more suitable for actual production application. As shown in Figure 7a 、 Figure 7b and Figure 7c A total of 483 original images were collected on the Longyan No. 1 production line for model training, and an additional 100 images of two different brands of tobacco, Qianma and Qianglang soft gray, were collected for verification of the generalization performance of the model.

[0038] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model provided by the application, the tobacco suitable for width measurement in the tobacco processing workshop environment is interwoven with the sliver and the running film, and the image background color is extremely similar. In this paper, combined with the characteristics of the tobacco data set, the DeepLabv3+ model is used as the basis, and through a large number of experimental analyses, the model parameters are continuously optimized in order to improve the detection accuracy and real-time performance of the model.

[0039] The improved DeepLabv3+ network model-based dense tobacco cutting method and device, DeepLabv3+ is a kind of codec architecture for pixel-by-pixel semantic segmentation. DeepLabv3+ improves its predecessor DeepLabv3, retains the original DeepLabv3 structure as the encoding module, and adds a decoding module to recover the details and dimension information of the target boundary, so as to improve the segmentation accuracy of the model. In the encoding module, the DeepLabv3+ network takes the whole image as the input of the Xception backbone network to obtain the features of the target and input into the decoder. At the same time, in order to reduce the loss of feature information and improve the recognition ability of the model to different scale objects, DeepLabv3+ uses an Atrous spatial pyramid pooling (ASPP) feature fusion module, which uses 1x1 convolution, pooling (Image pooling) and convolution with different dilated rates to extract feature information, wherein the dilated rates are 6, 12 and 18 respectively, to capture multi-scale information and obtain higher semantic information. Finally, the deep features fused by ASPP are input into the 1x1 convolution kernel for channel adjustment and input into the decoding module. In the decoding module, a 4-fold Bilinear interpolation is used to deconvolve the feature map from the output of the encoder. The corresponding low-level features from the backbone network are cascaded. After concatenation (Concat), the feature map is gradually restored to the original spatial dimension through 3x3 convolution and upsampling and refined, so as to realize the semantic segmentation of the target region.

[0040] The improved DeepLabv3+ network model-based dense tobacco cutting method and device proposed by the application adopts the original DeepLabv3+ model to conduct a pre-experiment on the tobacco data set in the cutting workshop, and the recognition result is as follows Figure 3a 、 Figure 3b and Figure 3cThe model fails to accurately restore the detailed tobacco boundary and other detailed features in the case of shielding overlap between the running film, the sliver and the tobacco. In addition, due to the complex structure of the model itself and the large number of parameters, a large amount of video memory and computing time are required, thereby limiting its application in embedded devices. The improvements in this paper are as follows: ①Use a lightweight MobileNetv2 network instead of the original Xception network with a large number of parameters to reduce the model parameter calculation and improve the network segmentation speed, which is better suited for embedded devices. ②Fusion ECA attention module to increase the feature attention weight of the tobacco, so that the network can focus more on the key features of the tobacco, thereby improving the segmentation accuracy. ③In view of the problem that the original model is poor in restoring the edge detail information of the tobacco, a multi-dilated convolution receptive field cascade fusion structure is proposed to improve the ASPP module, which increases the sharing of tobacco features between different levels of ASPP, and different expansion rate convolutions can depend on each other, increasing the receptive field. The improved model structure is as shown in Figure 4

[0041] In an optional embodiment of the present application, for the MobileNetv2 network, first, use 1x1 convolution to map low-dimensional spatial features to high-dimensional space, then use 3x3 deep separable convolution to extract features, and finally use 1x1 convolution to reduce the features to map the high-dimensional space to the low-dimensional space to obtain the feature information, and finally use the shortcut connection to add the input feature and the output feature. Through the shortcut connection to add the input feature and the output feature, not only the information loss in the training process is reduced, but also the direct propagation of the gradient is helpful, thereby accelerating the convergence speed of the model.

[0042] ​Specifically, under the environment of the tobacco factory, the cut tobacco and the sliver are intertwined together and the image background color is extremely similar. In combination with the characteristics of the cut tobacco data set, based on the DeepLabv3+ model, a large number of experimental analyses are conducted to continuously optimize the model parameters, so as to improve the detection accuracy and real-time performance of the model. DeepLabv3+ is a kind of encoder-decoder architecture for pixel-wise semantic segmentation. DeepLabv3+ improves its predecessor DeepLabv3, retains the original DeepLabv3 structure as the encoding module, and adds a decoding module to restore the details and dimension information of the target boundary, so as to improve the segmentation accuracy of the model. In the encoding module, the DeepLabv3+ network takes the whole image as the input of the Xception backbone network to obtain the features of the target and transmit them into the decoder. At the same time, in order to reduce the loss of feature information and improve the recognition ability of the model to different scale objects, DeepLabv3+ uses an Atrous spatial pyramid pooling (ASPP) feature fusion module, which uses 1×1 convolution, pooling (Image pooling), and convolution with different dilation rates to extract feature information, wherein the dilation rates are 6, 12 and 18 respectively, so as to capture multi-scale information and obtain higher semantic information. Finally, the deep features fused by ASPP are input into the 1×1 convolution kernel for channel adjustment and input into the decoding module. In the decoding module, a 4 times bilinear interpolation is used to deconvolve the feature map from the output of the encoder. The corresponding low-level features from the backbone network are concatenated. After splicing (Concat), the feature map is gradually restored to the original spatial dimension through 3×3 convolution and upsampling and is refined, so as to realize the semantic segmentation of the target region.

[0043] In an optional embodiment of the present application, the activation function in the linear bottleneck layer of the MobileNetv2 network is converted into a linear function. By using a linear activation function in the linear bottleneck layer of the MobileNetv2 network, efficient preservation and transmission of feature information can be realized. The information loss caused by the non-linear activation function is reduced, so that the network can more completely capture and utilize the features of the input data. Since the preservation of feature information is more complete, the model can more accurately recognize and segment the target object when processing images, especially in the dense cut tobacco segmentation task under complex background, and this improvement significantly improves the segmentation accuracy and generalization ability of the model.

[0044] Specifically, the original DeepLabv3+ semantic segmentation model uses Xception as the backbone network. The Xception model has the characteristics of deep structure hierarchy and large parameter quantity, which makes the inference time long and the running speed slow on the resource-limited embedded devices in practical applications. In order to improve the inference speed and computational efficiency, a lightweight MobileNetv2 is used as the backbone feature extraction network. MobileNetv2 is a lightweight network model proposed by Google. MobileNetV2 is an improved version based on MobileNetV1. MobileNetV2 introduces inverted residual blocks and linear bottlenecks on the basis of using depth separable convolution, which reduces the number of model parameters, so that the network converges faster. For this network, first, use 1x1 convolution to map low-dimensional spatial features to high-dimensional space, then use 3x3 depth separable convolution to extract features, and finally use 1x1 convolution to reduce the features to map high-dimensional space to low-dimensional space to obtain feature information. Finally, use shortcut (shortcut connection) to add the input features and the output features, so as to improve the feature extraction capability of the network and the effective transmission of multi-layer feature information. And change the activation function in the linear bottleneck layer to a linear function to prevent the nonlinear layer from destroying too much feature information. The structure parameters of MobileNetV2 used in this experiment are shown in Table 1.

[0045] Table 1:

[0046]

[0047]

[0048] In an optional embodiment of the present application, an efficient channel attention mechanism is introduced at the deep features output by the backbone network and the low-level features output by the decoder. The efficient channel attention mechanism obtains a weight distribution by calculating the information interaction within each channel. The introduction of the efficient channel attention mechanism significantly improves the model's ability to recognize and segment tobacco features. Through the ECA mechanism, the model can automatically learn and emphasize those feature channels that are most critical to the tobacco segmentation task, while suppressing those less important features, thereby improving the feature expression ability and the generalization ability of the model. This mechanism is particularly suitable for processing dense tobacco images in complex backgrounds, and can effectively improve the recognition accuracy of the model for tobacco boundaries.

[0049] Specifically, in the ECA attention mechanism module, in the fine segmentation process of cut tobacco and running piece, drawing, and crimped tobacco, due to the complexity of the dense background environment, the cut tobacco and the running piece, drawing are similar in color and are easily blocked. Therefore, an efficient channel attention mechanism is introduced at the deep features output by the backbone network and the low-level features output by the decoder to suppress the interference of non-important targets in the complex background on the network while increasing the attention weight of the network to the boundary features of the cut tobacco.

[0050] In an optional example of this embodiment, the efficient channel attention mechanism provides global feature description for the features in each channel, uses a one-dimensional convolution kernel to convolve the feature vectors in each channel, normalizes the convolution result using a Sigmoid function, and multiplies the original feature vector element by element to obtain the final weighted feature vector. Since the ECA mechanism has low computational complexity, it does not significantly increase the computational burden of the model, allowing the model to maintain high efficiency while achieving higher segmentation accuracy.

[0051] Specifically, unlike the traditional SE (squeeze and excitation module) attention mechanism, as shown in Figure 5 The ECA module avoids dimensionality reduction, is faster, and has lower computational complexity. The ECA structure obtains a weight distribution by calculating the information interaction in each channel, allowing the model to pay more attention to key channel features. The ECA structure provides global feature description for the features in each channel, then uses a one-dimensional convolution kernel to convolve the feature vectors in each channel, normalizes the convolution result using a Sigmoid function, and multiplies the original feature vector element by element to obtain the final weighted feature vector.

[0052] In an optional example of this embodiment, the specific steps of the efficient channel attention mechanism for obtaining a weight distribution by calculating the information interaction in each channel include: performing global average pooling on the input feature map to obtain global channel information; calculating an adaptive convolution kernel size through a fully connected layer; applying the adaptive convolution kernel to one-dimensional convolution operation to obtain an adaptive weight for each channel in the feature map; and multiplying the original input feature and the channel weight by channel to obtain a feature with channel attention.

[0053] First, the input feature map is processed by global average pooling, i.e., the input feature map (H, W, C) is changed to (1, 1, C) to obtain global channel information. That is,

[0054]

[0055] In the formula: U c represents the input feature vector Zc This represents the output feature vector.

[0056] Secondly, an adaptive convolutional kernel size is calculated using a fully connected layer for subsequent convolutional operations.

[0057]

[0058] In the formula: b represents the bias when using an adaptive convolution kernel; γ represents scaling the global average pooling output coefficients; and k represents the convolution kernel size. Represents the mapping function |·| odd Represents the closest odd-numbered object

[0059] Then, an adaptive convolution kernel is applied to the one-dimensional convolution operation to obtain the adaptive weights for each channel in the feature map. A one-dimensional convolution operation with a kernel size of k is performed, and the weights w for each channel are obtained after passing through the σ activation function.

[0060] The formula for calculating 1D convolution is:

[0061] w=σ(Conv K (y))(3)

[0062] In the formula: y is the result after average pooling (Conv) k For a one-dimensional convolution operation with kernel size k

[0063] Finally, the original input features are multiplied by the channel weights one channel at a time to obtain features with channel attention, thereby enhancing the learning and extraction of information of interest to the target region.

[0064] In an optional embodiment of the present invention, the ASPP module of the decoder module adopts a multi-dilated convolutional receptive field cascade fusion structure. By processing dilated convolutions with different dilation rates in parallel, rich multi-scale contextual information is captured, enhancing the network's ability to recognize small objects and large backgrounds.

[0065] Specifically, multi-dilated convolutional receptive field cascaded fusion structures, such as Figure 6 As shown, the multi-dilated convolutional receptive field cascaded fusion structure uses a multi-layer cascaded fusion approach to replace the original parallel structure, thereby enhancing the fusion of feature information at different scales.

[0066] In one alternative embodiment of this implementation, the number of channels is adjusted using 1×1 convolutions, and a multi-dilated convolutional pyramid is constructed to capture global information from the tobacco image using multi-scale receptive fields. The use of a multi-dilated convolutional pyramid allows the model to simultaneously consider both local details and global context of the image, which is particularly important for accurate segmentation in complex backgrounds.

[0067] Specifically, in the ASPP structure, different dilation rates of the empty convolution and the global average pooling operation are adopted to obtain semantic information of the feature map at different levels. However, the parallel structure adopted in the ASPP structure cannot obtain the sharing of the tobacco feature information between different levels, and the original dilation rate interval is set too large, which cannot obtain the extraction of the edge feature information of the tobacco. In order to overcome the insufficient extraction of the tobacco feature information of the ASPP structure, the dilation rate in the ASPP module is adjusted, and a small amount of parameters is added to enhance the extraction of the tobacco feature information.

[0068] In an optional example, the receptive field calculation formula is:

[0069] R f =(Rate-1)×(k-1)+k

[0070] Where R f is the receptive field; rate is the dilation rate; k is the convolution kernel size. Through 1x1 convolution, the channel number is adjusted to 256, and the feature information of the receptive field between different levels is obtained by controlling the dilation rate, step and padding parameters to construct a multi-dilation convolution pyramid. The range of different scale receptive fields is used to better capture the global information of the tobacco image. Through accurate calculation and control of the receptive field, we can ensure that the model can effectively extract features at different levels, thereby improving the recognition ability of the model for small objects and large backgrounds.

[0071] In an optional example of the embodiment, a dilated convolution with a dilation rate of 3 is introduced in the ASPP module. To ensure that the feature information near the edge of the tobacco is given important attention, reduce the loss of global information, and make the network segmentation more accurate. By introducing a dilated convolution with a dilation rate of 3 in the ASPP module, the accuracy of tobacco image segmentation is significantly improved, especially in the feature extraction of the edge region. The improvement makes the network pay more attention to the feature information near the edge of the tobacco, reducing the loss of global information in the segmentation process.

[0072] In an optional example of the embodiment, each level of the ASPP module obtains a feature layer from the backbone network and learns with the features obtained by different dilation rate convolutions at that level. Each level of the ASPP module independently extracts features through different dilation rate convolutions, and then fuses them, so that the model can consider both local details and global context of the image. This multi-scale feature fusion method is particularly suitable for handling occlusion and stacking problems in dense tobacco images, allowing the model to more accurately identify and segment tobacco, even in the presence of overlapping and occluded tobacco.

[0073] Specifically, a multi-expansion convolution cascade fusion structure is adopted to reduce the loss of global semantic information. The output of each layer is adjusted to 320 channels through a 1x1 convolution operation as the input feature of the next layer branch, and a multi-layer cascade fusion method is adopted to increase the sharing degree of tobacco features obtained by different expansion rate dilated convolution. Through the MobileNetv2 feature extraction network, each layer level in the ASPP module obtains a feature layer. Then, these features are fused and learned with the features obtained by different expansion rate convolution of each layer to solve the problem of insufficient global feature extraction capability, so as to ensure that the tobacco feature information can be shared between layers, and the convolution with different expansion rates can be dependent on each other, increasing the range of receptive field.

[0074] In an optional example of this embodiment, the ASPP module outputs 6 feature layers and performs a splicing operation on the 6 feature layers. The 6 feature layers output after the fusion of the feature layers are spliced, that is, the features after 5 times of convolution are stacked with the features of the global average pooling, so as to better capture the global semantic feature information of the tobacco image by using feature layers of different scales. The ASPP module outputs and splices 6 feature layers of different scales, so that the model can comprehensively consider the feature information from different levels, which not only enhances the recognition ability of the model for the details of the tobacco image, but also improves the understanding of the global context.

[0075] In an optional embodiment of the present application, the data set is divided into a training set and a validation set, the training set is sent to the dense tobacco segmentation model for training, and the validation set is used to verify the trained dense tobacco segmentation model. The method of dividing the training set and the validation set not only improves the stability and convergence speed of the model in the training process, but also effectively prevents the overfitting phenomenon of the model through the real-time performance feedback of the validation set. Through the training and verification process, it is ensured that the model can achieve higher segmentation precision and better generalization ability when processing dense tobacco images in complex backgrounds.

[0076] Specifically, since the resolution of the original image obtained by the industrial camera is high, it will cause too much video memory to occupy when training the model, and reduce the operation speed. In order to adapt to the performance of the computer, and at the same time speed up the model loss convergence speed, 483 original image sizes are scaled to 806 pixels x 756 pixels in proportion, the objects contained in the image are divided into two categories suitable for width measurement, namely tobacco and background, and the running film, sliver, stem, and feature insignificant tobacco are labeled as background class, then the Labelme labeling tool is used to generate the tobacco label mask, and the label file is saved in.json format and converted into a grayscale image for training the model. In order to avoid the problem of insufficient number of samples causing low robustness of the model, the data is enhanced by up-down flipping, translation, Gaussian noise and color transformation. The enhanced image is 1170, which is divided into training set and validation set according to the ratio of 8:2, wherein the training set is 936, and the validation set is 234. In order to verify the segmentation effect of the model on the tobacco in the dense stacking state, the robustness of the model is further tested in the case of different brand tobaccos and widths.

[0077] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is characterized in that the computer program is executed by a processor to realize the steps of the method.

[0078] The application further provides a computer device, which comprises a memory and a processor, and the memory stores a computer program capable of running on the processor, and the processor is characterized in that the processor executes the computer program to realize the steps of the method.

[0079] The specific implementation process of the application will be described in detail in combination with embodiments:

[0080] In order to verify the robustness and accuracy of the dense tobacco recognition algorithm based on the improved DeepLabv3+, the dense tobacco segmentation experiment is carried out.

[0081] 3.1 Test environment

[0082] The host processor CPU of the experiment is i9-12900KF, the main frequency is 2GHz, the running memory is 64GB, the GPU is NVIDIA GeForce RTX 4090 graphics card, the memory is 24GB, and the operating system is Windows 10. Anaconda3 is used to configure the deep learning virtual environment, the torch version is 1.7.1, the Cuda version is 11.3, and the Python version is 3.7. In the model training, the pre-training weight provided by MobileNetv2 is used, the SGD optimizer is selected to update the parameters through multiple tests, the initial learning rate is set to 0.007, the batch size is set to 8, the weight decay factor is set to 0.0001, and the image resolution of the input network is 806 pixels x 756 pixels. Dice Loss is used as the loss function, the weight file is saved every 10 iterations, the number of iterations is set to 300, and the model is fully converged.

[0083] 3.2 Evaluation index

[0084] In order to objectively evaluate the segmentation effect of the segmentation model on the dense tobacco image, the mean pixel accuracy (Mean pixel accuracy, mPA), recall, mean intersection over union (Mean intersection over union, MIoU), recall, accuracy (Precision), detection speed (Frames per second, FPS) and memory occupation are selected as the performance evaluation index of different semantic segmentation models. Through the detection speed and memory occupation, the feasibility of the model in practical application is verified.

[0085] 3.3 Model segmentation result

[0086] 3.3.1 Comparison of segmentation performance of different models

[0087] The improved DeepLabv3+ model is trained under the same verification set and configuration parameters as several popular semantic segmentation models such as UNet, PSPNet and HrNet. The training loss curve of the improved model is shown in Figure 8 As the number of training rounds increases, the loss value shows a decreasing trend, and gradually converges and stabilizes. The improved model compares the mPA, MIoU, detection speed, memory occupation and parameter quantity of different models, as shown in Table 2.

[0088] Table 2:

[0089]

[0090]

[0091] From Table 2, it can be seen that the model in this paper performs better in real-time and accuracy. The average pixel accuracy, average intersection over union, recall rate and accuracy of the five models are the best, among which the mPA is 84.39%, which is 1.14 percentage points higher than the original model, the MIoU is 68.3%, which is 2.71 percentage points higher than the original model, the Recall is 80.29%, which is 2.04 percentage points higher than the original model, and the accuracy is improved from 80.5% to 81.89%, which is 1.39 percentage points. This shows that the model in this paper can more accurately identify the edge features of the tobacco. In addition, the improved model also has a significant improvement in detection speed. The memory occupancy is 22.52MB, which is more lightweight compared with other mainstream semantic segmentation models, and the inference time is shorter, reaching 90.14f / s on the validation set, i.e. the processing time of a single image is only 11.08ms. This ensures the rapid deployment and application in resource-limited industrial scenarios, and provides support for fast and accurate measurement of tobacco width.

[0092] The prediction results of the improved model and four mainstream models on the validation set are shown in Figure 9 , where the red box marked area represents the phenomenon of misidentification, missed identification and insufficient identification of tobacco edge details, i.e. the area is incorrectly identified as measurable tobacco or background. As shown in Figure 9 , compared with the improved model, the segmentation results of the original DeepLabv3+ have missed identification, misidentification and inaccurate edge detail segmentation; UNet can achieve target segmentation, but the misidentification rate increases; PSPNet has discontinuous, discontinuous and misidentification segmentation results; HrNet has insufficient identification of tobacco edge details, and has more misidentification and missed identification. The improved model has higher identification accuracy than the other four models, has lower misidentification and missed identification rate, and can extract the edge information of the tobacco and effectively restore the detailed features of the tobacco.

[0093] 3.3.2 Ablation experiment

[0094] In order to verify the effectiveness of replacing the original DeepLabv3+ model's backbone feature extraction network with the lightweight MobileNetv2, and the influence of adding ECA attention mechanism and replacing ASPP with multi-dilated convolution receptive field cascade fusion structure on segmentation accuracy, four different ablation experiments were designed based on the original model, where the multi-dilated convolution receptive field cascade fusion structure is denoted as MCR, and the experimental results are shown in Table 3.

[0095] Table 3:

[0096]

[0097]

[0098] According to Table 3, the improved DeepLabv3+ has greatly improved in detection accuracy and operation speed. The added modules not only improve the segmentation accuracy of the model, but also effectively shorten the inference time. After replacing the backbone network of DeepLabv3+ with the lightweight MobileNetv2, the detection speed is increased by 72.51f / s due to the reduction of network layers and the use of depth separable convolution instead of ordinary convolution, and the model memory occupation is greatly reduced. After adding the ECA attention mechanism module in the model, the attention to the details of the cut tobacco is improved, and the segmentation accuracy is improved; after adding the multi-dilated convolution receptive field cascade fusion of the ASPP module, due to the addition of a dilated rate of 3 and the fusion of feature information of different layers, MIoU and mPA are improved by 0.94 and 0.57 percentage points respectively, and the detection speed is reduced by 0.13f / s. The improved model has stronger edge detail feature capture ability for cut tobacco, and also performs best in real-time, which can better meet the online detection task of cut tobacco width measurement in modernized cut tobacco workshop.

[0099] 3.3.3 Influence of different brands and cut tobacco width on the performance of cut tobacco recognition model

[0100] In the cut tobacco production process, different production lines usually have different requirements for the quality and width of cut tobacco. In order to ensure the segmentation accuracy of different brands and widths of cut tobacco, and verify the robustness and generalization ability of the improved DeepLabv3+ dense cut tobacco segmentation model, the best model weight trained was used to verify the collected samples of other production lines. 40 cut tobacco images of different brands and widths were extracted for testing, and the test results are shown in Table 4.

[0101] Table 4:

[0102]

[0103] The average pixel accuracy of two different brands and widths of cut tobacco is 83.36% and 83.2% respectively, and the actual segmentation results of the model for different brands and widths of cut tobacco are shown in Table 5. Figure 10 The above experiments prove that the improved model can meet the recognition task of different brands and widths of cut tobacco.

[0104] 4. CONCLUSION

[0105] The improved DeepLabv3+ dense tobacco recognition model is compared with the improved DeepLabv3+, HrNet, PSPNet and UNet models, and the results prove that the recognition effect of the improved DeepLabv3+ model on the tobacco suitable for width measurement is the best, the average intersection over union is 68.3%, the average pixel accuracy is 84.39%, and the recall rate is 80.29%. The real-time performance of the improved model is the best, and the model inference speed is 90.14f / s.

[0106] After using MobileNetv2 as the backbone network, fusing the ECA attention module, and increasing the multi-dilated convolution receptive field cascade fusion structure, the model performs best on the validation set, and the MIoU and mPA of the original model are improved by 2.71 and 1.14 percentage points respectively, and the model inference speed is improved by 68.67f / s, which verifies the effectiveness of the improvement.

[0107] The model still maintains stable segmentation performance in the face of different production lines and complex environments, accurately reflects the edge details and texture features of the tobacco, realizes accurate segmentation, and has strong generalization performance, meeting the requirements of industrial application. The method provides a new idea for the online detection of tobacco width, and provides technical support and guarantee for the cutting quality evaluation, feedback regulation of the previous process, and tobacco processing application scenarios.

[0108] The purpose of the detailed explanation of the above embodiments is only to explain the present application, so as to better understand the present application, but these descriptions cannot be explained as limitations of the present application for any reason, especially, the features described in different embodiments can be arbitrarily combined with each other to form other embodiments, except for the explicit opposite description, these features should be understood as applicable to any embodiment, and not limited to the described embodiments.

Claims

1. A dense tobacco segmentation method based on an improved DeepLabv3+ network model, characterized in that, a tobacco image on a conveying belt is collected and a data set of the tobacco image is constructed, wherein the tobacco in the tobacco image is densely stacked; a dense tobacco segmentation model based on the improved DeepLabv3+ network model is constructed, the dense tobacco segmentation model comprising an encoding module and a decoding module, and the encoding module taking a MobileNetv2 network as a backbone network; the dense tobacco segmentation model is trained through the data set; the trained dense tobacco segmentation model is used to identify the tobacco in a tobacco stack; an efficient channel attention mechanism is introduced at a deep layer feature output by the backbone network and a low layer feature output by the decoder, the efficient channel attention mechanism obtaining a weight distribution by calculating information interaction in each channel; the efficient channel attention mechanism provides a global feature description for features in each channel, convolves a feature vector in each channel using a one-dimensional convolution kernel, normalizes a convolution result using a Sigmoid function, and multiplies the convolution result with the original feature vector element by element to obtain a final weighted feature vector; the specific steps of obtaining the weight distribution by the efficient channel attention mechanism include: performing global average pooling on an input feature map to obtain global channel information; calculating an adaptive convolution kernel size through a fully connected layer; applying the adaptive convolution kernel to one-dimensional convolution operation to obtain an adaptive weight of each channel in the feature map; multiplying the original input feature and the channel weight channel by channel to obtain a feature with channel attention.

2. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 1, wherein, For the MobileNetv2 network, first, a 1×1 convolution is used to map low-dimensional spatial features to high-dimensional spatial features, then a 3×3 depth separable convolution is used to extract features, finally, a 1×1 convolution is used to reduce the features to map the high-dimensional spatial features to low-dimensional spatial features to obtain feature information, and finally, a shortcut connection is used to add the input feature and the output feature.

3. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 1, wherein, In the MobileNetv2 network, the activation function in the linear bottleneck layer is converted into a linear function.

4. The dense tobacco shred segmentation method based on the improved DeepLabv3+ network model according to claim 1, wherein, The ASPP module of the decoder module adopts a multi-dilated convolution receptive field cascade fusion structure.

5. The dense tobacco shred segmentation method based on the improved DeepLabv3+ network model according to claim 4, wherein, The number of channels is adjusted through a 1×1 convolution, and a multi-dilated convolution pyramid is constructed to capture global information of the tobacco image using a multi-scale receptive field.

6. The dense tobacco shred segmentation method based on the improved DeepLabv3+ network model according to claim 5, wherein, The receptive field calculation formula is: wherein, is a receptive field; is an inflation rate; is a convolution kernel size.

7. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 4, wherein, A dilated convolution with a dilated rate of 3 is introduced in the ASPP module.

8. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 4, wherein, Each level of the ASPP module obtains a feature layer from the backbone network and learns by fusing the features obtained through different dilated rates.

9. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 4, wherein, The ASPP module outputs six feature layers and performs a splicing operation on the six feature layers.

10. The dense tobacco shreds segmentation method based on the improved DeepLabv3+ network model according to claim 1, wherein, The data set is divided into a training set and a validation set, the training set is sent to the dense tobacco segmentation model for training, and the trained dense tobacco segmentation model is verified using the validation set.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method according to any one of claims 1 to 10.

12. A computer device comprising a memory and a processor, having stored on the memory a computer program capable of running on the processor, characterized in that, The processor, when executing the computer program, implements the steps of the method according to any one of claims 1 to 10.

Images