Dense cut tobacco segmentation method and device based on improved DeepLabv3 + network model

Abstract

The invention provides a dense cut tobacco segmentation method and device based on an improved DeepLabv3 + network model, and belongs to the technical field of cigarette detection. Collecting tobacco shred images on the conveying belt and constructing a data set of the tobacco shred images, wherein tobacco shreds in the tobacco shred images are densely stacked; a dense cut tobacco segmentation model based on an improved DeepLabv3 + network model is constructed, the dense cut tobacco segmentation model comprises a coding module and a decoding module, and the coding module takes a MobileNetv2 network as a backbone network; training the dense cut tobacco segmentation model through the data set; and identifying tobacco shreds in a tobacco shred pile by using the trained dense tobacco shred segmentation model. According to the method, the dense tobacco shreds are efficiently and accurately segmented under the complex background, so that technical support is provided for online measurement of the width of the tobacco shreds on a tobacco shred production line, and the monitoring efficiency and accuracy of the processing quality of the tobacco shreds are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cigarette detection, and in particular to a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model. Background Art

[0002] Tobacco is one of the important cash crops. To make tobacco into finished cigarettes, it must go through a strict processing process, and shredding is one of the important links. The width of shredded tobacco is the main quality indicator of the shredded production process. However, the current shredded process cannot realize online detection of shredded tobacco width, and most of them are mainly measured manually. During the shredded process, the technicians cannot grasp the shredded quality information in real time, and cannot find problems in the shredded process in time, which can easily cause the shredded quality to deviate from the industry standard and affect the subsequent processing technology. In addition, the difficulty in detecting the shredded tobacco width in a dense state also restricts the development of the shredded process towards intelligence, thereby affecting production efficiency.

[0003] At present, in the research on tobacco width detection, researchers usually take discrete single tobacco as the research object, use digital image processing technology to extract tobacco using the color, shape, texture and other characteristics of tobacco, and then use Hough transform, matrix flipping and other methods to detect tobacco width. At present, there are studies on the measurement of tobacco width in discrete and artificially placed states. However, in complex environments such as densely stacked silk production lines, image processing algorithms are easily affected by occlusion and noise. Moreover, the silk production line belongs to the industrial production application scenario, which has high real-time requirements. Traditional image processing detection cannot be fully applicable to actual production.

[0004] In recent years, with the rapid development of deep learning in various fields, more and more researchers have applied deep learning technology to image recognition research in the agricultural field. Compared with traditional semantic segmentation methods, semantic segmentation methods based on deep learning can learn the feature information of images through neural networks and obtain semantic information at different levels, and have better generalization capabilities. For example, a Mask RCNN-based Densenet121 and U-FPN replace the backbone network and feature pyramid structure, and the network's recognition accuracy for overlapping tobacco is 89.1%. A two-level Unet tobacco debris detection model has an average detection accuracy of 96.6% for debris and an image processing speed of 21ms / sheet.

[0005] However, in the dense stacking state recognition task, tobacco shreds are intertwined and blocked, which blurs the boundary information of tobacco shreds and increases the difficulty of identifying tobacco shreds suitable for measuring width. In addition, most studies pay less attention to the lightweight problem of the model, and the constructed network structure is too complex, resulting in excessive computational complexity, which is not conducive to deployment in the tobacco production line with limited computing power.

[0006] In view of this, the inventor, based on many years of production design experience in this field and related fields, has designed a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model after repeated experiments, in order to solve the problems existing in the prior art. Summary of the invention

[0007] The object of the present invention is to provide a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, which can accurately measure the width of tobacco online.

[0008] To achieve the above-mentioned object of the invention, the present invention proposes a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, wherein tobacco images on a conveyor belt are collected and a data set of tobacco images is constructed, and the tobacco in the tobacco images is densely stacked;

[0009] A dense tobacco segmentation model based on the improved DeepLabv3+ network model is constructed. The dense tobacco segmentation model includes an encoding module and a decoding module. The encoding module uses the MobileNetv2 network as the backbone network.

[0010] The dense tobacco segmentation model is trained using the dataset;

[0011] The trained dense tobacco segmentation model is used to identify tobacco in the tobacco pile.

[0012] The present invention also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the steps of the above method when executed by a processor.

[0013] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, wherein the steps of the above method are implemented when the processor executes the computer program.

[0014] Compared with the prior art, the present invention has the following characteristics and advantages:

[0015] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model proposed in the present invention solve the problems of complex image background, easy occlusion and difficult model deployment. By replacing the feature extraction network in the original network with a lightweight MobileNetv2 backbone network, the network parameters are effectively reduced, thereby reducing the complexity of the model. Further optimization solves the problems of spatial detail loss and insufficient feature extraction, and improves the model's ability to extract tobacco features. By introducing an efficient channel attention mechanism (ECA), the model's attention to tobacco boundary features is enhanced, so that the model can more accurately capture the edge information of tobacco. In addition, by modifying the parameters of the spatial pyramid pooling module, a multi-expansion convolution receptive field cascade fusion structure is proposed to replace the original parallel structure, which not only increases the model's sharing of tobacco features, but also improves the accuracy and real-time performance of the model. These improvements enable the model to achieve more accurate segmentation when processing dense tobacco images under complex backgrounds, providing strong technical support for online width measurement of the tobacco production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings described herein are only for explanation purposes and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the various components in the figures are only schematic, used to help understand the present invention, and are not specifically limited to the shapes and proportional dimensions of the various components of the present invention. Those skilled in the art can select various possible shapes and proportional dimensions to implement the present invention according to the teachings of the present invention.

[0017] Figure 1 It is a schematic flow chart of a dense tobacco segmentation method based on an improved DeepLabv3+ network model of the present invention;

[0018] Figure 2 A schematic diagram of the specific steps of obtaining weight distribution by calculating information interaction in each channel by the efficient channel attention mechanism of the present invention;

[0019] Figure 3a This is the original image of the dense tobacco recognition by DeepLabv3+ of the present invention;

[0020] Figure 3b This is a label image of dense tobacco recognition by DeepLabv3+ of the present invention;

[0021] Figure 3c This is the segmentation map of dense tobacco recognition by DeepLabv3+ of the present invention;

[0022] Figure 4 This is the improved DeepLabv3+ model network structure diagram of the present invention;

[0023] Figure 5 This is the structure diagram of the ECA attention mechanism of the present invention;

[0024] Figure 6 It is a schematic diagram of the cascade fusion of multiple dilated convolution receptive fields of the present invention;

[0025] Figure 7a This is an example of a tobacco cut image of Septwolves White of the present invention;

[0026] Figure 7b This is an example of a tobacco cut image of Septwolves gray of the present invention;

[0027] Figure 7c This is an example of a tobacco image of the sharp edge of Septwolves of the present invention;

[0028] Figure 8 This is a graph showing the loss function of the improved model training process of the present invention;

[0029] Fig. 9 It is a schematic diagram of the segmentation results of tobacco cut image by different models of the present invention;

[0030] Fig.10 It is a schematic diagram of the tobacco segmentation results under different brands of the present invention. DETAILED DESCRIPTION

[0031] The details of the present invention can be more clearly understood with reference to the accompanying drawings and the description of the specific embodiments of the present invention. However, the specific embodiments of the present invention described herein are only used for the purpose of explaining the present invention and cannot be understood as limiting the present invention in any way. Under the guidance of the present invention, technicians can conceive of any possible variations based on the present invention, which should all be considered to belong to the scope of the present invention.

[0032] The present invention proposes a dense shredded tobacco segmentation method and device based on an improved DeepLabv3+ network model, wherein shredded tobacco images on a conveyor belt are collected and a dataset of shredded tobacco images is constructed, wherein shredded tobacco in the shredded tobacco images are densely stacked;

[0033] A dense tobacco segmentation model based on the improved DeepLabv3+ network model is constructed. The dense tobacco segmentation model includes an encoding module and a decoding module. The encoding module uses the MobileNetv2 network as the backbone network.

[0034] The dense tobacco segmentation model is trained using the dataset;

[0035] The trained dense tobacco segmentation model is used to identify tobacco in the tobacco pile.

[0036] The dense tobacco shreds segmentation method and device proposed in the present invention based on the improved DeepLabv3+ network model, by collecting and constructing an image dataset of densely stacked tobacco shreds, and using a segmentation model composed of an encoding module with a MobileNetv2 network as the backbone and a decoding module, are trained and recognized for tobacco shreds images, thereby achieving efficient and accurate segmentation of dense tobacco shreds in complex backgrounds, thereby providing technical support for online measurement of tobacco shred width on a tobacco production line and improving the monitoring efficiency and accuracy of tobacco processing quality.

[0037] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model proposed in the present invention are obtained from the tobacco image data collected from the tobacco making workshop of Longyan Tobacco Industry Co., Ltd. in Fujian Province in November 2023. The shooting equipment is the MARS-1231-46G5C-P Mars series camera produced by Daheng Company. The image data is transmitted through the acquisition card and saved in the computer, with automatic white balance and exposure; the image resolution is 4096 pixels × 3000 pixels, stored in JPG format, and the acquisition frame rate is 5Hz. During data collection, the digital camera is fixed on the bracket, and the shooting angle is parallel to the direction of the conveyor belt. According to the research objectives, this article only collects data on the tobacco just cut by the cutter, which is more in line with actual production applications. Figure 7a , Figure 7b and Figure 7c As shown in the figure, a total of 483 original images were collected from the Longyan District 1 production line for model training, and an additional 100 images of two different brands of tobacco, Soft Gray and Sharp, of Septwolves were collected to verify the generalization performance of the model.

[0038] The present invention proposes a dense tobacco segmentation method and device based on the improved DeepLabv3+ network model. In the tobacco making workshop environment, the tobacco suitable for width measurement is intertwined with the drawing and running sheets, and the image background color is very similar. This paper combines the characteristics of the tobacco dataset and takes the DeepLabv3+ model as the basis. Through a large number of experimental analyses, the model parameters are continuously optimized to improve the detection accuracy and real-time performance of the model.

[0039] The present invention proposes a dense tobacco segmentation method and device based on an improved DeepLabv3+ network model, and DeepLabv3+ is a codec architecture for pixel-by-pixel semantic segmentation. DeepLabv3+ improves its predecessor DeepLabv3, retains the original DeepLabv3 structure as an encoding module, and adds a decoding module to restore the details and dimensional information of the target boundary to improve the segmentation accuracy of the model. In the encoding module, the DeepLabv3+ network uses the entire image as the input of the Xception backbone network to obtain the features of the target and pass it into the decoder. At the same time, in order to reduce the loss of feature information and improve the model's recognition ability for objects of different scales, DeepLabv3+ uses an atrous spatial pyramid pooling (ASPP) feature fusion module, in which 1×1 convolution, pooling (Image pooling), and convolutions with different expansion rates are used to extract feature information, where the expansion rates are 6, 12, and 18, respectively, to capture multi-scale information and obtain higher semantic information. Finally, the deep features fused by ASPP are input into the 1×1 convolution kernel for channel adjustment and input into the decoding module. In the decoding module, a 4-fold bilinear upsampling operation (Bilinearinterpolation) is used to deconvolve the feature map from the output of the encoder. The result is cascaded with the corresponding low-level features from the backbone network. After concatenation, the feature map is gradually restored to the original spatial dimension and refined through 3×3 convolution and upsampling, thereby achieving semantic segmentation of the target area.

[0040] The dense tobacco segmentation method and device based on the improved DeepLabv3+ network model proposed in this invention adopts the original DeepLabv3+ model to conduct preliminary experiments on the tobacco dataset of the tobacco processing workshop. The recognition results are as follows: Figure 3a , Figure 3b and Figure 3cAs shown. In the case of occlusion and overlap between the running film, the strip and the tobacco, the model fails to accurately restore the detailed features such as the boundaries of the tobacco. In addition, the backbone network structure of the model itself is complex and has a large number of parameters. It requires a large amount of video memory and computing time, which limits its application in embedded devices. This paper improves it as follows: ① Use a lightweight MobileNetv2 network to replace the original Xception network with many parameters, reduce the calculation of model parameters, improve the network segmentation speed, and better suit it for embedded devices. ② Integrate the ECA attention module to increase the feature attention weight of tobacco, so that the network focuses more on the key features of tobacco, thereby improving segmentation accuracy. ③ In view of the problem that the original model has insufficient ability to restore the edge details of tobacco, a multi-expansion convolution receptive field cascade fusion structure is proposed to improve the ASPP module, increase the sharing of tobacco features between ASPP levels, and convolutions with different expansion rates can depend on each other, increasing the range of the receptive field. The improved model structure is shown in Figure 4 shown.

[0041] In an optional embodiment of the present invention, for the MobileNetv2 network, first use 1×1 convolution to map low-dimensional space features to high-dimensional space, then use 3×3 depth-separable convolution to extract features, and finally use 1×1 convolution to reduce the features to map high-dimensional space to low-dimensional space to obtain feature information, and finally use shortcut connection to add input features to output features. Adding input features to output features through shortcut connection not only reduces information loss during training, but also helps direct propagation of gradients, thereby accelerating the convergence of the model.

[0042] Specifically, in the tobacco workshop environment, the tobacco suitable for width measurement is intertwined with the drawing and running strips, and the image background color is very similar. This paper combines the characteristics of the tobacco dataset and takes the DeepLabv3+ model as the basis. Through a large number of experimental analyses, the model parameters are continuously optimized to improve the detection accuracy and real-time performance of the model. DeepLabv3+ is a codec architecture for pixel-by-pixel semantic segmentation. DeepLabv3+ improves on its predecessor DeepLabv3, retaining the original DeepLabv3 structure as the encoding module, and adding a decoding module to restore the details and dimensional information of the target boundary to improve the segmentation accuracy of the model. In the encoding module, the DeepLabv3+ network uses the entire image as the input of the Xception backbone network to obtain the characteristics of the target and pass it into the decoder. At the same time, in order to reduce the loss of feature information and improve the model's ability to recognize objects of different scales, DeepLabv3+ uses an atrous spatial pyramid pooling (ASPP) feature fusion module, in which 1×1 convolution, image pooling, and convolution with different dilation rates are used to extract feature information, with dilation rates of 6, 12, and 18, respectively, to capture multi-scale information and obtain higher semantic information. Finally, the deep features fused by ASPP are input into the 1×1 convolution kernel for channel adjustment and input into the decoding module. In the decoding module, a 4-fold bilinear upsampling operation (Bilinearinterpolation) is used to deconvolve the feature map from the output of the encoder. The result is cascaded with the corresponding low-level features from the backbone network. After concatenation, the feature map is gradually restored to the original spatial dimension and refined through 3×3 convolution and upsampling, thereby achieving semantic segmentation of the target area.

[0043] In an optional embodiment of the present invention, in the MobileNetv2 network, the activation function in the linear bottleneck layer is converted into a linear function. By adopting a linear activation function in the linear bottleneck layer of the MobileNetv2 network, efficient retention and transmission of feature information can be achieved. Reducing the information loss that may be caused by nonlinear activation functions enables the network to more completely capture and utilize the features of the input data. Since the feature information is more completely retained, the model can more accurately identify and segment the target object when processing the image, especially in the task of dense tobacco segmentation under complex backgrounds. This improvement significantly improves the accuracy of segmentation and the generalization ability of the model.

[0044] Specifically, the original DeepLabv3+ semantic segmentation model uses Xception as the backbone network. The Xception model has the characteristics of deep structure and large number of parameters, which makes the inference time long and the running speed slow on embedded devices with limited resources in practical applications. In order to improve the inference speed and computational efficiency, the lightweight MobileNetv2 is used as the backbone feature extraction network. MobileNetv2 is a lightweight network model proposed by Google. MobileNetV2 is an improved version based on MobileNetV1. MobileNetV2 introduces the inverted residual block and linear bottleneck layer based on the use of depthwise separable convolution, which reduces the number of model parameters and makes the network converge faster. For this network, first use 1×1 convolution to map low-dimensional space features to high-dimensional space, then use 3×3 depthwise separable convolution to extract features, and finally use 1×1 convolution to reduce the features to map high-dimensional space to low-dimensional space to obtain feature information. Finally, use shortcut to add input features to output features, thereby improving the network's feature extraction ability and the effective transmission of multi-layer feature information. And the activation function in the linear bottleneck layer is transformed into a linear function to prevent the nonlinear layer from destroying too much feature information. The structural parameters of MobileNetV2 used in this experiment are shown in Table 1.

[0045] Table 1:

[0046]

[0047]

[0048] In an optional embodiment of the present invention, an efficient channel attention mechanism is introduced at the deep features output by the backbone network and the low-level features output by the decoder. The efficient channel attention mechanism obtains the weight distribution by calculating the information interaction in each channel. The introduction of the efficient channel attention mechanism significantly improves the model's ability to recognize and segment tobacco features. Through the ECA mechanism, the model can automatically learn and emphasize those feature channels that are most critical to the tobacco segmentation task, while suppressing those less important features, thereby improving the feature expression ability and the generalization ability of the model. This mechanism is particularly suitable for processing dense tobacco images under complex backgrounds, and can effectively improve the model's recognition accuracy for tobacco boundaries.

[0049] Specifically, in the ECA attention mechanism module, during the fine segmentation of tobacco and run-off, drawn, and curled tobacco, due to the complexity of the dense background environment, tobacco suitable for width measurement has a certain similarity in color with run-off and drawn, and is easily blocked. Therefore, an efficient channel attention mechanism is introduced at the deep features output by the backbone network and the low-level features output by the decoder to suppress the interference of non-important targets on the network under complex backgrounds, while increasing the network's attention weight on tobacco boundary features.

[0050] In an optional example of this implementation, the efficient channel attention mechanism provides a global feature description for the features in each channel, uses a one-dimensional convolution kernel to convolve the feature vector in each channel, and the convolution result is normalized using the Sigmoid function and multiplied element-wise with the original feature vector to obtain the final weighted feature vector. Since the computational complexity of the ECA mechanism is low, it will not significantly increase the computational burden of the model, so that the model can achieve higher segmentation accuracy while maintaining high efficiency.

[0051] Specifically, it is different from the traditional SE (squeeze and excitation module) attention mechanism, such as Figure 5 As shown in the figure, the ECA module avoids dimensionality reduction operations, is faster, and has lower computational complexity. The ECA structure obtains weight distribution by calculating the information interaction in each channel, so that the model pays more attention to key channel features. The ECA structure provides a global feature description for the features in each channel, and then uses a one-dimensional convolution kernel to convolve the feature vector in each channel. The convolution result is normalized using the Sigmoid function and multiplied element-wise with the original feature vector to obtain the final weighted feature vector.

[0052] In an optional example of this implementation, the specific steps of the efficient channel attention mechanism obtaining the weight distribution by calculating the information interaction within each channel include: performing global average pooling on the input feature map and obtaining global channel information; calculating an adaptive convolution kernel size through a fully connected layer; applying the adaptive convolution kernel to a one-dimensional convolution operation to obtain an adaptive weight for each channel in the feature map; and multiplying the original input feature by the channel weight channel by channel to obtain a feature with channel attention.

[0053] First, the input feature map is processed by global average pooling, that is, the input feature map (H, W, C) is converted to (1, 1, C) to obtain global channel information.

[0054]

[0055] Where: U c Represents the input feature vector Zc Represents the output feature vector.

[0056] Secondly, an adaptive convolution kernel size is calculated through a fully connected layer for subsequent convolution operations.

[0057]

[0058] Where: b represents the bias of the adaptive convolution kernel, γ represents the scaling global average pooling output coefficient, and k represents the convolution kernel size. Represents the mapping function |·| odd Represents the closest odd-numbered object

[0059] Then, the adaptive convolution kernel is applied to the one-dimensional convolution operation to obtain the adaptive weight of each channel in the feature map. A one-dimensional convolution operation with a convolution kernel size of k is performed, and the weight w of each channel is obtained through the σ activation function.

[0060] The calculation formula for 1D convolution is

[0061] w=σ(Conv K (y))(3)

[0062] Where: y is the result after average pooling Conv k is a one-dimensional convolution operation with a convolution kernel size of k

[0063] Finally, the original input features are multiplied channel by channel weights to obtain features with channel attention, so as to enhance the learning and extraction of information of interest in the target area.

[0064] In an optional embodiment of the present invention, the ASPP module of the decoder module adopts a multi-expanded convolution receptive field cascade fusion structure. By processing hole convolutions with different expansion rates in parallel, rich multi-scale context information is captured, and the network's recognition ability for small objects and large backgrounds is enhanced.

[0065] Specifically, the multi-expanded convolution receptive field cascade fusion structure, such as Figure 6 As shown in Figure 2, the multi-expanded convolution receptive field cascade fusion structure uses a multi-layer cascade fusion approach to replace the original parallel structure, thereby enhancing the fusion of feature information at different scales.

[0066] In an optional example of this implementation, the number of channels is adjusted by 1×1 convolution, and a multi-expanded convolutional pyramid is constructed to capture the global information of the tobacco image using a multi-scale receptive field. The use of a multi-expanded convolutional pyramid enables the model to take into account both the local details and the global context of the image, which is particularly important for accurate segmentation in complex backgrounds.

[0067] Specifically, in the ASPP structure, dilated convolutions with different dilation rates and global average pooling operations are used to obtain semantic information of feature maps at different levels. However, the parallel structure used in the ASPP structure cannot obtain the sharing of tobacco feature information between different levels, and the original dilation rate interval is set too large to extract tobacco edge feature information. In order to overcome the inadequacy of the ASPP structure in extracting tobacco feature information, the dilation rate in the ASPP module is adjusted, and a small number of parameters are added to enhance the extraction of tobacco feature information.

[0068] In an optional example, the receptive field calculation formula is:

[0069] R f =(Rate-1)×(k-1)+k

[0070] Among them, R f is the receptive field; rate is the dilation rate; k is the convolution kernel size. The number of channels is adjusted to 256 through 1×1 convolution, and the feature information of the receptive fields between different levels is obtained by controlling the dilation rate, stride and padding parameters, and a multi-dilation convolution pyramid is constructed to better capture the global information of the tobacco image by using the range of receptive fields of different scales. By accurately calculating and controlling the receptive field, we can ensure that the model can effectively extract features at different levels, thereby improving the model's ability to recognize small objects and large backgrounds.

[0071] In an optional example of this embodiment, a dilated convolution with a dilation rate of 3 is introduced in the ASPP module. This ensures that the feature information near the edge of the tobacco is given important attention, reduces the loss of global information, and makes the network segmentation more accurate. By introducing a dilated convolution with a dilation rate of 3 in the ASPP module, the accuracy of tobacco image segmentation is significantly improved, especially in the feature extraction of the edge area. The improvement enables the network to pay more attention to the feature information near the edge of the tobacco, reducing the loss of global information during the segmentation process.

[0072] In an optional example of this implementation, each level of the ASPP module obtains a feature layer from the backbone network and fuses it with the features obtained by convolution at different expansion rates at this level. Each level of the ASPP module independently extracts features through convolution at different expansion rates and then fuses them, so that the model can take into account both the local details and the global context of the image. This multi-scale feature fusion method is particularly suitable for dealing with occlusion and stacking problems in dense tobacco images, allowing the model to more accurately identify and segment tobacco, even when there is overlap and occlusion between tobacco, it can maintain high accuracy.

[0073] Specifically, a multi-expansion convolution cascade fusion structure is used to reduce the loss of global semantic information. The output of each layer is adjusted to 320 channels through a 1×1 convolution operation as the input feature of the next layer branch, and a multi-layer cascade fusion method is used to increase the degree of sharing of tobacco features obtained by dilated convolutions with different expansion rates. Through the MobileNetv2 feature extraction network, each level in the ASPP module obtains a feature layer. Then, these features are fused and learned with the features obtained by convolutions with different expansion rates in each layer to solve the problem of insufficient ability to capture global feature extraction, so as to ensure that tobacco feature information can be shared between layers, and convolutions with different expansion rates can rely on each other to increase the range of the receptive field.

[0074] In an optional example of this implementation, the ASPP module outputs 6 feature layers and performs a splicing operation on the 6 feature layers. The 6 feature layers obtained after the feature layer fusion are spliced, that is, the features after 5 convolutions are stacked with the features of the global average pooling, so as to use feature layers of different scales to better capture the global semantic feature information of the tobacco image. By outputting and splicing 6 feature layers of different scales, the ASPP module enables the model to comprehensively consider feature information from different levels, which not only enhances the model's ability to recognize the details of the tobacco image, but also improves the understanding of the global context.

[0075] In an optional embodiment of the present invention, the data set is divided into a training set and a validation set, the training set is sent to the dense tobacco segmentation model for training, and the trained dense tobacco segmentation model is verified using the validation set. The practice of dividing the training set and the validation set not only improves the stability and convergence speed of the model during the training process, but also effectively prevents the overfitting of the model through the real-time performance feedback of the validation set. The training and validation process ensures that the model can achieve higher segmentation accuracy and better generalization ability when processing dense tobacco images under complex backgrounds.

[0076] Specifically, due to the high resolution of the original images obtained by industrial cameras, this will cause too much video memory to be occupied when training the model, reducing the computing speed. In order to adapt to the performance of the computer and speed up the convergence of the model loss, the size of the 483 original images was proportionally scaled to 806 pixels × 756 pixels, and the objects contained in the image were divided into two categories: tobacco suitable for width measurement and background. The run film, the strip, the stem label, and the tobacco with insignificant features were marked as the background class. The Labelme annotation tool was used to generate the annotation mask of the tobacco. The obtained label file was saved in .json format and converted into a grayscale image for training the model. In order to avoid the problem of low model robustness caused by insufficient number of collected samples, this paper performs data enhancement by flipping up and down, translating, Gaussian noise and color conversion. There are 1170 enhanced images in total, which are divided into training set and validation set in a ratio of 8:2, including 936 training sets and 234 validation sets. In order to verify the segmentation effect of the model on tobacco in a densely stacked state, the robustness of the model is further tested under different brands of tobacco and widths.

[0077] The present invention also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the steps of the above method when executed by a processor.

[0078] The present invention also proposes a computer device, comprising a memory and a processor, wherein a computer program that can be run on the processor is stored in the memory, and wherein the processor implements the steps of the above method when executing the computer program.

[0079] The specific implementation process of the present invention is now described in detail with reference to the embodiments:

[0080] In order to verify the robustness and accuracy of the dense tobacco recognition algorithm based on the improved DeepLabv3+, a dense tobacco segmentation experiment was carried out.

[0081] 3.1 Experimental environment

[0082] The host processor CPU of this experiment is i9-12900KF, with a main frequency of 2GHz and a running memory of 64GB. The GPU is NVIDIAGeForce RTX 4090 graphics card with a memory of 24GB, and the operating system is Windows 10. Anaconda3 is used to configure the deep learning virtual environment, using torch version 1.7.1, Cuda version 11.3, and Python version 3.7. The pre-trained weights provided by MobileNetv2 are used in model training. The SGD optimizer is selected to update the parameters through multiple experiments. The initial learning rate is set to 0.007, the batch size is set to 8, the weight decay factor is set to 0.0001, and the image resolution of the input network is 806 pixels × 756 pixels. Dice Loss is used as the loss function, and the weight file is saved every 10 iterations of training. The number of iterations is set to 300 times to ensure that the model is fully converged.

[0083] 3.2 Evaluation indicators

[0084] In order to objectively evaluate the segmentation effect of the segmentation model on dense tobacco images, mean pixel accuracy (mPA), recall, mean intersection over union (MIoU), recall, precision, detection speed (Frames per second, FPS) and memory usage are selected as performance evaluation indicators of different semantic segmentation models. The feasibility of the model in practical applications is verified by detecting speed and memory usage.

[0085] 3.3 Model segmentation results

[0086] 3.3.1 Comparison of segmentation performance of different models

[0087] The improved DeepLabv3+ model was trained with several popular semantic segmentation models, including UNet, PSPNet, and HrNet, under the same validation set and configuration parameters. The training loss curve of the improved model changes as shown in the figure below. Figure 8 As shown in Table 2, the loss value shows a decreasing trend with the increase of training rounds, and gradually converges and stabilizes. The improved model compares the mPA, MIoU, detection speed, memory usage and parameter amount of different models, as shown in Table 2.

[0088] Table 2:

[0089]

[0090]

[0091] As shown in Table 2, the proposed model performs better in terms of real-time and accuracy. The average pixel accuracy, average intersection-over-union ratio, recall rate and accuracy are the best among the five models. Among them, mPA is 84.39%, which is 1.14 percentage points higher than the original model, MIoU is 68.3%, which is 2.71 percentage points higher than the original model, Recall is 80.29%, which is 2.04 percentage points higher than the original model, and the accuracy is increased from 80.5% to 81.89%, which is 1.39 percentage points higher. This shows that the proposed model can more accurately identify the edge features of tobacco. In addition, the improved model has also been significantly improved in terms of detection speed, with a memory usage of 22.52MB. Compared with other mainstream semantic segmentation models, it is lighter and has a shorter inference time. It reaches 90.14f / s on the verification set, that is, the processing time of a single image is only 11.08ms. It ensures rapid deployment and application in industrial scenarios with limited resources, and can provide support for the rapid and accurate measurement of tobacco width.

[0092] The prediction results of the improved model and the four mainstream models on the validation set are as follows: Fig. 9 As shown in the figure, the area marked by the red frame indicates the phenomenon of misidentification, missed identification and insufficient identification of tobacco edge details, that is, the area is mistakenly identified as measurable tobacco or background. Fig. 9 As shown in the figure, compared with the improved model, the original DeepLabv3+ segmentation results have omissions, misidentifications, and inaccurate edge detail segmentation; UNet can achieve target segmentation, but the misidentification rate increases; PSPNet has intermittent, discontinuous, and misidentified segmentation results; HrNet is not good at recognizing tobacco edge details, and there are many misidentifications and omissions. The improved model has better recognition accuracy than the other four models, with lower omissions and misidentification rates, and can achieve the extraction of tobacco edge information, while effectively restoring the detailed features of tobacco.

[0093] 3.3.2 Ablation Experiment

[0094] In order to verify the effectiveness of replacing the backbone feature extraction network of the original DeepLabv3+ model with the lightweight MobileNetv2, and to verify the impact of adding the ECA attention mechanism and replacing ASPP with the multi-expanded convolution receptive field cascade fusion structure on the segmentation accuracy, four different ablation experiments were designed based on the original model, in which the multi-expanded convolution receptive field cascade fusion structure was denoted as MCR. The experimental results are shown in Table 3.

[0095] Table 3:

[0096]

[0097]

[0098] According to Table 3, the improved DeepLabv3+ has greatly improved the detection accuracy and operation speed. The added modules not only improve the segmentation accuracy of the model, but also effectively shorten the inference time. After replacing the backbone network of DeepLabv3+ with the lightweight MobileNetv2, the detection speed increased by 72.51f / s due to the reduction of the number of network layers and the use of depthwise separable convolution instead of ordinary convolution, and the model memory usage was greatly reduced. After adding the ECA attention mechanism module to the model, the model's attention to the detailed features of tobacco was improved, and the segmentation accuracy was improved; after adding the multi-expanded convolution receptive field cascade fusion of the ASPP module, due to the addition of a dilated convolution with a dilation rate of 3 and the fusion of feature information from different layers, MIoU and mPA increased by 0.94 and 0.57 percentage points respectively, and the detection speed decreased by 0.13f / s. The improved model has a stronger ability to capture the edge detail features of tobacco, and also performs best in real-time performance, which can better meet the online detection task of tobacco width measurement in the dense state of modern silk workshops.

[0099] 3.3.3 Effects of different brands and cut widths on the performance of tobacco recognition models

[0100] In the tobacco production process, different production lines usually have different requirements for the quality and width of the cut tobacco. In order to ensure the segmentation accuracy of tobacco of different brands and widths, and to verify the robustness and generalization ability of the improved DeepLabv3+ dense tobacco segmentation model, the best model weights trained were used to verify the samples collected from other production lines, and 40 tobacco images of different brands and widths were extracted for testing. The test results are shown in Table 4.

[0101] Table 4:

[0102]

[0103] The average pixel accuracy of the two different brands and widths of tobacco is 83.36% and 83.2% respectively. The actual segmentation results of the model for different brands and widths are shown in the figure below. Fig.10 The above experiments all prove that the improved model can meet the task of identifying tobacco of different brands and widths.

[0104] 4 Conclusion

[0105] An improved DeepLabv3+ dense tobacco recognition model is proposed, and compared with the DeepLabv3+, HrNet, PSPNet, and UNet models before the improvement. The results show that on the same validation set, the proposed model has the best recognition effect on tobacco suitable for width measurement, with an average intersection-over-union ratio of 68.3%, an average pixel accuracy of 84.39%, and a recall rate of 80.29%. The improved model has the best real-time performance, with a model inference speed of 90.14f / s.

[0106] After using MobileNetv2 as the backbone network, integrating the ECA attention module, and adding a multi-expanded convolution receptive field cascade fusion structure, the model performed best on the validation set. Compared with the original model, MIoU and mPA increased by 2.71 and 1.14 percentage points respectively, and the model inference speed increased by 68.67f / s, verifying the effectiveness of the improvement.

[0107] The model has been tested for different brands and widths of tobacco, and it has maintained stable segmentation performance in different production lines and complex environments, accurately reflecting the edge details and texture features of tobacco, and achieving precise segmentation. The model has shown strong generalization performance and has met the requirements of industrial applications. This method provides a new idea for the online detection of tobacco width, and provides technical support and guarantee for application scenarios such as cutting quality evaluation, feedback adjustment of previous processes, and silk processing.

[0108] The detailed explanation of the above-mentioned embodiments is only intended to explain the present invention so as to facilitate a better understanding of the present invention. However, these descriptions cannot be interpreted as limitations on the present invention for any reason. In particular, the various features described in different embodiments may also be arbitrarily combined with each other to form other embodiments. Unless there is a clear description to the contrary, these features should be understood to be applicable to any embodiment and are not limited to the described embodiments.

Claims

1. A dense tobacco segmentation method based on an improved DeepLabv3+ network model, characterized in that: Collecting tobacco shreds images on a conveyor belt and constructing a dataset of tobacco shreds images, wherein tobacco shreds in the tobacco shreds images are densely stacked; Constructing a dense tobacco segmentation model based on an improved DeepLabv3+ network model, wherein the dense tobacco segmentation model includes an encoding module and a decoding module, wherein the encoding module uses a MobileNetv2 network as a backbone network; Training the dense tobacco segmentation model using the data set; The trained dense tobacco segmentation model is used to identify tobacco in the tobacco pile.

2. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 1, characterized in that: For the MobileNetv2 network, a 1×1 convolution is first used to map the low-dimensional space features to a high-dimensional space, and then a 3×3 depth-separable convolution is used to extract features. Finally, a 1×1 convolution is used to reduce the features to map the high-dimensional space to a low-dimensional space to obtain feature information, and finally a shortcut connection is used to add the input features to the output features.

3. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 1, characterized in that: In the MobileNetv2 network, the activation function in the linear bottleneck layer is transformed into a linear function.

4. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 1, characterized in that: An efficient channel attention mechanism is introduced between the deep features of the backbone network output and the low-level features of the decoder output. The efficient channel attention mechanism obtains the weight distribution by calculating the information interaction within each channel.

5. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 4, characterized in that: The efficient channel attention mechanism provides a global feature description for the features in each channel, uses a one-dimensional convolution kernel to convolve the feature vector in each channel, normalizes the convolution result using the Sigmoid function, and multiplies the result element-wise with the original feature vector to obtain the final weighted feature vector.

6. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 4, characterized in that: The specific steps of the efficient channel attention mechanism to obtain the weight distribution by calculating the information interaction in each channel include: Perform global average pooling on the input feature map and obtain global channel information; An adaptive convolution kernel size is calculated through a fully connected layer; Apply the adaptive convolution kernel to the one-dimensional convolution operation to obtain the adaptive weight of each channel in the feature map; The original input features are multiplied channel by channel weights to obtain features with channel attention.

7. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 1, characterized in that: The ASPP module of the decoder module adopts a multi-expanded convolution receptive field cascade fusion structure.

8. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 6, characterized in that: The number of channels is adjusted through 1×1 convolution, and a multi-expanded convolution pyramid is constructed to capture the global information of the tobacco image using a multi-scale receptive field.

9. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 8, characterized in that: The receptive field calculation formula is: R f =(Rate-1)×(k-1)+k Among them, R f is the receptive field; rate is the expansion rate; k is the convolution kernel size.

10. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 6, characterized in that: A dilated convolution with a dilation rate of 3 is introduced in the ASPP module.

11. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 6, characterized in that: Each level of the ASPP module obtains a feature layer from the backbone network and performs fusion learning with the features obtained by convolution at different expansion rates at the level.

12. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 6, characterized in that: The ASPP module outputs 6 feature layers and performs a splicing operation on the 6 feature layers.

13. The dense tobacco segmentation method based on the improved DeepLabv3+ network model according to claim 1, characterized in that: The data set is divided into a training set and a validation set, the training set is transmitted to the dense tobacco segmentation model for training, and the validation set is used to validate the trained dense tobacco segmentation model.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

15. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

Images