A tobacco safety foreign matter identification method based on an image segmentation algorithm
By combining the ConvNeXt model with the improved Mask2Former model, the problem of missed detection and misjudgment of tiny foreign objects in cigarette storage was solved, achieving high-precision foreign object identification and automated management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG QINGDAO TOBACCO
- Filing Date
- 2026-04-17
- Publication Date
- 2026-05-29
AI Technical Summary
Existing image segmentation algorithms struggle to effectively distinguish between small and varied foreign objects in cigarette storage foreign object detection, resulting in high false negative and false positive rates. Furthermore, they lack the ability to characterize features of complex backgrounds, affecting the reliability and traceability of detection results.
The ConvNeXt model is used for hierarchical feature fusion, and the Mask2Former model is improved by introducing a scale-aware query mechanism and a local texture attention branch. By combining multi-scale texture details with high-level semantic information, small targets are captured and accurate instance segmentation masks are generated.
It significantly improves the recognition accuracy of tiny foreign objects, reduces the false judgment rate, achieves pixel-level accurate detection and classification, provides reliable recognition basis, and supports automated removal operations.
Smart Images
Figure CN122116000A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning, and in particular to a method for identifying foreign objects in tobacco safety based on an image segmentation algorithm. Background Technology
[0002] Deep learning technology based on image segmentation algorithms has been widely applied in recent years in fields such as industrial quality inspection, medical imaging, and autonomous driving due to its superior accuracy in target recognition and localization, becoming a key development direction for realizing intelligent machine vision. However, in the actual foreign object detection scenario of cigarette storage, the detection task faces many challenges, such as small target size, different shapes, complex background textures, and high real-time requirements. The deployment effect of existing image segmentation algorithms is still constrained by many factors.
[0003] Currently, most warehouse foreign object identification methods rely on traditional convolutional neural networks or object detection algorithms, which struggle to effectively distinguish foreign objects with similar colors to the background but minimal texture differences, resulting in insufficient feature representation capabilities for complex targets. Some systems employ only single semantic segmentation or instance segmentation models, neglecting the necessity for refined pixel-level segmentation and individual differentiation of tiny targets on high-speed production lines, thus limiting the model's accuracy in identifying dense, small foreign objects. Furthermore, the model's decision-making process lacks a dedicated mechanism for considering the local texture of the target, making it difficult to provide reliable identification information to operators or quality management systems, impacting the credibility and traceability of the detection results.
[0004] Furthermore, when dealing with small foreign objects, the query mechanism of existing mainstream instance segmentation models is often more sensitive to large-scale targets, making it difficult to effectively capture and generate accurate masks for small targets. When fusing multi-scale features, their decoders fail to make full use of local texture information to guide decision-making, resulting in a large number of false positives and false negatives for foreign objects with similar textures but different materials. This makes it difficult to meet the stringent requirements of high accuracy and low false alarm rate in cigarette storage environments, seriously affecting the practical value and stability of the model in real storage environments.
[0005] Therefore, how to provide a method for identifying foreign objects in tobacco safety based on image segmentation algorithms is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose a tobacco safety foreign object identification method based on image segmentation algorithms. This invention constructs an intelligent tobacco foreign object identification process that includes standardized image preprocessing, hierarchical feature fusion using the ConvNeXt model, accurate segmentation using an improved Mask2Former model, instance masking post-processing, and traceable identification results. This achieves pixel-level accurate detection and classification of minute foreign objects in complex commercial contexts. This invention effectively fuses multi-scale texture details and high-level semantic information using the ConvNeXt model and innovatively improves the Mask2Former model by introducing a scale-aware query mechanism to capture minute targets. Simultaneously, it utilizes a local texture attention branching mechanism to guide decoder decisions, thereby significantly improving the identification accuracy for minute, low-contrast, and texture-similar foreign objects. This effectively solves the core problems of existing methods, such as high false negative rates for minute targets, insensitivity to texture features, and inaccurate segmentation masks.
[0007] A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to an embodiment of the present invention includes the following steps: S1. Acquire images of the tobacco to be detected to obtain the original tobacco image; S2. Preprocess the original tobacco image to generate a standardized image; S3. Input the standardized image into the pre-trained ConvNeXt model for feature extraction. Through hierarchical convolution stacking, multi-scale texture details and high-level semantic information are fused to generate a feature image set. S4. Input the feature image set into the improved Mask2Former model, introduce the scale-aware query mechanism and the local texture attention branch mechanism, capture small targets through multiple sets of dedicated query vectors, and use texture confidence to guide the decoder decision and output the instance segmentation mask. S5. Based on the instance segmentation mask, determine whether there are foreign objects in the standardized image. If there are, calculate and output the foreign object category label and its specific coordinate information in the original tobacco image. S6. Package the original tobacco image, foreign object category label and specific coordinate information into a data package to generate a recognition result data package, and output the recognition result data package to the display terminal.
[0008] Optionally, S1 specifically includes: detecting the arrival of tobacco through an external photoelectric sensor and triggering the camera to perform a single exposure, capturing the image of the tobacco to be detected within the current field of view and saving it as the original tobacco image.
[0009] Optionally, S2 specifically includes: S21. Calculate the average gray values of the R, G, and B channels of all pixels in the original tobacco image, and multiply them by a preset gain coefficient so that the adjusted average gray values of the three channels are equal to a preset target gray value, thus obtaining the color-corrected original tobacco image. S22. Perform median filtering on the original tobacco image after color correction. Use a sliding window of a preset size pixels to traverse the image and replace the gray value of the center pixel of the window with the median of the gray values of the nine pixels in the window. S23. Perform histogram equalization on the original tobacco image after noise suppression, and output the original tobacco image after color correction, noise suppression and contrast enhancement as a normalized image.
[0010] Optionally, S3 specifically includes: S31. Input the standardized image into a convolutional layer with a preset convolutional kernel and stride size to reduce the size of the standardized image to a preset ratio in the spatial dimension. At the same time, stack all the obtained feature maps according to the number of convolutional kernels to obtain a preliminary feature map with the number of channels expanded to the preset number of channels. S32. Input the preliminary feature map into the first stage of the ConvNeXt model, perform LayerNorm normalization on the preliminary feature map, and use a preset number of 1x1 convolution kernels to increase the number of channels by a preset factor. Use a depthwise separable convolution with a preset kernel size to extract spatial features. S33. Use a 1x1 convolution kernel equal to the number of channels in the first-stage input feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the feature map of the first stage input element by element through a residual connection to obtain the first-stage output feature map. S34. Input the output feature map of the first stage into the second stage, pass it through a LayerNorm normalization layer with the same kernel size as the first stage, and downsample it through a convolutional layer with a kernel size and stride that are half the preset size and stride of the first stage. At the same time, increase the number of channels to twice the preset number of channels in the first stage, and use a depthwise separable convolution with a kernel size of the preset size to extract spatial features. S35. Use a 1x1 convolution kernel with the number of channels of the first-stage output feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the second-stage output feature map element by element through a residual connection to the input second-stage feature map, i.e., the first-stage output feature map, to obtain the second-stage output feature map. S36. Input the output feature map of the second stage into the third stage, repeat the process of halving the convolution kernel size and stride and increasing the number of channels to twice the number of channels increased in the previous stage, and then perform the same spatial feature extraction to obtain the output feature map of the second stage. Continue until the output feature map of the fourth stage, which is the feature map containing the highest level semantic information. S37. Combine the feature maps output from the four stages, which have different spatial resolutions and channel depths, to generate a feature image set consisting of shallow multi-scale texture details and deep high-level semantic information.
[0011] Optionally, the LayerNorm normalization process specifically includes: for the input feature map containing multiple channels, the processing is applied independently to each pixel in the feature map; for a specific pixel, all values corresponding to the pixel in all channels are obtained, and the arithmetic mean and variance of all values are calculated; the original value of the pixel in each channel is subtracted from the arithmetic mean and divided by the standard deviation to obtain the normalized feature map.
[0012] Optionally, S4 specifically includes: S41. Input the feature image set into the pixel decoder of the improved Mask2Former model. The spatial size of each feature map in the feature image set is enlarged by a preset multiple through a preset transposed convolutional layer. After each upsampling operation, the current upsampled feature image set is concatenated with the unupsampled feature image set along the channel dimension to supplement the shallow texture details into the deep semantic information and generate a high-resolution feature map. S42. Input the high-resolution feature map into the Transformer decoder of the improved Mask2Former model. The decoder initializes a preset number of query vector sets. In each layer of the Transformer decoder, the query vector set is input into a multi-head self-attention layer. The dot product between each query vector and all other query vectors is calculated and normalized to obtain the attention weight. Each query vector is then updated according to the attention weight. S43. Input the updated query vector set and the high-resolution feature map as key and value vectors into the multi-head cross attention layer, calculate the dot product similarity score between the query vector and each position of the high-resolution feature map, and normalize to obtain the attention weight of each position. Multiply each position of the high-resolution feature map by the corresponding attention weight and sum them up. Aggregate the information of the high-resolution feature map into a single vector to obtain the intermediate query vector. S44. Input the intermediate query vector into a parallel local texture attention branch, perform a depthwise separable convolution operation on the intermediate query vector through a convolution kernel of a preset size, extract the local texture features of the region of interest of each intermediate query vector, calculate the information entropy of the extracted local texture features, and normalize the information entropy value as the texture confidence, generate a texture confidence vector with the same dimension as the intermediate query vector, and the value of each element in the texture confidence vector represents the texture complexity of the region of interest of the corresponding query vector; S45. Multiply the texture confidence vector and the intermediate query vector element-wise to generate the texture-corrected query vector. Input the texture-corrected query vector into the next layer of the Transformer decoder for further processing. Repeat the steps of generating intermediate query vectors and texture correction until all preset layers of the decoder have been processed to obtain the final query vector. S46. Input the final query vector into the preset classification linear layer, output the probability score corresponding to each foreign object category in the preset foreign object category set, select the foreign object category label with the highest probability, and transform and reshape each final query vector into a dynamic convolution kernel of a specific shape with the same dimension and number of channels as the high-resolution feature map through matrix operations of the preset matrix using a multilayer perceptron. S47. Perform a convolution operation on the high-resolution feature map using a dynamic convolution kernel to generate a single-channel mask feature map with the same size as the high-resolution feature map. After activation by the sigmoid function, restore it to the same size as the original image through an upsampling operation to generate a binary instance segmentation mask.
[0013] Optionally, the calculation of information entropy specifically includes: The local texture features are transformed into texture feature values that can represent texture attributes through a convolution operation. The local feature values are divided into different preset value intervals and the frequency distribution of their occurrence in the whole region is statistically analyzed. The probability of each local feature value occurring in the region is obtained by dividing the number of occurrences of the value interval in which each local feature value is located by the total number of value intervals in the region. The probability of each local feature value appearing in the region is calculated using a base-2 logarithm. The probability value of each local feature value is multiplied by the corresponding logarithmic result to obtain the weighted information content. The weighted information contents of all different local feature values in this local region are summed to obtain the information entropy value of the local region.
[0014] Optionally, the 5 specifically includes: S51. Traverse each generated instance segmentation mask, sum up all pixels with a value of 1 in the instance segmentation mask to get a total number of pixels and compare it with a preset pixel threshold. If the total number of pixels is greater than the preset pixel threshold, mark the instance segmentation mask as a valid foreign object; otherwise, discard the instance segmentation mask. S52. For each instance segmentation mask marked as a valid foreign object, read the corresponding foreign object category label. For each instance segmentation mask marked as a valid foreign object, initialize an x-coordinate accumulation value and a y-coordinate accumulation value. Traverse all pixels with a pixel value of 1 in the instance segmentation mask. For each pixel, accumulate the x-coordinate value to the x-coordinate accumulation value and the y-coordinate value to the y-coordinate accumulation value. S53. After the traversal is completed, divide the accumulated value of the horizontal coordinate by the sum of the accumulated values of the horizontal and vertical coordinates to obtain the average value of the horizontal coordinate. Perform the same operation to obtain the average value of the vertical coordinate. Combine the average value of the horizontal coordinate and the average value of the vertical coordinate to form a two-dimensional coordinate point, which serves as the specific coordinate information of the foreign object in the original tobacco image.
[0015] Optionally, step 6 specifically includes: packaging the original tobacco image, foreign object category label, and specific coordinate information into a data package to generate a recognition result data package containing visual markers; integrating information in a preset structured format and outputting the result data package to the display terminal; after receiving the data, the display terminal draws a bounding box on the original tobacco image based on the coordinate information and marks the corresponding foreign object category label within the box to generate a visual result.
[0016] The beneficial effects of this invention are: First, by using the ConvNeXt model for hierarchical feature extraction, multi-scale features ranging from shallow texture details to deep semantic information are effectively integrated, which builds a high-resolution feature representation foundation for the accurate identification of tiny foreign objects and significantly improves the model's feature representation ability in complex backgrounds.
[0017] Secondly, the core of this invention lies in its innovative improvement of the Mask2Former model. By introducing a scale-aware query mechanism, multiple sets of dedicated query vectors are used to actively capture foreign objects of different sizes, effectively solving the problem of missed detection caused by the insensitivity of traditional models to small targets. Simultaneously, the unique local texture attention branch mechanism guides the decoder's decision by calculating texture confidence, enhancing the ability to distinguish between targets with similar colors but different textures, and significantly reducing the false positive rate. This improvement significantly enhances the model's accuracy and robustness in recognizing small, low-contrast, and textured foreign objects.
[0018] Furthermore, the instance segmentation mask output by this invention can be accurate to the pixel level. Combined with post-processing steps, it can accurately calculate the category and coordinate information of foreign objects and generate a visual result data package, providing an accurate and reliable decision-making basis for subsequent automated removal operations, and realizing closed-loop management from detection to location and disposal.
[0019] In summary, this invention significantly improves the accuracy, recall, and system practicality of tobacco foreign object recognition by integrating the advanced ConvNeXt feature extraction network with the improved Mask2Former model, effectively solving the key technical challenges of small target detection and texture differentiation in cigarette storage scenarios. Attached Figure Description
[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a tobacco safety foreign object identification method based on an image segmentation algorithm proposed in this invention; Figure 2 This is a flowchart of the multi-scale feature extraction and feature image set generation based on the ConvNeXt model proposed in this invention; Figure 3 This is a flowchart of the scale-aware query and texture attention-guided segmentation based on the improved Mask2Former model proposed in this invention. Detailed Implementation
[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0022] refer to Figures 1-3 A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm includes the following steps: S1. Acquire images of the tobacco to be detected on the conveyor belt to obtain the original tobacco image; S2. Preprocess the original tobacco image, including color correction, noise suppression and contrast enhancement, to generate a standardized image; S3. Input the standardized image into the pre-trained ConvNeXt model for feature extraction. Through hierarchical convolution stacking, multi-scale texture details and high-level semantic information are fused to generate a feature image set. S4. Input the feature image set into the improved Mask2Former model, introduce the scale-aware query mechanism and the local texture attention branch mechanism, capture small targets through multiple sets of dedicated query vectors, and use texture confidence to guide the decoder decision, outputting an accurate instance segmentation mask corresponding to the foreign object category. S5. Based on the instance segmentation mask, determine whether there are foreign objects in the standardized image. If there are, calculate and output the foreign object category label and its specific coordinate information in the original tobacco image. S6. Package the original tobacco image, foreign object category label, and specific coordinate information into a data package to generate a recognition result data package containing visual markers. Output the recognition result data package to the display terminal for operators to confirm and perform subsequent removal operations.
[0023] This implementation significantly improves the accuracy and efficiency of identifying and processing minute foreign objects in tobacco sorting lines. By employing the ConvNeXt model for hierarchical feature extraction, it effectively integrates multi-scale texture details and high-level semantic information, laying a solid foundation for subsequent accurate segmentation. The core lies in the improved Mask2Former model, which introduces a scale-aware query mechanism and a local texture attention branch. This proactively captures minute targets and uses texture confidence to guide decision-making, effectively solving the problem of high false negative rates for low-contrast and irregularly shaped foreign objects in traditional methods. The final output instance segmentation mask accurately marks the outline of the foreign object. Combined with a visual data package, it achieves full automation from detection to location and removal, significantly improving the intelligence level and security of cigarette storage safety control.
[0024] In this embodiment, S1 specifically includes: detecting the arrival of tobacco through an external photoelectric sensor and triggering the camera to perform a single exposure, capturing the image of the tobacco to be detected within the current field of view and saving it as the original tobacco image.
[0025] In this embodiment, S2 specifically includes: S21. Calculate the average gray values of the R, G, and B channels of all pixels in the original tobacco image, and multiply them by a preset gain coefficient so that the adjusted average gray values of the three channels are equal to a preset target gray value, thus obtaining the color-corrected original tobacco image. S22. Perform median filtering on the original tobacco image after color correction. Use a sliding window of a preset size pixels to traverse the image and replace the gray value of the center pixel of the window with the median of the gray values of the nine pixels in the window. S23. Perform histogram equalization on the original tobacco image after noise suppression, and output the original tobacco image after color correction, noise suppression and contrast enhancement as a normalized image.
[0026] In this embodiment, S3 specifically includes: S31. Input the standardized image into a convolutional layer with a preset convolutional kernel and stride size to reduce the size of the standardized image to a preset ratio in the spatial dimension. At the same time, stack all the obtained feature maps according to the number of convolutional kernels to obtain a preliminary feature map with the number of channels expanded to the preset number of channels. S32. Input the preliminary feature map into the first stage of the ConvNeXt model, perform LayerNorm normalization on the preliminary feature map, and use a preset number of 1x1 convolution kernels to increase the number of channels by a preset factor. Use a depthwise separable convolution with a preset kernel size to extract spatial features. S33. Use a 1x1 convolution kernel equal to the number of channels in the first-stage input feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the feature map of the first stage input element by element through a residual connection to obtain the first-stage output feature map. S34. Input the output feature map of the first stage into the second stage, pass it through a LayerNorm normalization layer with the same kernel size as the first stage, and downsample it through a convolutional layer with a kernel size and stride that are half the preset size and stride of the first stage. At the same time, increase the number of channels to twice the preset number of channels in the first stage, and use a depthwise separable convolution with a kernel size of the preset size to extract spatial features. S35. Use a 1x1 convolution kernel with the number of channels of the first-stage output feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the second-stage output feature map element by element through a residual connection to the input second-stage feature map, i.e., the first-stage output feature map, to obtain the second-stage output feature map. S36. Input the output feature map of the second stage into the third stage, repeat the process of halving the convolution kernel size and stride and increasing the number of channels to twice the number of channels increased in the previous stage, and then perform the same spatial feature extraction to obtain the output feature map of the second stage. Continue until the output feature map of the fourth stage, which is the feature map containing the highest level semantic information. S37. The feature maps output from the four stages, each with different spatial resolutions and channel depths, are combined to generate a feature image set composed of shallow multi-scale texture details and deep high-level semantic information. This implementation achieves multi-scale depth representation of tobacco images through the hierarchical feature extraction architecture of the ConvNeXt model. First, a preliminary feature map is constructed using convolutional layers. Then, in multiple consecutive stages, a combination of operations—LayerNorm normalization, 1x1 convolution dimensionality upscaling, depthwise separable convolutional space extraction, 1x1 convolution dimensionality reduction, GELU activation, and residual connections—is used to progressively downsample and deepen the features. This design effectively preserves shallow details at each stage through residual connections, avoiding the loss of subtle texture information in deep networks. Finally, the feature maps with different resolutions output from the four stages are combined to generate a feature image set that simultaneously contains shallow multi-scale texture details and deep high-level semantic information, providing a high-quality feature foundation with both detailed integrity and semantic richness for the subsequent accurate identification and segmentation of small foreign objects.
[0027] For example, the input standardized image size is 1920x1080 with 3 channels. Step S31 uses a convolutional layer with a kernel size of 4 and a stride of 4 to process the image, reducing the output feature map size to 480x270 while expanding the number of channels to 96, resulting in a preliminary feature map. In stages S32-S36, the first stage outputs a feature map size of 480x270 with 96 channels; the second stage, after downsampling, outputs a size of 240x135 with 192 channels; the third stage outputs a size of 120x68 with 384 channels; and the fourth stage outputs a size of 60x34 with 768 channels. Step S37 combines these four feature maps with different resolutions and channel numbers to generate a highly information-rich feature image set containing everything from high-level semantics (60x34) to texture details (480x270), providing a multi-scale feature foundation for subsequent accurate segmentation.
[0028] This implementation achieves multi-scale deep representation of tobacco images through a hierarchical feature extraction architecture of the ConvNeXt model. First, a preliminary feature map is constructed using convolutional layers. Then, in multiple consecutive stages, a combination of operations—LayerNorm normalization, 1x1 convolutional dimensionality upscaling, depthwise separable convolutional space extraction, 1x1 convolutional dimensionality reduction, GELU activation, and residual connections—is used to progressively downsample and deepen the features. This design effectively preserves shallow details at each stage through residual connections, avoiding the loss of subtle texture information in deeper networks. Finally, the feature maps at different resolutions output from the four stages are combined to generate a feature image set that simultaneously contains shallow multi-scale texture details and deep high-level semantic information. This provides a high-quality feature foundation with both detailed integrity and semantic richness for the subsequent accurate identification and segmentation of small foreign objects.
[0029] In this embodiment, the LayerNorm normalization process specifically includes: for the input feature map containing multiple channels, the processing is applied independently to each pixel in the feature map; for a specific pixel, all values corresponding to the pixel in all channels are obtained, and the arithmetic mean and variance of all values are calculated; the original value of the pixel in each channel is subtracted from the arithmetic mean and divided by the standard deviation to obtain the normalized feature map.
[0030] In this embodiment, S4 specifically includes: S41. Input the feature image set into the pixel decoder of the improved Mask2Former model. The spatial size of each feature map in the feature image set is enlarged by a preset multiple through a preset transposed convolutional layer. After each upsampling operation, the current upsampled feature image set is concatenated with the unupsampled feature image set along the channel dimension to supplement the shallow texture details into the deep semantic information and generate a high-resolution feature map. S42. Input the high-resolution feature map into the Transformer decoder of the improved Mask2Former model. The decoder initializes a preset number of query vector sets. In each layer of the Transformer decoder, the query vector set is input into a multi-head self-attention layer. The dot product between each query vector and all other query vectors is calculated and normalized to obtain the attention weight. Each query vector is then updated according to the attention weight. S43. Input the updated query vector set and the high-resolution feature map as key and value vectors into the multi-head cross attention layer, calculate the dot product similarity score between the query vector and each position of the high-resolution feature map, and normalize to obtain the attention weight of each position. Multiply each position of the high-resolution feature map by the corresponding attention weight and sum them up. Aggregate the information of the high-resolution feature map into a single vector to obtain the intermediate query vector. S44. Input the intermediate query vector into a parallel local texture attention branch, perform a depthwise separable convolution operation on the intermediate query vector through a convolution kernel of a preset size, extract the local texture features of the region of interest of each intermediate query vector, calculate the information entropy of the extracted local texture features, and normalize the information entropy value as the texture confidence, generate a texture confidence vector with the same dimension as the intermediate query vector, and the value of each element in the texture confidence vector represents the texture complexity of the region of interest of the corresponding query vector; S45. Multiply the texture confidence vector and the intermediate query vector element-wise to generate a texture-corrected query vector. Use the texture confidence vector to weight each dimension of the intermediate query vector to enhance the query vector response in complex texture areas and suppress the query vector response in smooth texture areas. Input the texture-corrected query vector into the next layer of the Transformer decoder for further processing. Repeat the steps of generating intermediate query vectors and texture correction until all preset layers of the decoder have been processed to obtain the final query vector containing precise location, category, and texture information. S46. Input the final query vector into the preset classification linear layer, output the probability score corresponding to each foreign object category in the preset foreign object category set, select the foreign object category label with the highest probability, and transform and reshape each final query vector into a dynamic convolution kernel of a specific shape with the same dimension and number of channels as the high-resolution feature map through matrix operations of the preset matrix using a multilayer perceptron. S47. Perform a convolution operation on the high-resolution feature map using a dynamic convolution kernel to generate a single-channel mask feature map with the same size as the high-resolution feature map. After activation by the sigmoid function, restore it to the same size as the original image through an upsampling operation to generate a binary instance segmentation mask.
[0031] In this embodiment, the calculation of information entropy specifically includes: The local texture features are transformed into texture feature values that can represent texture attributes through a convolution operation. The local feature values are divided into different preset value intervals and the frequency distribution of their occurrence in the whole region is statistically analyzed. The probability of each local feature value occurring in the region is obtained by dividing the number of occurrences of the value interval in which each local feature value is located by the total number of value intervals in the region. The probability of each local feature value appearing in the region is calculated using a base-2 logarithm. The probability value of each local feature value is multiplied by the corresponding logarithmic result to obtain the weighted information content. The weighted information contents of all different local feature values in this local region are summed to obtain the information entropy value of the local region.
[0032] This implementation achieves accurate segmentation of tiny foreign objects using an improved Mask2Former model. First, a pixel decoder uses transposed convolution and feature concatenation to supplement shallow texture details with deep semantic information, generating a high-resolution feature map. Then, the Transformer decoder uses self-attention and cross-attention mechanisms to progressively aggregate target information from the query vector. The core innovation lies in introducing a parallel local texture attention branch, which extracts local texture features and calculates information entropy through depthwise separable convolution, generating a texture confidence vector. This vector weights and corrects intermediate query vectors, significantly enhancing the model's response to complex, tiny targets while suppressing interference from smooth backgrounds. Finally, the corrected query vector generates a dynamic convolution kernel, which directly acts on the high-resolution feature map to output an accurate instance segmentation mask, effectively solving the problem of insufficient accuracy in identifying low-contrast, irregular, tiny foreign objects using traditional methods.
[0033] In this embodiment, the 5 specifically includes: S51. Traverse each generated instance segmentation mask, sum up all pixels with a value of 1 in the instance segmentation mask to get a total number of pixels and compare it with a preset pixel threshold. If the total number of pixels is greater than the preset pixel threshold, mark the instance segmentation mask as a valid foreign object; otherwise, discard the instance segmentation mask. S52. For each instance segmentation mask marked as a valid foreign object, read the corresponding foreign object category label. For each instance segmentation mask marked as a valid foreign object, initialize an x-coordinate accumulation value and a y-coordinate accumulation value. Traverse all pixels with a pixel value of 1 in the instance segmentation mask. For each pixel, accumulate the x-coordinate value to the x-coordinate accumulation value and the y-coordinate value to the y-coordinate accumulation value. S53. After the traversal is completed, divide the accumulated value of the horizontal coordinate by the sum of the accumulated values of the horizontal and vertical coordinates to obtain the average value of the horizontal coordinate. Perform the same operation to obtain the average value of the vertical coordinate. Combine the average value of the horizontal coordinate and the average value of the vertical coordinate to form a two-dimensional coordinate point, which serves as the specific coordinate information of the foreign object in the original tobacco image.
[0034] In this embodiment, step 6 specifically includes: packaging the original tobacco image, foreign object category label, and specific coordinate information into a data package to generate a recognition result data package containing visual markers; integrating the information in a preset structured format and outputting the result data package to the display terminal; after receiving the data, the display terminal draws a bounding box on the original tobacco image based on the coordinate information and marks the corresponding foreign object category label within the box to generate a visual result for the operator to review intuitively.
[0035] Example 1: To verify the feasibility of this invention in practice, it was applied to the online fire detection and control system of a cigarette warehouse of a large tobacco company in China. This warehouse, as a core hub for regional logistics, undertakes the centralized storage and high-frequency turnover of massive quantities of finished cigarettes and various supporting raw materials. The warehouse area comprises complex storage areas of various types, including elevated warehouses, tobacco leaf purchasing and storage warehouses, and special maintenance warehouses. The elevated warehouse area primarily relies on high-speed stacker cranes and electric forklifts for the automated storage, retrieval, sorting, and packaging of densely packed finished cigarettes. The finished cigarette sorting line uses belt conveyors for continuous sorting, conveying, and packaging of large quantities of finished cigarettes. The tobacco leaf purchasing and storage warehouses utilize belt conveyors for continuous conveying and packaging of large quantities of tobacco leaves. The internal environment of these warehouses is extremely complex, especially the tobacco leaf purchasing and storage warehouses, where the stacked tobacco leaves exhibit highly irregular yellowish-brown clump-like textures. Furthermore, due to dynamic differences in moisture content and color between different origins and batches of tobacco leaves, the color and texture of the warehouse background are constantly changing. During the high-frequency operation of equipment and the flow of finished cigarettes and tobacco leaves, various minute foreign objects posing safety risks can easily get mixed in. Typical risks include: tiny plastic films peeling off aging conveyor belts, metal shavings generated by mechanical friction of equipment, pieces of paper falling from damaged outer packaging, and fibers from clothing worn by workers. Traditional warehouse safety management systems mainly rely on manual periodic inspections and basic video surveillance backtracking, which have significant blind spots when dealing with minute foreign objects, and post-event traceability is often severely delayed. More critically, existing online fire detection and control systems only focus on abnormal temperature and humidity and open flame smoke recognition, completely lacking the ability to predict the risk of smoldering caused by the accumulation of these early minute foreign objects. Traditional machine vision algorithms also struggle to distinguish between foreign objects of the same color and background textures in such scenarios, leading to frequent false alarms and missed alarms, posing significant safety hazards to the highly concentrated storage of tobacco leaves and finished cigarettes.
[0036] In practical deployment, the method of this invention is seamlessly embedded into the front-end visual acquisition and intelligent analysis nodes of the existing online fire detection and control system in warehouses. We construct an all-weather image acquisition network, where the system acquires raw images of the area to be detected in real time when the conveyor belt or transfer equipment is running. These images first enter a preprocessing module, where a precise color correction algorithm eliminates background color differences caused by different batches of materials, and median filtering removes high-frequency electromagnetic noise in the industrial environment. Adaptive histogram equalization then significantly improves the contrast between the edges of weak foreign objects and the complex background, generating high-quality standardized images. Subsequently, the images are fed into a pre-trained ConvNeXt model, which, through hierarchical convolution stacking and residual connections, perfectly preserves the shallow, subtle texture details during the progressive downsampling process and deeply fuses them with deep macroscopic semantic features, generating a feature image set containing rich multi-scale information.
[0037] Following this, the improved Mask2Former model begins to play a core discriminative role. Within the Transformer decoder, a scale-aware query mechanism actively searches for and locks onto potential micro-targets within massive amounts of complex material textures using multiple sets of dedicated query vectors. More importantly, a parallel local texture attention branch performs in-depth analysis of each candidate region, generating a texture confidence vector by calculating local information entropy. This mechanism acts like giving the system eyes that can perceive material differences, keenly capturing the essential differences between the smooth texture of plastic film, the sharp edges of metal debris, and the rough fibers of the background. This texture confidence guides the decoder to make accurate decisions, outputting pixel-level instance segmentation masks. Finally, the system accurately calculates the specific category and spatial coordinates of the foreign object based on the mask and pushes a warning data packet with a visual bounding box to the fire detection and control center in real time. This achieves closed-loop management from accurate identification of micro-objects to early fire risk prevention, greatly ensuring the storage safety of tobacco leaves and finished cigarettes. Table 1 below shows detailed comparative data on the identification of small, secure foreign objects in warehouses, comparing the method of this invention with the existing YOLOv5-based target detection system during a three-month continuous high-intensity operational test. Table 1. Performance Comparison Data Between the Invention and Traditional Methods
[0038] Based on the comparative data shown in Table 1, it can be seen that the foreign object identification method based on image segmentation algorithm proposed in this invention has significant performance advantages over traditional detection methods in terms of safety management of tobacco leaf and finished cigarette storage. In particular, it has achieved leapfrog improvements in key indicators such as identification accuracy, processing timeliness, segmentation accuracy, and control of false alarms and missed alarms.
[0039] In terms of recognition accuracy, this invention maintains an extremely high level of over 95% in all four typical foreign object scenarios, far exceeding the approximately 60% accuracy of traditional methods for non-metallic foreign objects. For example, in the scenario of "shedding fibers during operation," traditional systems lack a deep understanding of local textures and are easily mistaken for natural veins in the background, resulting in an accuracy of only 60.0%. In contrast, this invention achieves 96.0% accurate recognition by extracting physical features through an innovative local texture attention branch and calculating information entropy for confidence weighting. This completely solves the core pain point of missing low-contrast, small, same-color targets in the circulation of tobacco leaves and finished cigarettes.
[0040] In terms of processing timeliness, this invention, with its optimized feature extraction architecture and efficient decoding process, maintains an average single-frame processing time of 31 to 35 milliseconds, significantly outperforming the 45 to 55 millisecond processing time of traditional methods. This microsecond-level performance improvement is particularly crucial in high-speed belt conveyor scenarios, ensuring that the system can capture and accurately analyze images of tobacco leaves and finished cigarettes in the instant they move rapidly, fully meeting the stringent real-time requirements of modern warehousing and logistics.
[0041] In terms of segmentation accuracy, this invention demonstrates a significant advantage in pixel-level segmentation. By directly applying dynamic convolutional kernels to high-resolution feature maps, this invention achieves an edge pixel fit rate exceeding 91% in various foreign object detection methods, far surpassing the less than 72% performance of traditional methods. This means that the system output is no longer a vaguely located region, but a mask accurate to the actual contour of foreign objects in tobacco leaves and finished cigarettes, providing a reliable geometric basis for subsequent automated equipment to accurately remove them.
[0042] The invention also demonstrates significant advantages in controlling false alarm and false negative rates. The average false alarm rate is strictly controlled to below 2%, and the false negative rate is maintained below 3.2%. Compared to the traditional methods with a false alarm rate as high as 10% and a false negative rate of nearly 30%, this greatly improves the overall stability of the online fire detection and control system. The extremely low false alarm rate avoids downtime and manual verification costs caused by frequent false alarms, while the extremely low false negative rate fundamentally eliminates the serious accident hazard of smoldering and even fire caused by the long-term accumulation of tiny foreign objects. Overall, the technical advantages of this invention are particularly prominent when dealing with tiny foreign objects in tobacco leaves and finished cigarettes in complex backgrounds, providing an excellent solution for high-precision warehouse safety detection.
[0043] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm, characterized in that, Includes the following steps: S1. Acquire images of the tobacco to be detected to obtain the original tobacco image; S2. Preprocess the original tobacco image to generate a standardized image; S3. Input the standardized image into the pre-trained ConvNeXt model for feature extraction. Through hierarchical convolution stacking, multi-scale texture details and high-level semantic information are fused to generate a feature image set. S4. Input the feature image set into the improved Mask2Former model, introduce the scale-aware query mechanism and the local texture attention branch mechanism, capture small targets through multiple sets of dedicated query vectors, and use texture confidence to guide the decoder decision and output the instance segmentation mask. S5. Based on the instance segmentation mask, determine whether there are foreign objects in the standardized image. If there are, calculate and output the foreign object category label and its specific coordinate information in the original tobacco image. S6. Package the original tobacco image, foreign object category label and specific coordinate information into a data package to generate a recognition result data package, and output the recognition result data package to the display terminal.
2. The method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, S1 specifically includes: detecting the arrival of tobacco through an external photoelectric sensor and triggering the camera to perform a single exposure, capturing the image of the tobacco to be detected within the current field of view and saving it as the original tobacco image.
3. The method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, S2 specifically includes: S21. Calculate the average gray values of the R, G, and B channels of all pixels in the original tobacco image, and multiply them by a preset gain coefficient so that the adjusted average gray values of the three channels are equal to a preset target gray value, thus obtaining the color-corrected original tobacco image. S22. Perform median filtering on the original tobacco image after color correction. Use a sliding window of a preset size pixels to traverse the image and replace the gray value of the center pixel of the window with the median of the gray values of the nine pixels in the window. S23. Perform histogram equalization on the original tobacco image after noise suppression, and output the original tobacco image after color correction, noise suppression and contrast enhancement as a normalized image.
4. The method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, S3 specifically includes: S31. Input the standardized image into a convolutional layer with a preset convolutional kernel and stride size to reduce the size of the standardized image to a preset ratio in the spatial dimension. At the same time, stack all the obtained feature maps according to the number of convolutional kernels to obtain a preliminary feature map with the number of channels expanded to the preset number of channels. S32. Input the preliminary feature map into the first stage of the ConvNeXt model, perform LayerNorm normalization on the preliminary feature map, and use a preset number of 1x1 convolution kernels to increase the number of channels by a preset factor. Use a depthwise separable convolution with a preset kernel size to extract spatial features. S33. Use a 1x1 convolution kernel equal to the number of channels in the first-stage input feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the feature map of the first stage input element by element through a residual connection to obtain the first-stage output feature map. S34. Input the output feature map of the first stage into the second stage, pass it through a LayerNorm normalization layer with the same kernel size as the first stage, and downsample it through a convolutional layer with a kernel size and stride that are half the preset size and stride of the first stage. At the same time, increase the number of channels to twice the preset number of channels in the first stage, and use a depthwise separable convolution with a kernel size of the preset size to extract spatial features. S35. Use a 1x1 convolution kernel with the number of channels of the first-stage output feature map to compress the number of channels back to the original dimension and apply the GELU activation function. Add the second-stage output feature map element by element through a residual connection to the input second-stage feature map, i.e., the first-stage output feature map, to obtain the second-stage output feature map. S36. Input the output feature map of the second stage into the third stage, repeat the process of halving the convolution kernel size and stride and increasing the number of channels to twice the number of channels increased in the previous stage, and then perform the same spatial feature extraction to obtain the output feature map of the second stage. Continue until the output feature map of the fourth stage, which is the feature map containing the highest level semantic information. S37. Combine the feature maps output from the four stages, which have different spatial resolutions and channel depths, to generate a feature image set consisting of shallow multi-scale texture details and deep high-level semantic information.
5. A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 4, characterized in that, The LayerNorm normalization process specifically includes: for the input feature map containing multiple channels, the processing is applied independently to each pixel in the feature map; for a specific pixel, all values corresponding to the pixel in all channels are obtained, and the arithmetic mean and variance of all values are calculated; the original value of the pixel in each channel is subtracted from the arithmetic mean and divided by the standard deviation to obtain the normalized feature map.
6. The method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, S4 specifically includes: S41. Input the feature image set into the pixel decoder of the improved Mask2Former model. The spatial size of each feature map in the feature image set is enlarged by a preset multiple through a preset transposed convolutional layer. After each upsampling operation, the current upsampled feature image set is concatenated with the unupsampled feature image set along the channel dimension to supplement the shallow texture details into the deep semantic information and generate a high-resolution feature map. S42. Input the high-resolution feature map into the Transformer decoder of the improved Mask2Former model. The decoder initializes a preset number of query vector sets. In each layer of the Transformer decoder, the query vector set is input into a multi-head self-attention layer. The dot product between each query vector and all other query vectors is calculated and normalized to obtain the attention weight. Each query vector is then updated according to the attention weight. S43. Input the updated query vector set and the high-resolution feature map as key and value vectors into the multi-head cross attention layer, calculate the dot product similarity score between the query vector and each position of the high-resolution feature map, and normalize to obtain the attention weight of each position. Multiply each position of the high-resolution feature map by the corresponding attention weight and sum them up. Aggregate the information of the high-resolution feature map into a single vector to obtain the intermediate query vector. S44. Input the intermediate query vector into a parallel local texture attention branch, perform a depthwise separable convolution operation on the intermediate query vector through a convolution kernel of a preset size, extract the local texture features of the region of interest of each intermediate query vector, calculate the information entropy of the extracted local texture features, and normalize the information entropy value as the texture confidence, generate a texture confidence vector with the same dimension as the intermediate query vector, and the value of each element in the texture confidence vector represents the texture complexity of the region of interest of the corresponding query vector; S45. Multiply the texture confidence vector and the intermediate query vector element-wise to generate the texture-corrected query vector. Input the texture-corrected query vector into the next layer of the Transformer decoder for further processing. Repeat the steps of generating intermediate query vectors and texture correction until all preset layers of the decoder have been processed to obtain the final query vector. S46. Input the final query vector into the preset classification linear layer, output the probability score corresponding to each foreign object category in the preset foreign object category set, select the foreign object category label with the highest probability, and transform and reshape each final query vector into a dynamic convolution kernel of a specific shape with the same dimension and number of channels as the high-resolution feature map through matrix operations of the preset matrix using a multilayer perceptron. S47. Perform a convolution operation on the high-resolution feature map using a dynamic convolution kernel to generate a single-channel mask feature map with the same size as the high-resolution feature map. After activation by the sigmoid function, restore it to the same size as the original image through an upsampling operation to generate a binary instance segmentation mask.
7. A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 6, characterized in that, The calculation of information entropy specifically includes: The local texture features are transformed into texture feature values that can represent texture attributes through a convolution operation. The local feature values are divided into different preset value intervals and the frequency distribution of their occurrence in the whole region is statistically analyzed. The probability of each local feature value occurring in the region is obtained by dividing the number of occurrences of the value interval in which each local feature value is located by the total number of value intervals in the region. The probability of each local feature value appearing in the region is calculated using a base-2 logarithm. The probability value of each local feature value is multiplied by the corresponding logarithmic result to obtain the weighted information content. The weighted information contents of all different local feature values in this local region are summed to obtain the information entropy value of the local region.
8. A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, The 5 specifically includes: S51. Traverse each generated instance segmentation mask, sum up all pixels with a value of 1 in the instance segmentation mask to get a total number of pixels and compare it with a preset pixel threshold. If the total number of pixels is greater than the preset pixel threshold, mark the instance segmentation mask as a valid foreign object; otherwise, discard the instance segmentation mask. S52. For each instance segmentation mask marked as a valid foreign object, read the corresponding foreign object category label. For each instance segmentation mask marked as a valid foreign object, initialize an x-coordinate accumulation value and a y-coordinate accumulation value. Traverse all pixels with a pixel value of 1 in the instance segmentation mask. For each pixel, accumulate the x-coordinate value to the x-coordinate accumulation value and the y-coordinate value to the y-coordinate accumulation value. S53. After the traversal is completed, divide the accumulated value of the horizontal coordinate by the sum of the accumulated values of the horizontal and vertical coordinates to obtain the average value of the horizontal coordinate. Perform the same operation to obtain the average value of the vertical coordinate. Combine the average value of the horizontal coordinate and the average value of the vertical coordinate to form a two-dimensional coordinate point, which serves as the specific coordinate information of the foreign object in the original tobacco image.
9. A method for identifying foreign objects in tobacco safety based on an image segmentation algorithm according to claim 1, characterized in that, The 6th step specifically includes: packaging the original tobacco image, foreign object category label, and specific coordinate information into a data package to generate a recognition result data package containing visual markers; integrating the information in a preset structured format and outputting the result data package to the display terminal; after receiving the data, the display terminal draws a bounding box on the original tobacco image based on the coordinate information and marks the corresponding foreign object category label within the box to generate a visual result.