Tobacco shred impurity detection method, device, medium and equipment

Abstract

The invention discloses a tobacco shred impurity detection method and device, a medium and equipment. The method comprises the following steps: acquiring a tobacco shred image of at least one tobacco shred object; performing impurity detection on each tobacco shred image based on a first preset detection model so as to screen out to-be-determined tobacco shred objects which are confirmed to have no tobacco shred impurities, and reporting the tobacco shred objects which are confirmed to have the tobacco shred impurities; performing impurity detection on the tobacco shred image of the to-be-determined tobacco shred object based on a second preset detection model so as to screen out an over-detected tobacco shred object in which no tobacco shred impurity is confirmed, and reporting the tobacco shred object in which the tobacco shred impurity is confirmed; wherein the first preset detection model is one of a multi-scale sundry detection model and a white fine line extraction positioning model, and the second preset detection model is the other one of the multi-scale sundry detection model and the white fine line extraction positioning model. By means of the method, various tobacco shred impurities in different scales can be effectively detected, and the tobacco shred impurity detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the technical field of cigarette production, and in particular to a method, device, medium and equipment for detecting tobacco debris. Background Art

[0002] During the cigarette production process, especially after the cigarette packs are cut into shreds, there are often various types of tobacco debris attached. These debris may include white threads, white plastic, white paper, white feathers, red ribbons, blue plastic, blue strips, brown sponges, human hands during manual intervention, and other non-tobacco substances. The presence of these tobacco debris not only affects the appearance and quality of cigarettes, but may also pose a potential threat to the health and safety of consumers. Therefore, in the cigarette pack cutting process of cigarette production, it is urgent to detect and remove tobacco debris.

[0003] At present, in the construction of automation systems in cigarette factories, the process of removing tobacco debris is relatively simple. Only automatic optical inspection and removal are carried out before cutting, and manual inspection and removal are carried out after cutting. The inspection method is inefficient and labor-intensive, and is greatly affected by manual experience and subjective factors. There is a great risk of missed detection, and existing users have higher and higher requirements for the purity of tobacco. Traditional manual inspection methods can no longer meet the existing requirements for the purity of tobacco, affecting the quality and taste of cigarettes. It is inefficient and costly, and it is difficult to meet the needs of modern cigarette production. Summary of the invention

[0004] The present application mainly provides a tobacco debris detection method, device, medium and equipment, aiming to solve the technical problem of low efficiency of existing tobacco debris detection.

[0005] In order to solve the above technical problems, the technical solution adopted by the present application is: to provide a tobacco debris detection method. The tobacco debris detection method comprises: obtaining a tobacco image of at least one tobacco object; performing debris detection on each of the tobacco images based on a first preset detection model to screen out pending tobacco objects confirmed to be free of tobacco debris, and reporting the tobacco objects confirmed to be free of tobacco debris; performing debris detection on the tobacco images of the pending tobacco objects based on a second preset detection model to screen out inspected tobacco objects confirmed to be free of tobacco debris, and reporting the tobacco objects confirmed to be free of tobacco debris; wherein the first preset detection model is one of a multi-scale debris detection model and a white thin line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white thin line extraction and positioning model.

[0006] In some embodiments, the multi-scale debris detection model is obtained by training an original detection model based on a training atlas, wherein the training atlas is obtained based on enhancing the original image.

[0007] In some embodiments, before performing debris detection on each of the tobacco images based on the first preset detection model, it also includes: acquiring the original image and adding the original image to the training atlas; extracting the debris mask in the original image based on a preset segmentation model, replacing the background image of the debris mask, and adding the obtained new image to the training atlas; and / or performing visual enhancement on the original image based on a multi-scale retinal enhancement algorithm with color restoration, and adding the obtained image to the training atlas.

[0008] In some embodiments, the original detection model is obtained by adding a small target detection layer on the basis of the original YOLOv5 model, and the small target detection layer includes a newly added detection head with a sampling multiple of 4 and a prediction branch for the newly added detection head.

[0009] In some embodiments, before performing debris detection on each of the tobacco images based on the first preset detection model, the method further includes: scaling and cropping the original image, and adding the obtained new image to the training atlas.

[0010] In some embodiments, when the first preset detection model is the white thin line extraction and positioning model, the performing debris detection on each of the tobacco images based on the first preset detection model, or when the second preset detection model is the white thin line extraction and positioning model, the performing debris detection on the tobacco image of the pending tobacco object based on the second preset detection model, includes: converting the corresponding tobacco image into a hue-saturation-lightness color space and a hue-brightness-saturation color space respectively, and acquiring a debris extraction image corresponding to the white thin line debris in the tobacco image based on a preset foreground hue range, a preset foreground saturation range, a preset foreground lightness range, and a preset foreground brightness range; performing grayscale conversion and binarization processing on the debris extraction image to obtain a binarized debris image; determining a connected area of ​​the white thin line debris in the tobacco image based on a connection relationship and a similarity relationship between pixels in the binarized debris image; determining that an area of ​​the connected area in the tobacco image is greater than a preset debris area threshold, and confirming the corresponding tobacco object as a tobacco object with tobacco debris.

[0011] In some embodiments, the tobacco image of each of the tobacco objects covers a roller conveying area where the tobacco objects are produced.

[0012] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide a tobacco debris detection device, which includes: an acquisition module, used to acquire a tobacco image of at least one tobacco object; a first detection module, used to perform debris detection on each of the tobacco images based on a first preset detection model, so as to screen out pending tobacco objects that are confirmed to be free of tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; a second detection module, used to perform debris detection on the tobacco images of the pending tobacco objects based on a second preset detection model, so as to screen out inspected tobacco objects that are confirmed to be free of tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; wherein the first preset detection model is one of a multi-scale debris detection model and a white thin line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white thin line extraction and positioning model.

[0013] In order to solve the above technical problems, another technical solution adopted by the present application is: to provide a storage medium on which program data is stored, characterized in that when the program data is executed by a processor, the steps of the tobacco debris detection method as described above are implemented.

[0014] In order to solve the above technical problems, another technical solution adopted in the present application is: to provide a computer device, which includes a processor and a memory connected to each other, the memory stores a computer program, and when the processor executes the computer program, the steps of the tobacco debris detection method as mentioned above are implemented.

[0015] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a method, device, medium and equipment for detecting tobacco debris. The present application detects debris on the tobacco images of the acquired tobacco objects in turn based on the multi-scale debris detection model and the white fine line extraction and positioning model, and determines the tobacco objects that have passed the inspection and the tobacco objects that need to be reported because of tobacco debris. By using the multi-scale debris detection model and the white fine line extraction and positioning model, various debris in the tobacco can be identified and located more efficiently and accurately, and darker and smaller white fine line tobacco debris that are difficult to identify in the tobacco image by traditional methods can be effectively detected, thereby reducing missed detection and false detection, improving the automation level of cigarette production, and avoiding the problems of high labor intensity and being greatly affected by manual experience and subjective factors in the traditional manual detection method, which is conducive to improving the quality and taste of cigarette products, reducing production costs, and meeting the market's higher requirements for tobacco purity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 It is a flow chart of an embodiment of a tobacco debris detection method provided by the present application; Figure 2 yes Figure 1 A schematic diagram of a flow chart of an embodiment before step 20 in the embodiment; Figure 3 yes Figure 1 A schematic flow chart of an embodiment of step 20 or step 30 in the embodiment; Figure 4 It is a structural schematic diagram of an embodiment of a tobacco debris detection device provided by the present application; Figure 5 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application; Figure 6 It is a structural diagram of an embodiment of a computer device provided by the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0018] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0019] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0020] This application provides a method for detecting tobacco debris. Figure 1 , Figure 1 1 is a flow chart of an embodiment of a tobacco impurity detection method provided by the present application, the tobacco impurity detection method comprising: Step 10: Acquire a tobacco image of at least one tobacco object.

[0021] Step 20: Perform debris detection on each tobacco image based on the first preset detection model to screen out pending tobacco objects that are confirmed to have no tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris.

[0022] Step 30: Performing foreign matter detection on the tobacco images of the tobacco objects to be determined based on the second preset detection model to screen out tobacco objects that have been confirmed to have no tobacco foreign matter and pass inspection, and reporting tobacco objects that have been confirmed to have tobacco foreign matter.

[0023] Among them, the first preset detection model is one of the multi-scale debris detection model and the white thin line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white thin line extraction and positioning model.

[0024] In this embodiment, the tobacco object refers to the tobacco bundle formed by the shredding process in the cigarette production process. These tobacco objects move on the conveyor belt of the assembly line, and the corresponding tobacco images are obtained by the camera device installed on the assembly line, and these images are reported and processed. A tobacco object includes at least one corresponding tobacco image, and the corresponding tobacco image is used to describe the visual characteristics of the tobacco object. Therefore, by obtaining at least one tobacco image of the tobacco object, the sundries contained in the tobacco object can be effectively detected later.

[0025] In this embodiment, the detection of debris in the tobacco image is performed in a secondary detection manner, and the tobacco debris is detected jointly by deep learning and computer vision methods. Specifically, the detection based on computer vision can be performed first and then the detection based on deep learning, or the detection based on deep learning can be performed first and then the detection based on computer vision, or both can be performed simultaneously and then the detection results are integrated. In this solution, the detection based on deep learning is specifically performed by a multi-scale debris detection model, and the detection based on computer vision is specifically performed by the other of the white thin line extraction and positioning models.

[0026] In this embodiment, the multi-scale debris detection model is an image processing technology based on deep learning, which can identify debris in the image by learning a large amount of tobacco image data. The model is implemented based on a constructed deep neural network, which includes multiple levels, each of which is responsible for extracting tobacco debris features of different scales from the input tobacco image. For example, it is easy to identify large-scale tobacco debris such as white paper, white feathers, red ribbons, and human hands during manual intervention at some scales, and it is easy to identify medium-scale tobacco debris features such as white plastic, blue plastic, and brown sponge at some scales, and it is easy to identify small-scale tobacco debris such as white thin lines and blue thin strips at some scales. In the process of training to obtain the multi-scale debris detection model, the model is trained by sample images of tobacco debris of different categories and different scales, so that the model can automatically learn how to distinguish between tobacco and debris, and improve the accuracy of detection by multi-scale feature fusion.

[0027] In this embodiment, although the multi-scale debris detection model is compatible with detection at multiple scales, extremely small debris such as white thin line debris is usually difficult to detect in the high-scale range due to scale mismatch, and in the low-scale range, it is easy to be disturbed by other elements and difficult to be accurately identified by the model, so missed detection often occurs. In addition, since the white thin line image belongs to small sample image data, complex data collection and augmentation processing are inevitably required in the early stage of training the deep learning model, which is costly. Based on the shortcomings of deep learning processing methods including multi-scale debris detection models, this embodiment also specifically introduces a white thin line extraction and positioning model to compensate for it.

[0028] In this embodiment, since the computer vision-based method usually focuses more on image preprocessing and feature extraction, a specific algorithm can be designed for a specific target. For example, for white thin line debris, the corresponding color and morphological processing can be specialized for its visual bright, white and short features to identify image elements with bright, white and short features, so as to perform corresponding extraction and positioning. Through this computer vision-based white thin line extraction and positioning model, the shortcomings of the deep learning model in processing small sample images can be effectively compensated, and the visibility of white thin lines in the image can be enhanced, so as to extract and locate these thin line features.

[0029] In this embodiment, by performing target debris detection using the first preset detection model and the second preset detection model corresponding to the two models respectively, efficient detection of various debris in the tobacco image can be achieved. Specifically, the first preset detection model is a multi-scale debris detection model, and the second detection model is a white thin line extraction and positioning model, or the second preset detection model is a multi-scale debris detection model, and the first detection model is a white thin line extraction and positioning model. In actual tasks, the front-back relationship of the model can be determined to meet the needs of different types of debris detection. For example, when it is necessary to focus on detecting white thin line debris, the white thin line extraction and positioning model can be used preferentially for detection, because the model is more sensitive and accurate for the detection of small targets. When focusing on detecting other types of debris, such as larger pieces of paper or feathers, the multi-scale debris detection model can be used preferentially for detection, and the accuracy of detection can be improved by multi-scale feature fusion. By using the two models in this way, the comprehensiveness and accuracy of detection can be effectively improved, and the cases of missed detection and false detection can be reduced, thereby improving the purity and production efficiency of tobacco.

[0030] In this embodiment, after determining whether there are foreign objects in the tobacco objects, the tobacco objects that have passed the inspection can be screened, and the tobacco objects that have not passed the inspection can be reported, so as to adjust the production process in time, so that the downstream of the production line can carry out subsequent impurity removal or discarding of the tobacco objects that have not passed the inspection, and carry out subsequent factory processing such as rolling and packaging of the tobacco objects that have passed the inspection. It can effectively improve the production efficiency and automation of products, reduce the losses caused by the mixing of foreign objects, ensure product quality, reduce the need for manual inspection, reduce production costs, and facilitate the realization of intelligent industrial management.

[0031] Optionally, in some embodiments, the tobacco image of each tobacco object covers a roller conveying area for producing the tobacco object.

[0032] In this optional embodiment, the roller conveying area refers to the area where the tobacco is conveyed by the roller during the production process. In this area, the tobacco will undergo a series of processing, such as drying, puffing and flavoring, and finally form a tobacco bundle suitable for cigarettes. Since various debris may be mixed into the tobacco during the transmission process, the tobacco debris detection method can be ensured to fully cover the key links of tobacco production by covering the tobacco image acquisition of the roller conveying area, so as to effectively identify and eliminate the mixed debris and ensure the purity and quality of the tobacco. In addition, through real-time monitoring, abnormal conditions in the production process can be discovered in time, such as a sudden increase in debris, so that rapid response and measures can be taken to avoid production losses and improve production efficiency.

[0033] Optionally, in some embodiments, the multi-scale debris detection model is obtained by training the original detection model based on a training atlas, wherein the training atlas is obtained based on enhancing the original image.

[0034] In this optional embodiment, the original image refers to a sample tobacco image captured by a camera or obtained from a gallery without preprocessing for model training. Enhancement processing includes but is not limited to methods such as image rotation, scaling, cropping, and color adjustment, the purpose of which is to augment the original image and obtain more comprehensive and diverse training data to enhance the generalization ability of the model, so that it can learn more diverse tobacco debris features, thereby improving the accuracy and robustness of detection.

[0035] In this optional embodiment, the original detection model refers to a basic model that has not been trained and optimized with a specific data set in the target detection task. These models may have some basic network structures and functions, but they lack optimization and adjustment for specific tasks and data sets, and therefore cannot be directly applied to actual target detection tasks. The detection model can specifically be a YOLO (You Only Look Once) series model, an SSD (Single Shot MultiBox Detector, single-stage target detector) model, a Faster R-CNN (Faster Region-based Convolutional Neural Networks, faster region-based convolutional neural network) model, etc. that have not been additionally trained. By applying a specific training atlas to these basic models for training, an optimized model for tobacco debris detection, i.e., a multi-scale debris detection model, can be obtained. During the training process, the model can identify various debris in the tobacco image through continuous learning and adjustment, including but not limited to paper pieces, feathers, thin wires, etc., thereby achieving precise control of the purity of the tobacco.

[0036] Optional, see Figure 2In some embodiments, before performing debris detection on each tobacco image based on the first preset detection model, the method further includes: Step 11: Get the original image and add it to the training atlas.

[0037] Step 12: Extract the debris mask in the original image based on the preset segmentation model, replace the background image of the debris mask, and add the obtained new image to the training atlas. And / or Step 13: Visually enhance the original image based on the multi-scale retinal enhancement algorithm with color restoration, and add the obtained image to the training atlas.

[0038] In this optional embodiment, two specific methods for enhancing the tobacco image are described. Specifically, the preset segmentation model is an algorithm model for separating the debris area from the original image, such as the Segment Anything Model (SAM), U-Net (U-shaped network) model, Mask R-CNN (Mask Region-basedConvolutional Neural Network, mask region-based convolutional neural network) model, etc. These models can identify debris in the image and generate corresponding masks. By changing the background image of the mask, the visual effect of debris in different backgrounds can be simulated, thereby enriching the training atlas.

[0039] The Multi-Scale Retinex with Color Restoration (MSRCR) algorithm is an image enhancement technology that aims to improve the visual effect of images under uneven lighting or low-light conditions. It can simulate the sensitivity of the human retina to different scales and colors, and enhance the contrast and color saturation of the image to make the debris features in the image more prominent, which is convenient for model learning and recognition. By introducing a small number of MSRCR-processed images in the training atlas, the model can be guided to be compatible with darker debris scenes, solve the problem of darker tobacco debris being indistinguishable, improve the accuracy and robustness of the model in detecting tobacco debris, and further reduce missed detections and false detections.

[0040] This optional embodiment can further improve the performance of the detection model through the above two enhancement processing methods. In practical applications, the multi-scale debris detection model obtained by training the image data set after enhancement processing can better adapt to various complex production environments and debris types, thereby showing higher accuracy and robustness in actual tobacco debris detection. In addition, by continuously updating and expanding the training atlas, the model can continuously learn and adapt to new debris features, ensuring the long-term effectiveness and adaptability of the detection system, helping to improve the quality control level of the entire tobacco production process, reducing economic losses caused by the mixing of debris, and improving production efficiency and automation.

[0041] Optionally, in some embodiments, before performing debris detection on each tobacco image based on the first preset detection model, the method further includes: scaling and cropping the original image, and adding the obtained new image to the training atlas.

[0042] In this optional embodiment, a data enhancement method is also specifically proposed, that is, scaling and cropping the original image to generate more diverse training samples. The scaling and cropping transformation includes scaling the original image at different proportions, and then selecting a specific area in the scaled image for cropping to obtain a new image. This method can simulate the visual effects of tobacco in different positions and sizes, and increase the adaptability of the model to tobacco debris detection. In this way, the training atlas can be effectively expanded to improve the generalization ability of the model for tobacco debris detection, especially the accuracy when dealing with debris of different scales. In addition, the scaling and cropping transformation can also help the model better identify those debris features that may be ignored in the original image, thereby further reducing the situation of missed detection and false detection. In practical applications, by continuously updating and expanding the training atlas, it can be ensured that the detection model can continue to learn and adapt to new debris features, thereby showing higher accuracy and robustness in actual tobacco debris detection.

[0043] Optionally, in some embodiments, the original detection model is obtained by adding a small target detection layer on the basis of the original YOLOv5 model, and the small target detection layer includes a newly added detection head with a sampling multiple of 4 and a prediction branch for the newly added detection head.

[0044] In this optional embodiment, the original YOLOv5 model is an advanced target detection model, which has a higher detection speed and higher accuracy than other target detection models. In the YOLOv5 model, the size of the input image is usually 640x640. After 32 times, 16 times, and 8 times downsampling, the model will produce feature maps with grid sizes of 20x20, 40x40, and 80x80. Three anchor frames are set for each pixel of these feature maps, and corresponding prediction branches are configured to detect feature targets for feature maps of different sizes. However, the detection heads of these multiples of the original YOLOv5 model can usually only handle larger tobacco debris when detecting tobacco debris in tobacco objects, and are prone to ignoring some smaller tobacco debris.

[0045] In this optional embodiment, a small target detection layer is added to the original detection model to be compatible with smaller tobacco debris. Specifically, the small target detection layer includes a newly added detection head with a sampling multiple of 4 and a prediction branch for the newly added detection head, so as to realize the detection of small-sized debris. Among them, the sampling multiple of the newly added detection head is 4, which means that it can perform more detailed downsampling of the input image, so as to capture smaller-scale features. The prediction branch is responsible for analyzing and identifying these features to distinguish small-sized debris in the tobacco. In this way, the model can more accurately identify those small debris that may be ignored by traditional detection methods, such as thin wires, filaments, etc., thereby further improving the comprehensiveness and accuracy of the detection. In addition, by adding a small target detection layer on the basis of the original YOLOv5 model, the original fast detection advantage of the model can be maintained, while enhancing the detection capability of small-sized debris, which is of great significance for improving the quality control and automation level in the tobacco production process.

[0046] In this optional embodiment, 4 times sampling is used instead of 3 times sampling, 2 times sampling and other lower multiple sampling, because of the consideration of the regularity of the original sampling multiple in the original YOLOv5 model, that is, the regular sampling multiples of 32, 16, 8 and 4 are obtained, and the setting of this regularity is conducive to maintaining the stability and efficiency of model training. And because the output feature map is usually particularly large when the sampling multiple is too low, such as 2 times sampling, for example, a feature map of size 640 is used as the input of the YOLOv5 model, and a feature map of size 320 is output. The feature map obtained under this low sampling multiple is not effectively trained, and the corresponding significant features cannot be extracted. The effect is usually not good, and the model speed and model size must be compatible, so generally only one detection head is added. Therefore, the 4 times sampling adopted in this scheme is a comprehensive consideration of the balance between model performance and efficiency, which is conducive to more efficient and accurate detection of tobacco debris.

[0047] Optional, see Figure 3In some embodiments, when the first preset detection model is a white thin line extraction and positioning model, performing debris detection on each tobacco image based on the first preset detection model, or when the second preset detection model is a white thin line extraction and positioning model, performing debris detection on the tobacco image of the tobacco object to be determined based on the second preset detection model, includes: Step 21: Convert the corresponding tobacco image into the hue-saturation-lightness color space and the hue-brightness-saturation color space respectively, and obtain the debris extraction image corresponding to the white fine line debris in the tobacco image based on the preset foreground hue range, the preset foreground saturation range, the preset foreground lightness range and the preset foreground brightness range.

[0048] Step 22: grayscale conversion and binarization processing are performed on the debris extraction image to obtain a binary debris image.

[0049] Step 23: based on the connection relationship and similarity relationship between pixels in the binary debris image, determine the connected area of ​​the white thin line debris in the tobacco image; Step 24: Determine that the area of ​​the connected region in the tobacco image is greater than a preset debris area threshold, and confirm the corresponding tobacco object as a tobacco object with tobacco debris.

[0050] In this optional embodiment, Hue-Saturation-Value (HSV) color space and Hue-Lightness-Saturation (HLS) color space are two commonly used color spaces. In view of the white and bright characteristics of the white thin line itself, the image is first converted from the RGB color space to the visually balanced HSV color space, and the other color areas of brown, blue and yellow are segmented and eliminated by limiting the upper and lower thresholds of hue, saturation and lightness. The color information is separated from the brightness information to facilitate the extraction of debris within a specific color range, and then the brighter area is screened by setting the brightness threshold in the HLS color space, so that the white thin line area can be extracted. By converting the tobacco image into these color spaces, the white thin line debris can be more accurately located and extracted, and the white thin line debris can be effectively separated from the background, because these debris are significantly different from the tobacco body in hue, saturation and lightness.

[0051] In this optional embodiment, after the white thin line debris area is extracted, the extracted debris image is subjected to grayscale conversion and binarization processing in order to further simplify the image information and highlight the debris area. Grayscale conversion converts a color image into a grayscale image, while binarization processing converts a grayscale image into a black and white image or other two-color image, so that the debris area is one color and the background is another color. This can effectively reduce the complexity of subsequent processing and facilitate the identification of the connected area of ​​the debris through subsequent computer vision algorithm processing.

[0052] In this optional embodiment, by analyzing the connection relationship and similarity between pixels in the binary debris image, the connected area of ​​the white thin line debris can be determined. This step usually involves morphological operations, such as dilation and erosion, and image processing techniques such as connected area marking to clearly separate the debris area from the background. Specifically, the determination of the connected area can be implemented using the connectedComponentsWithStats function in opencv, which is based on the concept of connectivity in graph theory, is used to mark the connected areas in the image, and provides statistical information for each connected area, thereby obtaining the connected area. Once these areas are identified, they can be matched with the actual debris positions in the tobacco image to confirm whether there are debris in the tobacco object.

[0053] In this optional embodiment, by setting a preset debris area threshold, it is possible to determine which connected areas are caused by debris. If the area of ​​the connected area is greater than this threshold, it can be considered that there is debris in the tobacco object. The setting of this threshold needs to be adjusted according to the size and shape of the debris in actual production to ensure the accuracy of the detection. In this way, tobacco objects containing debris can be effectively screened out for subsequent processing to ensure the purity of the tobacco, avoid production losses caused by the mixing of debris, and at the same time improve production efficiency and automation level to ensure the quality of the final product.

[0054] See also Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a tobacco debris detection device provided in the present application.

[0055] The tobacco debris detection device 40 includes: an acquisition module 41, which is used to acquire a tobacco image of at least one tobacco object; a first detection module 42, which is used to perform debris detection on each of the tobacco images based on a first preset detection model, so as to screen out pending tobacco objects that are confirmed to have no tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; a second detection module 43, which is used to perform debris detection on the tobacco images of the pending tobacco objects based on a second preset detection model, so as to screen out inspected tobacco objects that are confirmed to have no tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; wherein the first preset detection model is one of a multi-scale debris detection model and a white fine line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white fine line extraction and positioning model.

[0056] Optionally, in some embodiments, the tobacco image of each tobacco object covers a roller conveying area for producing the tobacco object.

[0057] Optionally, in some embodiments, the multi-scale debris detection model is obtained by training the original detection model based on a training atlas, wherein the training atlas is obtained based on enhancing the original image.

[0058] Optionally, in some embodiments, the tobacco debris detection device 40 also includes an image preprocessing module 44, which is specifically used to: obtain an original image and add the original image to a training atlas; extract a debris mask in the original image based on a preset segmentation model, replace the background image of the debris mask, and add the obtained new image to the training atlas; and / or perform visual enhancement on the original image based on a multi-scale retinal enhancement algorithm with color restoration, and add the obtained image to the training atlas.

[0059] Optionally, in some embodiments, the original detection model is obtained by adding a small target detection layer on the basis of the original YOLOv5 model, and the small target detection layer includes a newly added detection head with a sampling multiple of 4 and a prediction branch for the newly added detection head.

[0060] Optionally, in some embodiments, the image preprocessing module 44 is further specifically used to: perform scaling and cropping transformations on the original image, and add the obtained new image to the training atlas.

[0061] Optionally, in some embodiments, the first detection module 42 or the second detection module 43 is further specifically used for: Step 21: converting the corresponding tobacco image into a hue-saturation-lightness color space and a hue-brightness-saturation color space respectively, and obtaining a debris extraction image corresponding to the white thin line debris in the tobacco image based on a preset foreground hue range, a preset foreground saturation range, a preset foreground lightness range, and a preset foreground brightness range; performing grayscale conversion and binarization on the debris extraction image to obtain a binary debris image; determining a connected area of ​​the white thin line debris in the tobacco image based on a connection relationship and a similarity relationship between pixels in the binary debris image; determining that an area of ​​a connected area in the tobacco image is greater than a preset debris area threshold, and confirming the corresponding tobacco object as a tobacco object with tobacco debris.

[0062] Since the embodiments of the device part correspond to the embodiments of the above-mentioned method, please refer to the above-mentioned method embodiment for the introduction of the tobacco debris detection device 40 provided in the embodiment of the present invention. The embodiment of the present invention will not be repeated here, and it has the same beneficial effects as the above-mentioned tobacco debris detection method.

[0063] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0064] The storage medium 50 stores program data 51. When the program data 51 is executed by the processor, the following is achieved: Figures 1 to 3 The described tobacco foreign matter detection method.

[0065] The program data 51 is stored in a storage medium 50, and includes a number of instructions for enabling a network device (such as a router, a personal computer, a server, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application.

[0066] Optionally, the storage medium 50 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other medium that can store the program data 51 .

[0067] See also Figure 6 , Figure 6 It is a structural schematic diagram of an embodiment of a computer device provided by the present application.

[0068] The computer device 60 includes a processor 62 and a memory 61 connected to each other. The memory 61 stores a computer program. When the processor 62 executes the computer program, the following is achieved: Figures 1 to 3 The described tobacco impurity detection method. The memory 61 may include the storage medium 50, or may be other independently developed memory.

[0069] Different from the prior art, the present application discloses a method, device, medium and equipment for detecting tobacco debris. Based on the multi-scale debris detection model and the white fine line extraction and positioning model, the tobacco images of the acquired tobacco objects are respectively detected for debris, and the tobacco objects that have passed the inspection and the tobacco objects that need to be reported due to tobacco debris are determined. By using the multi-scale debris detection model and the white fine line extraction and positioning model, various debris in the tobacco can be identified and located more efficiently and accurately, and darker and smaller white fine line tobacco debris that are difficult to identify in traditional methods can be effectively detected in the tobacco image, thereby reducing missed detection and false detection, improving the automation level of cigarette production, and avoiding the problems of high labor intensity and being greatly affected by manual experience and subjective factors in traditional manual detection methods, which is conducive to improving the quality and taste of cigarette products, reducing production costs, and meeting the market's higher requirements for tobacco purity.

[0070] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, storage medium embodiment and computer equipment embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0071] The present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed methods, devices, storage media and computer equipment can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0074] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] The above are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting tobacco impurities, characterized in that: include: Acquire a tobacco image of at least one tobacco object; Performing debris detection on each of the tobacco images based on the first preset detection model to screen out pending tobacco objects that are confirmed to have no tobacco debris, and reporting the tobacco objects that are confirmed to have tobacco debris; Performing foreign matter detection on the tobacco image of the pending tobacco object based on the second preset detection model to screen out the tobacco objects that have been inspected and are confirmed to have no tobacco foreign matter, and reporting the tobacco objects that have been confirmed to have tobacco foreign matter; The first preset detection model is one of a multi-scale debris detection model and a white thin line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white thin line extraction and positioning model.

2. The tobacco impurity detection method according to claim 1, characterized in that: The multi-scale debris detection model is obtained by training an original detection model based on a training atlas, wherein the training atlas is obtained by enhancing the original image.

3. The tobacco impurity detection method according to claim 2, characterized in that: Before performing debris detection on each of the tobacco images based on the first preset detection model, the method further includes: Acquire the original image, and add the original image to the training atlas; Extracting a debris mask from the original image based on a preset segmentation model, replacing the background image of the debris mask, and adding the obtained new image to the training atlas; and / or The original image is visually enhanced based on a multi-scale retinal enhancement algorithm with color restoration, and the obtained image is added to the training atlas.

4. The tobacco impurity detection method according to claim 2, characterized in that: The original detection model is obtained by adding a small target detection layer on the basis of the original YOLOv5 model. The small target detection layer includes a newly added detection head with a sampling multiple of 4 and a prediction branch for the newly added detection head.

5. The tobacco impurity detection method according to claim 2, characterized in that: Before performing debris detection on each of the tobacco images based on the first preset detection model, the method further includes: The original image is scaled and cropped, and the resulting new image is added to the training atlas.

6. The tobacco impurity detection method according to claim 1, characterized in that: When the first preset detection model is the white thin line extraction and positioning model, the performing of debris detection on each of the tobacco images based on the first preset detection model, or when the second preset detection model is the white thin line extraction and positioning model, the performing of debris detection on the tobacco image of the to-be-determined tobacco object based on the second preset detection model, comprises: The corresponding tobacco image is converted into a hue-saturation-lightness color space and a hue-brightness-saturation color space respectively, and based on a preset foreground hue range, a preset foreground saturation range, a preset foreground lightness range, and a preset foreground brightness range, a debris extraction image corresponding to the white thin line debris in the tobacco image is obtained; Performing grayscale conversion and binarization processing on the debris extraction image to obtain a binary debris image; Determine the connected area of ​​the white thin line debris in the tobacco image based on the connection relationship and similarity relationship between pixels in the binary debris image; It is determined that the area of ​​the connected region in the tobacco image is greater than a preset debris area threshold, and the corresponding tobacco object is confirmed as a tobacco object with tobacco debris.

7. The tobacco impurity detection method according to claim 1, characterized in that: The tobacco cut image of each of the tobacco cut objects covers the roller conveying area where the tobacco cut objects are produced.

8. A tobacco debris detection device, characterized in that: include: An acquisition module, used to acquire a tobacco image of at least one tobacco object; A first detection module is used to perform debris detection on each of the tobacco images based on a first preset detection model to screen out pending tobacco objects that are confirmed to have no tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; A second detection module is used to perform debris detection on the tobacco image of the pending tobacco object based on a second preset detection model, so as to screen out the inspected tobacco objects that are confirmed to have no tobacco debris, and report the tobacco objects that are confirmed to have tobacco debris; The first preset detection model is one of a multi-scale debris detection model and a white thin line extraction and positioning model, and the second preset detection model is the other of the multi-scale debris detection model and the white thin line extraction and positioning model.

9. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the tobacco impurity detection method according to any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the tobacco impurity detection method according to any one of claims 1 to 7 are implemented.

Images