Cigar defect detection method, device, equipment, storage medium and program product

Abstract

The invention discloses a cigar defect detection method, device and equipment, a storage medium and a program product. The method comprises the following steps: acquiring a cigarette image of a to-be-detected cigar; calling a pre-trained first defect detection model to detect the cigarette image to obtain a first detection result of the cigar to be detected; the first detection result is a non-defective cigarette or a defective cigarette; under the condition that the first detection result is a non-defective cigarette, calling a pre-trained second defect detection model to detect the cigarette image to obtain a second detection result, and taking the second detection result as a target detection result of the to-be-detected cigar; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result comprises any one of the following items: a non-defective cigarette, a non-full cigarette, a flash cigarette, a surface-damaged cigarette and a cigarette without a cigar coat. By adopting the method, the production efficiency and the product quality of cigars can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a cigar defect detection method, apparatus, device, storage medium, and program product. Background Art

[0002] The production of machine-made cigars consists of three steps: rolling, cutting, and packaging. Each step is completed independently by different machines. Before the rolling and cutting processes are complete, the quality of the cigars is unstable, with common problems such as damaged wrappers, incomplete wrapping, and undersaturated tobacco appearing. Since all potential quality issues have already been produced, new quality issues are less likely to arise by the time the cigarettes enter the packaging machine. Therefore, performing quality inspection on the packaging machine can effectively control the quality of cigars as they are packaged.

[0003] Traditionally, cigar quality inspection is mainly done through manual screening, which is not only inefficient but also carries the risk of missed or incorrect detection, resulting in low cigar production efficiency and product quality.

[0004] Therefore, how to improve the production efficiency and product quality of cigars has become an urgent problem to be solved. Summary of the Invention

[0005] The embodiments of the present application provide a cigar defect detection method, apparatus, device, storage medium, and program product, which are beneficial to improving the production efficiency and product quality of cigars.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting defects in cigars, the method comprising:

[0007] Obtaining an image of a cigar to be detected;

[0008] Invoking a pre-trained first defect detection model to detect the cigarette image and obtain a first detection result of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; the first detection result is a non-defective cigarette or a defective cigarette;

[0009] When the first detection result is a non-defective cigarette, a pre-trained second defect detection model is called to detect the cigarette image to obtain a second detection result, and the second detection result is used as the target detection result for the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0010] In one embodiment, the method further includes: obtaining N sample cigarette images corresponding to a plurality of sample cigars, and filtering the N sample cigarette images to obtain M sample cigarette images; N and M are both positive integers, M≤N; obtaining first actual categories corresponding to the M sample cigarette images, respectively, the first actual category including defective cigarettes or non-defective cigarettes; calling a pre-constructed first initial defect detection model, and obtaining a first predicted category corresponding to each sample cigarette image based on the M sample cigarette images; the first initial defect detection model is constructed based on partial convolution; the first initial defect detection model is trained in the direction of reducing the difference between the first actual category and the first predicted category of each sample cigarette image to obtain a first defect detection model.

[0011] In one embodiment, filtering is performed on N sample cigarette images to obtain M sample cigarette images, including: filtering out a plurality of non-defective sample cigarette images from the N sample cigarette images to obtain M sample cigarette images.

[0012] In one embodiment, a pre-built first initial defect detection model is called to obtain a first prediction category corresponding to each sample cigarette image based on M sample cigarette images, including: performing data labeling and normalization processing on the M sample cigarette images to obtain M processed sample cigarette images; and inputting the M processed sample cigarette images into the pre-built first initial defect detection model to obtain a first prediction category corresponding to each sample cigarette image.

[0013] In one embodiment, the method further includes: constructing a second initial defect detection model based on the first defect detection model; migrating the model parameters in the first defect detection model to the second initial defect detection model to obtain a second intermediate defect detection model; obtaining second actual categories corresponding to M sample cigarette images, respectively, where the second actual categories are obtained by labeling the M sample cigarette images based on a preset labeling strategy; the second actual category is any one of the following: non-defective cigarettes, incomplete cigarettes, flash cigarettes, surface damaged cigarettes, and wrapperless cigarettes; inputting the M sample cigarette images into the second intermediate defect detection model to obtain second predicted categories corresponding to the M sample cigarette images; training the second intermediate defect detection model in the direction of reducing the difference between the second actual category and the second predicted category of each sample cigarette image to obtain a second defect detection model.

[0014] In one embodiment, the method further includes: for each sample cigarette image in the M sample cigarette images, when the ratio between the hollow area of ​​the target sample cigar corresponding to the sample cigarette image and the total area of ​​the target sample cigar is greater than a preset ratio, and the hollow depth is greater than a preset depth threshold, determining that the second actual category corresponding to the sample cigarette image is an unfilled cigarette; when the wrapper of the target sample cigar corresponding to the sample cigarette image has flaky shedding, determining that the second actual category corresponding to the sample cigarette image is a flash cigarette; when the damaged area of ​​the target sample cigar corresponding to the sample cigarette image is greater than a preset area threshold, determining that the second actual category corresponding to the sample cigarette image is a surface-damaged cigarette; when the target sample cigar corresponding to the sample cigarette image has no wrapper, determining that the second actual category corresponding to the sample cigarette image is a wrapper-less cigarette.

[0015] In a second aspect, an embodiment of the present application provides a cigar defect detection device, the device comprising:

[0016] An acquisition module, used to acquire an image of a cigar to be detected;

[0017] A first detection module is configured to call a pre-trained first defect detection model to detect the cigarette image and obtain a first detection result of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; and the first detection result is a non-defective cigarette or a defective cigarette;

[0018] The second detection module is used to call a pre-trained second defect detection model to detect the cigarette image when the first prediction result indicates that the cigar to be inspected is not defective, obtain a second detection result, and use the second detection result as the target detection result for the cigar to be inspected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following items: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0019] In a third aspect, an embodiment of the present application provides a detection device, including a data processing module, which includes a processor and a memory, the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method provided in the first aspect above.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method provided in the first aspect when the computer program is executed by a processor.

[0021] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect above.

[0022] The above-mentioned cigar defect detection method, device, equipment, storage medium and program product, the detection equipment can obtain the image of the cigar to be detected; call the pre-trained first defect detection model to detect the cigarette image, and obtain a first detection result of the cigar to be detected; wherein, the first defect detection model is a lightweight neural network model; the first detection result is a non-defective cigarette, or a defective cigarette; when the first detection result is a non-defective cigarette, call the pre-trained second defect detection model to detect the cigarette image, obtain a second detection result, and use the second detection result as the target detection result of the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following: non-defective cigarettes, incomplete cigarettes, flash cigarettes, surface damaged cigarettes, and cigarettes without cigar wrappers. By adopting this method, a lightweight first defect detection model is used to perform preliminary detection on the image of the cigar to be inspected, so as to quickly determine whether the cigar to be inspected is a defective cigarette. If the first detection result is a non-defective cigarette, an end-to-end second defect detection model trained based on the first defect detection model is used to perform secondary detection on the image of the cigar to be inspected, so as to accurately determine the defect category of the cigar to be inspected. Therefore, this method can improve the efficiency and accuracy of cigar defect detection, thereby helping to improve the production efficiency and product quality of cigars. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a schematic structural diagram of a detection device provided in an embodiment of the present application;

[0025] Figure 2 1 is a flow chart of a cigar defect detection method provided in an embodiment of the present application;

[0026] Figure 3 This is a schematic structural diagram of a first initial defect detection model provided in an embodiment of the present application;

[0027] Figure 4 is a structural diagram of a second initial defect detection model provided in an embodiment of the present application;

[0028] Figure 51 is a flow chart of another cigar defect detection method provided in an embodiment of the present application;

[0029] Figure 6 1 is a schematic structural diagram of a cigar defect detection device provided in an embodiment of the present application;

[0030] Figure 7 It is a structural schematic diagram of another detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] See Figure 1 , Figure 1 This is a schematic diagram of the structure of a detection device provided in an embodiment of the present application. Figure 1 As shown, Figure 1 As shown, the detection device may include but is not limited to: an end surface detection unit 101, an appearance detection unit 102, an inner lining paper detection unit 103, a single box count detection unit 104, and a data processing unit 105. The end surface detection unit 101, the appearance detection unit 102, the inner lining paper detection unit 103, and the single box count detection unit 104 are all connected to the data processing unit 105.

[0033] The end-face detection unit 101 is equipped with two acquisition terminals for collecting data from the two end faces of a cigar. The appearance detection unit 102 is equipped with two acquisition terminals for collecting data from the upper and lower surfaces of a cigar. The liner paper detection unit 103 is equipped with a single acquisition terminal for collecting data from the liner paper of a boxed cigar. The number of cigars per box detection unit 104 is equipped with a single acquisition terminal for collecting data on the number of cigars in a box. The data processing unit 105 is responsible for processing various types of data.

[0034] In some embodiments, each of the above-mentioned acquisition terminals may include a detection camera and a detection light source, and the detection camera may transmit the collected data to the data processing unit 105 .

[0035] The inspection camera can be a high-speed area scan camera, which captures real-time images at a frame rate of 560 fps. All images are captured based on a packaging machine production speed of 420 cigarettes per minute. The maximum frame rate of the area scan camera used is greater than a preset multiple (e.g., 10 times) of the cigarette production speed to capture images of the cigarette filter. Based on the cigar packaging machine's production rhythm, real-time images of both ends of the cigar are captured using an external trigger. A higher frame rate results in a shorter minimum exposure time, less smear during high-speed photography, and higher algorithmic accuracy.

[0036] The detection light source can use a 24V light source controller to control the light source and cooperate with the camera to collect images. The light source used can be a white cold light source. According to the production rhythm of the cigar packaging machine, an external trigger is used to trigger the light source for short-term exposure, which can enable the high-speed area array camera to collect high-quality images.

[0037] In some embodiments, the data processing unit 105 may include an industrial computer and an I / O board. The I / O board may utilize a high-speed digital I / O module to collect encoder and photoelectric sensor information. It may also output control signals to control camera image acquisition, control the light source controller and light exposure, output a stop signal to stop the packaging machine, and output a reject signal to reject individual cigars. The industrial computer may include 11 Universal Serial Bus (USB) interfaces and a 12G graphics card. The graphics card can be used to detect and process cigarette images transmitted by the detection camera at the acquisition end to obtain detection results.

[0038] In some embodiments, the end face detection unit 101, the appearance detection unit 102, the liner paper detection unit 103, and the number of cigars in a single box detection unit 104 can be used to collect images of cigars to be detected; the data processing unit 105 can be used to obtain images of cigars to be detected; a pre-trained first defect detection model is called to detect the cigarette images to obtain a first detection result of the cigars to be detected; wherein the first defect detection model is a lightweight neural network model; the first detection result is a non-defective cigarette or a defective cigarette; when the first detection result is a non-defective cigarette, a pre-trained second defect detection model is called to detect the cigarette image to obtain a second detection result, and the second detection result is used as the target detection result of the cigars to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following: non-defective cigarettes, incomplete cigarettes, flash-edged cigarettes, surface-damaged cigarettes, and cigarettes without cigar wrappers. By adopting this embodiment, a lightweight first defect detection model is used to perform preliminary detection on the image of the cigar to be inspected, so as to quickly determine whether the cigar to be inspected is a defective cigarette. If the first detection result is a non-defective cigarette, an end-to-end second defect detection model trained based on the first defect detection model is used to perform secondary detection on the image of the cigar to be inspected, so as to accurately determine the defect category of the cigar to be inspected. Therefore, the use of this embodiment can improve the efficiency and accuracy of cigar defect detection, thereby helping to improve the production efficiency and product quality of cigars.

[0039] The following describes the cigar defect detection method provided in the embodiments of the present application.

[0040] See Figure 2 , Figure 2 This is a flow chart of a cigar defect detection method provided by an embodiment of the present application. The method can be performed by (for example) Figure 1 As shown in the test equipment). Figure 2 As shown, the cigar defect detection method may include but is not limited to the following steps:

[0041] S201: Acquire an image of a cigar to be detected.

[0042] The image of the cigar to be detected may include but is not limited to an end face image, a surface image, etc. of the cigar to be detected, which is not limited here.

[0043] In an optional embodiment, the detection device obtains the image of the cigar to be detected by Figure 1 The collection end provided in the end face detection unit 101, the appearance detection unit 102, the inner lining paper detection unit 103 and the number of cigars in a single box detection unit 104 collects the image of the cigar to be detected.

[0044] S202: Call a pre-trained first defect detection model to detect the cigarette image to obtain a first detection result of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; the first detection result is a non-defective cigarette or a defective cigarette.

[0045] In an optional implementation, the first defect detection model may be stored in a detection device (eg, a data processing unit of the detection device) or in a database server, which is not limited here.

[0046] In which case, when the first defect detection model is stored in the database server, the detection device calls the pre-trained first defect detection model, that is, the detection device obtains the pre-trained first defect detection model from the database server.

[0047] S203. When the first detection result is a non-defective cigarette, call a pre-trained second defect detection model to detect the cigarette image to obtain a second detection result, and use the second detection result as the target detection result for the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0048] In an optional embodiment, after step S203, the detection device may further output a target detection result of the cigar to be detected, so that relevant staff can take appropriate repair measures for the cigar to be detected based on the target detection result.

[0049] In an optional embodiment, the inspection device can also output the first inspection result if the first inspection result indicates a defective cigarette. This allows relevant personnel to remove cigars to be inspected from the production line based on the first inspection result, thereby improving the product quality of cigars.

[0050] In some embodiments, after outputting the first inspection result, the inspection device may also utilize a pre-trained second defect detection model to perform a secondary inspection on the cigarette image to obtain a second inspection result. The second inspection result may include any of the following: non-defective cigarettes, underfilled cigarettes, flashed cigarettes, surface damaged cigarettes, and cigarettes without wrappers. This second inspection result facilitates personnel to take appropriate repair measures for the inspected cigars based on the second inspection result.

[0051] In an embodiment of the present application, the detection device can obtain an image of a cigar to be detected; call a pre-trained first defect detection model to detect the cigarette image, and obtain a first predicted category of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; when the first predicted category is a non-defective cigarette, call a pre-trained second defect detection model to detect the cigarette image, and obtain a second predicted category; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model, and the second predicted defect category is used as the target category of the cigar to be detected; the target category includes any of the following: non-defective cigarettes, underfilled cigarettes, flash-edged cigarettes, surface-damaged cigarettes, and cigarettes without cigar wrappers. Using this method, since the first defect detection model is a lightweight neural network model, the first defect detection model is used to detect the cigarette image of the cigar to be inspected, thereby improving the detection efficiency. When the output result of the first defect detection model (i.e., the first prediction category) is a non-defective cigarette, the cigarette image is subjected to secondary inspection by using an end-to-end second defect detection model trained based on the first defect detection model, thereby improving the accuracy of the detection result. That is, this method can improve the efficiency and accuracy of cigar defect detection, thereby helping to improve the production efficiency and product quality of cigars.

[0052] In an optional embodiment, Figure 2 In the cigar defect detection method shown, the first defect detection model can be trained by the detection device in the following manner: obtaining N sample cigarette images corresponding to multiple sample cigars, and filtering the N sample cigarette images to obtain M sample cigarette images; N and M are both positive integers, M≤N; obtaining the first actual categories corresponding to the M sample cigarette images, where the first actual categories include defective cigarettes or non-defective cigarettes; calling a pre-constructed first initial defect detection model, and based on the M sample cigarette images, obtaining the first predicted category corresponding to each sample cigarette image; the first initial defect detection model is constructed based on partial convolution; and the first initial defect detection model is trained in the direction of reducing the difference between the first actual category and the first predicted category of each sample cigarette image to obtain the first defect detection model.

[0053] In some embodiments, the N sample cigarette images corresponding to the multiple sample cigars may be obtained by the detection device from a database server. The N sample cigarette images may be captured by a user using a detection camera from multiple sample cigars at a cigar factory production site and then sent to the database server. Optionally, the pixel size of each of the N sample cigarette images may be 640*640.

[0054] In some embodiments, the detection device filters N sample cigarette images to obtain M sample cigarette images. This can be accomplished by filtering out multiple non-defective sample cigarette images from the N sample cigarette images to obtain the M sample cigarette images. This helps reduce the risk of overfitting during subsequent training of the first defect detection model, thereby improving the detection performance of the first defect detection model and enhancing the accuracy of the detection results.

[0055] Optionally, the detection device filters out multiple non-defective sample cigarette images from N sample cigarette images to obtain M sample cigarette images, which may be by obtaining the first actual categories corresponding to the N sample cigarette images respectively; displaying the first actual categories corresponding to the N sample cigarette images respectively on the user interface; and in response to a filtering operation input for multiple sample cigarette images whose first actual category is the non-defective category in the user interface, filtering out multiple non-defective sample cigarette images from the N sample cigarette images to obtain M sample cigarette images.

[0056] In some embodiments, the first actual categories corresponding to the M sample cigarette images can be obtained after the detection device performs labeling processing on the M sample cigarette images respectively.

[0057] In some embodiments, the detection device invokes a pre-built first initial defect detection model and, based on M sample cigarette images, obtains a first predicted category corresponding to each sample cigarette image. This includes: performing data labeling and normalization on each of the M sample cigarette images to obtain M processed sample cigarette images; and inputting the M processed sample cigarette images into the pre-built first initial defect detection model to obtain a first predicted category corresponding to each sample cigarette image. This helps narrow the differences between features and facilitates direct comparison and in-depth analysis of different features.

[0058] The detection device performs data labeling and normalization on each of the M sample cigarette images. This can be done using the LabelImg (an open-source image labeling tool) toolkit, which labels the M sample cigarette images and generates an XML-formatted label file containing precise label box coordinates and label category information. The label box coordinates and label category information in the label file are then normalized, and the M sample cigarette images are processed based on the normalized label box coordinates and label category information to obtain M processed sample cigarette images. By mapping the label information to a preset data interval, the differences between features can be narrowed, facilitating direct comparison and in-depth analysis of different features.

[0059] The first initial defect detection model can be the FasterNet model (a lightweight and efficient neural network model) based on partial convolution (PConv). PConv is a variant of convolutional neural networks. Its core concept is to dynamically adjust convolution calculations to rely only on valid pixels in the input, thereby ignoring the impact of missing or masked areas. FasterNet aims to achieve higher computational efficiency and faster inference speed while maintaining high accuracy.

[0060] For example, see Figure 3 , Figure 3 This is a schematic diagram of the structure of a first initial defect detection model provided by an embodiment of the present application. Figure 3 As shown, the first initial defect detection model is a FasterNet model including a 3*3 PConv layer and a 1*1 convolution layer.

[0061] In some embodiments, the detection device trains the first initial defect detection model to reduce the difference between the first actual category and the first predicted category of each sample cigarette image, thereby obtaining the first defect detection model. This training may be performed using a strongly supervised training method based on the first actual category and the first predicted category of each sample cigarette image, with the goal of determining the minimum value of a preset loss function. This can enhance the first defect detection model's anti-interference capability and improve detection accuracy.

[0062] The expression of the preset loss function can be shown as the following formula (1).

[0063] (1)

[0064] In formula (1), N represents the total number of images in the cigar detection model training process, that is, the total number of sample cigarette images; Indicates the The probability that the model prediction corresponding to the sample cigarette image is accurate; Represents the binary variable predicted by the model. When the cigar defect feature is a positive sample (i.e., a non-defective cigarette), its value is 1; when the cigar defect feature label is a negative sample (i.e., a defective cigarette), its value is 0; Representative The regression parameters of the predicted feature label box corresponding to the sample cigarette image; Indicates the The value of the true calibration frame corresponding to the predicted feature label frame of the sample cigarette image; Represents the total number of images required for the first epoch training, ranging from 0 to 256; Indicates the total number of predicted feature marker boxes, ranging from 0 to 2400; It represents the classification loss function, which can be determined by the following formula (2); It represents the regression loss function of predicting the feature label box, which can be determined by the following formula (3).

[0065] (2)

[0066] (3)

[0067] The physical meanings of the parameters in formulas (2) and (3) can be found in the description of formula (1) above and will not be repeated here.

[0068] Using this embodiment, the detection device trains a pre-constructed first initial defect detection model based on N sample cigarette images corresponding to multiple sample cigars, and can obtain a trained first defect detection model. This is conducive to the subsequent use of the trained first defect detection model to accurately determine the first detection result of the cigar to be detected.

[0069] In an optional embodiment, Figure 2 In the cigar defect detection method shown, the second defect detection model can be trained by the detection device in the following manner: based on the first defect detection model, a second initial defect detection model is constructed; the model parameters in the first defect detection model are transferred to the second initial defect detection model to obtain a second intermediate defect detection model; the second actual categories corresponding to M sample cigarette images are obtained, and the second actual categories are obtained by labeling the M sample cigarette images based on a preset labeling strategy; the second actual category is any one of the following: non-defective cigarettes, incomplete cigarettes, flash cigarettes, surface damaged cigarettes, and cigarettes without wrappers; the M sample cigarette images are input into the second intermediate defect detection model to obtain the second predicted categories corresponding to the M sample cigarette images; the second intermediate defect detection model is trained in the direction of reducing the difference between the second actual category and the second predicted category of each sample cigarette image to obtain a second defect detection model.

[0070] For example, the first defect detection model may be a FasterNet model. In this case, the second initial defect detection model constructed by the detection device based on the first defect detection model may be as follows: Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a second initial defect detection model provided by an embodiment of the present application. Figure 4 As shown, the FasterNet network is included. The second initial defect detection model can also be called a YOLOv8 model based on FasterNet.

[0071] In some embodiments, the detection device may also, for each sample cigarette image in the M sample cigarette images, determine that the second actual category corresponding to the sample cigarette image is an unfilled cigarette if the ratio between the hollow area of ​​the target sample cigar corresponding to the sample cigarette image and the total area of ​​the target sample cigar is greater than a preset ratio, and the hollow depth is greater than a preset depth threshold; determine that the second actual category corresponding to the sample cigarette image is a fin-shaped cigarette if the wrapper of the target sample cigar corresponding to the sample cigarette image has flaky shedding; determine that the second actual category corresponding to the sample cigarette image is a surface-damaged cigarette if the damaged area of ​​the target sample cigar corresponding to the sample cigarette image is greater than a preset area threshold; and determine that the second actual category corresponding to the sample cigarette image is a cigarette without a wrapper if the target sample cigar corresponding to the sample cigarette image has no wrapper.

[0072] For example, assuming that the preset ratio is 1 / 3, the preset depth threshold is 1mm, the ratio between the sunken area of ​​the target sample cigar corresponding to the sample cigarette image 1 and the total area of ​​the target sample cigar 1 is 2 / 5, and the sunken depth is 1.2mm. In this case, the detection equipment can determine that the ratio of the sunken area of ​​the target sample cigar 1 corresponding to the sample cigarette image 1 to the total area of ​​the target sample cigar 1 is 2 / 5, which is greater than the preset ratio 1 / 3, and the sunken depth of 1.2mm is greater than the preset depth threshold 1mm. At this time, the detection equipment can determine that the second actual category corresponding to the sample cigarette image 1 is an unfilled cigarette.

[0073] For example, if the criterion for determining whether the wrapper is flaky is that the area of ​​the wrapper is greater than or equal to 3mm. 2 Assume that the target sample cigar 2 corresponding to the sample cigarette image 2 has a cigar wrapper shedding area of ​​3.5mm 2 In this case, the detection device can determine that the target sample cigar 2 corresponding to the sample cigarette image 2 has flaky strips of wrapper falling off. At this time, the detection device can determine that the second actual category corresponding to the sample cigarette image 2 is a flash cigarette.

[0074] For another example, suppose the preset area threshold is 3mm 2 Assume that the damaged area of ​​target sample cigar 3 corresponding to sample cigarette image 3 is 3.2mm 2 In this case, the detection device can determine that the damaged area of ​​the target sample cigar 3 corresponding to the sample cigarette image 3 is 3.2mm 2 Larger than the preset area threshold of 3mm 2 At this time, the detection device can determine that the second actual category corresponding to the sample cigarette image 3 is a surface damaged cigarette.

[0075] By adopting this embodiment, on the one hand, the second initial defect detection model constructed based on the first defect detection model is trained using transfer learning technology, which can simplify the training steps of the second defect detection model, improve training efficiency, and thus reduce resource consumption; on the other hand, by training the second initial defect detection model, a second defect detection model with better detection effect can be obtained, which is conducive to subsequently determining the detection results of the cigars to be detected more accurately based on the second defect detection model.

[0076] The following combination Figure 5 , the overall process of the cigar defect detection method provided in the embodiment of the present application is described. Figure 5 , Figure 5 is a flow chart of another cigar defect detection method provided by an embodiment of the present application, which can be performed by a detection device (e.g. Figure 1 As shown in the test equipment). Figure 6 As shown, the cigar defect detection method may include but is not limited to the following steps:

[0077] S501. Obtain N sample cigarette images corresponding to a plurality of sample cigars, and filter the N sample cigarette images to obtain M sample cigarette images; N and M are both positive integers, M≤N.

[0078] In some embodiments, the sample cigarette image corresponding to each sample cigar may include but is not limited to an end face image, an appearance image (or surface image), and the like.

[0079] The detection device filters N sample cigarette images to obtain M sample cigarette images, which may be achieved by filtering out multiple non-defective sample cigarette images from the N sample cigarette images to obtain M sample cigarette images.

[0080] S502. Label the M sample cigarette images to obtain M classified sample cigarette images, a first label corresponding to each sample cigarette image, and a second label corresponding to each sample cigarette image; wherein the first label is any one of the following: a non-defective cigarette, a half-filled cigarette, a cigarette with a flashing edge, a cigarette with a damaged surface, or a cigarette without a wrapper; and the second label is a non-defective cigarette or a defective cigarette.

[0081] Among them, the marking processing can be that the detection device marks M sample cigarette images respectively based on a preset marking strategy to obtain the first labels corresponding to the M sample cigarette images respectively; based on the first labels, determine the second labels corresponding to the M sample cigarette images respectively.

[0082] Optionally, the preset marking strategy may include: (1) when the hollow area of ​​the target sample cigar corresponding to the sample cigarette image exceeds 1 / 3 of the total area of ​​the target sample cigar, and the hollow depth exceeds 1mm, the sample cigarette image is determined to be an unfilled cigarette; (2) when the wrapper of the target sample cigar corresponding to the sample cigarette image is flaky (the wrapper detachment area is greater than or equal to 3mm 2 ) in the case of the sample cigarette image being determined to be a flash cigarette; (3) when the damage area of ​​the target sample cigar corresponding to the sample cigarette image is greater than 3mm 2 In the case of the sample cigarette image being a damaged cigarette; (4) in the case of the target sample cigar corresponding to the sample cigarette image being without a wrapper, the sample cigarette image being determined to be a cigarette without a wrapper.

[0083] In an optional embodiment, the detection device determines the second labels corresponding to the M sample cigarette images based on the first labels. For example, if the first label of any sample cigarette image indicates a non-defective cigarette, the second label of the sample cigarette image is determined to be a non-defective cigarette; and if the first label of any sample cigarette image indicates an underfilled cigarette, a cigarette with a flashing edge, a cigarette with a damaged surface, or a cigarette without a wrapper, the second label of the sample cigarette image is determined to be a defective cigarette. In other words, the second label of the sample cigarette image indicates either a non-defective cigarette or a defective cigarette.

[0084] In some embodiments, the detection device may further divide the M sample cigarette images into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set is used to train the initial defect detection model constructed subsequently, the validation set is used to optimize the trained defect detection model, and the test set is used to conduct a final evaluation of the generalization ability of the optimized defect detection model.

[0085] S503: An initial FasterNet model is constructed based on partial convolution, and the initial FasterNet model is called to obtain a first predicted category corresponding to each sample cigarette image based on M sample cigarette images.

[0086] Among them, the first prediction category is a defective cigarette or a non-defective cigarette.

[0087] S504 : Train the first initial defect detection model in a direction of reducing the difference between the second label and the first predicted category of each sample cigarette image to obtain a trained FasterNet model.

[0088] S505: Build an initial YOLOv8 model based on the FasterNet model, and migrate the model parameters in the trained FasterNet model to the initial YOLOv8 model to obtain an intermediate YOLOv8 model.

[0089] S506: Input the M sample cigarette images into the intermediate YOLOv8 model to obtain the second predicted categories corresponding to the M sample cigarette images.

[0090] Among them, the second prediction category is any one of the following: non-defective cigarettes, partially filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0091] S507 . Train the intermediate YOLOv8 model in a direction of reducing the difference between the second actual category and the first label of each sample cigarette image to obtain a trained YOLOv8 model.

[0092] S508: Acquire an image of the cigar to be detected.

[0093] S509: Call the trained FasterNet model to detect the cigarette image to obtain a first detection result of the cigar to be detected; the first detection result is a non-defective cigarette or a defective cigarette.

[0094] S510. When the first detection result is a non-defective cigarette, call the trained YOLOv8 model to detect the cigarette image to obtain a second detection result, and use the second detection result as the target detection result of the cigar to be detected; the target detection result includes any of the following: non-defective cigarettes, unfilled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0095] In an optional implementation, the relevant descriptions of steps S508 to S510 can refer to the descriptions of the aforementioned steps S201 to S203, and will not be repeated here.

[0096] By adopting this method, the detection equipment can obtain a trained FasterNet model by training the initial FasterNet model; then, an initial YOLOv8 model is constructed based on the trained FasterNet model; the model parameters in the trained FasterNet model are transferred to the initial YOLOv8 model to obtain an intermediate YOLOv8 model, and the intermediate YOLOv8 model is trained to obtain a trained YOLOv8 model. In this way, by using the transfer learning technology to train the initial YOLOv8 model constructed based on the trained FasterNet model, the training steps of the second defect detection model can be simplified, the training efficiency can be improved, and thus resource consumption can be reduced; thereafter, the trained FasterNet model and the trained YOLOv8 model are used to perform defect detection on the cigars to be inspected, which can not only improve the detection efficiency, but also improve the accuracy of the detection results, thereby helping to improve the production efficiency and product quality of cigars.

[0097] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0098] Based on the same inventive concept, embodiments of the present application also provide a cigar defect detection device for implementing the aforementioned cigar defect detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the cigar defect detection device can be found in the aforementioned limitations of the cigar defect detection method and will not be further elaborated here.

[0099] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a cigar defect detection device provided in an embodiment of the present application. Figure 6 As shown, the cigar defect detection device may include but is not limited to:

[0100] An acquisition module 601 is used to acquire an image of a cigar to be detected;

[0101] A first detection module 602 is configured to call a pre-trained first defect detection model to detect the cigarette image and obtain a first detection result of the cigar to be detected. The first defect detection model is a lightweight neural network model. The first detection result is a non-defective cigarette or a defective cigarette.

[0102] The second detection module 603 is used to call a pre-trained second defect detection model to detect the cigarette image when the first prediction result indicates that the cigar to be detected is a non-defective cigarette, obtain a second detection result, and use the second detection result as the target detection result of the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any of the following items: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

[0103] In some embodiments, the device may further include a training module. The training module is configured to obtain N sample cigarette images corresponding to a plurality of sample cigars, and filter the N sample cigarette images to obtain M sample cigarette images, where N and M are both positive integers, M≤N; obtain first actual categories corresponding to the M sample cigarette images, where the first actual categories include defective cigarettes or non-defective cigarettes; invoke a pre-built first initial defect detection model, and based on the M sample cigarette images, obtain a first predicted category corresponding to each sample cigarette image; the first initial defect detection model is constructed based on partial convolution; and the first initial defect detection model is trained in a direction that reduces the difference between the first actual category and the first predicted category of each sample cigarette image to obtain a first defect detection model.

[0104] In some embodiments, when the training module is used to filter N sample cigarette images to obtain M sample cigarette images, it is specifically used to: filter out multiple non-defective sample cigarette images from the N sample cigarette images to obtain M sample cigarette images.

[0105] In some embodiments, when the training module is used to call a pre-built first initial defect detection model and obtain the first prediction category corresponding to each sample cigarette image based on M sample cigarette images, it is specifically used to: perform data labeling and normalization processing on the M sample cigarette images respectively to obtain M processed sample cigarette images; input the M processed sample cigarette images into the pre-built first initial defect detection model to obtain the first prediction category corresponding to each sample cigarette image.

[0106] In some embodiments, the apparatus may further include a construction module. The construction module is configured to construct a second initial defect detection model based on the first defect detection model; the training module is further configured to: transfer model parameters in the first defect detection model to the second initial defect detection model to obtain a second intermediate defect detection model; obtain second actual categories corresponding to M sample cigarette images, respectively, where the second actual categories are obtained by labeling the M sample cigarette images based on a preset labeling strategy; the second actual categories are any of the following: non-defective cigarettes, underfilled cigarettes, flashed cigarettes, surface damaged cigarettes, and wrapperless cigarettes; input the M sample cigarette images into the second intermediate defect detection model to obtain second predicted categories corresponding to the M sample cigarette images; and train the second intermediate defect detection model in a direction that reduces the difference between the second actual category and the second predicted category of each sample cigarette image to obtain a second defect detection model.

[0107] In some embodiments, the device may further include a determination module, which is used to, for each sample cigarette image in the M sample cigarette images, determine that the second actual category corresponding to the sample cigarette image is an unfilled cigarette when the ratio between the hollow area of ​​the target sample cigar corresponding to the sample cigarette image and the total area of ​​the target sample cigar is greater than a preset ratio and the hollow depth is greater than a preset depth threshold; determine that the second actual category corresponding to the sample cigarette image is a fringe cigarette when the wrapper of the target sample cigar corresponding to the sample cigarette image has flaky shedding; determine that the second actual category corresponding to the sample cigarette image is a surface-damaged cigarette when the damaged area of ​​the target sample cigar corresponding to the sample cigarette image is greater than a preset area threshold; and determine that the second actual category corresponding to the sample cigarette image is a cigarette without a wrapper when the target sample cigar corresponding to the sample cigarette image has no wrapper.

[0108] Each module in the aforementioned cigar defect detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the detection device in hardware form, or stored in memory in the vehicle control device in software form, allowing the processor to call and execute the corresponding operations of each module.

[0109] In an exemplary embodiment, a detection device is provided, the internal structure of which can be shown as follows: Figure 7 As shown. The detection device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, while the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the detection device provides computing and control capabilities. The memory of the detection device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the detection device is used to exchange information between the processor and external devices. The communication interface of the detection device is used to communicate with external terminals via wired or wireless means, and the wireless means can be achieved via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a method for detecting cigar defects. The display unit of the detection device is used to produce a visual image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the detection device can be a touch layer covered on the display screen, or a button, trackball or touchpad set in the detection device.

[0110] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the detection equipment to which the scheme of the present application is applied. The specific detection equipment may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0111] In an exemplary embodiment, the present application provides a detection device including a memory and a processor, wherein the memory stores a computer program; when the processor executes the computer program, the steps in the above-mentioned cigar defect detection methods are implemented.

[0112] In an exemplary embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned cigar defect detection methods.

[0113] In an exemplary embodiment, the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned cigar defect detection methods when executed by a processor.

[0114] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0115] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0116] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0117] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting defects in cigars, characterized in that: The method comprises: Obtaining an image of a cigar to be detected; Invoking a pre-trained first defect detection model to detect the cigarette image to obtain a first detection result of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; and the first detection result is a non-defective cigarette or a defective cigarette; When the first detection result is a non-defective cigarette, a pre-trained second defect detection model is called to detect the cigarette image to obtain a second detection result, and the second detection result is used as the target detection result of the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any one of the following: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining N sample cigarette images corresponding to a plurality of sample cigars, and filtering the N sample cigarette images to obtain M sample cigarette images; N and M are both positive integers, M≤N; Obtaining first actual categories corresponding to the M sample cigarette images, respectively, where the first actual categories include defective cigarettes or non-defective cigarettes; Invoking a pre-built first initial defect detection model, based on the M sample cigarette images, to obtain a first predicted category corresponding to each of the sample cigarette images; the first initial defect detection model is built based on partial convolution; The first initial defect detection model is trained in a direction of reducing the difference between the first actual category and the first predicted category of each of the sample cigarette images to obtain the first defect detection model.

3. The method according to claim 2, characterized in that The filtering process on the N sample cigarette images to obtain M sample cigarette images includes: From the N sample cigarette images, a plurality of non-defective sample cigarette images are filtered out to obtain M sample cigarette images.

4. The method according to claim 2, characterized in that The calling of the pre-built first initial defect detection model to obtain a first prediction category corresponding to each of the M sample cigarette images based on the M sample cigarette images includes: performing data labeling and normalization processing on the M sample cigarette images respectively to obtain M processed sample cigarette images; The M processed sample cigarette images are input into a pre-built first initial defect detection model to obtain a first predicted category corresponding to each of the sample cigarette images.

5. The method according to claim 2, characterized in that The method further comprises: Based on the first defect detection model, construct a second initial defect detection model; Migrating model parameters in the first defect detection model to the second initial defect detection model to obtain a second intermediate defect detection model; Obtaining second actual categories corresponding to the M sample cigarette images, respectively, where the second actual categories are obtained by labeling the M sample cigarette images based on a preset labeling strategy; the second actual categories are any one of the following: non-defective cigarette, underfilled cigarette, flash-edged cigarette, surface-damaged cigarette, and wrapper-less cigarette; Inputting the M sample cigarette images into the second intermediate defect detection model to obtain second predicted categories corresponding to the M sample cigarette images respectively; The second intermediate defect detection model is trained in a direction of reducing the difference between the second actual category and the second predicted category of each of the sample cigarette images to obtain the second defect detection model.

6. The method according to claim 5, characterized in that The method further comprises: For each of the M sample cigarette images, if a ratio between a hollow area of ​​the target sample cigar corresponding to the sample cigarette image and a total area of ​​the target sample cigar is greater than a preset ratio, and a hollow depth is greater than a preset depth threshold, determining that the second actual category corresponding to the sample cigarette image is a half-filled cigarette; When the wrapper of the target sample cigar corresponding to the sample cigarette image is flaked, determining that the second actual category corresponding to the sample cigarette image is a flash-edge cigarette; When the damaged area of ​​the target sample cigar corresponding to the sample cigarette image is greater than a preset area threshold, determining that the second actual category corresponding to the sample cigarette image is a surface-damaged cigarette; In a case where the target sample cigar corresponding to the sample cigarette image has no wrapper, it is determined that the second actual category corresponding to the sample cigarette image is a cigarette without a wrapper.

7. A cigar defect detection device, characterized in that: The device comprises: An acquisition module, used to acquire an image of a cigar to be detected; a first detection module configured to invoke a pre-trained first defect detection model to detect the cigarette image and obtain a first detection result of the cigar to be detected; wherein the first defect detection model is a lightweight neural network model; and the first detection result is a non-defective cigarette or a defective cigarette; The second detection module is used to call a pre-trained second defect detection model to detect the cigarette image when the first detection result is a non-defective cigarette, obtain a second detection result, and use the second detection result as the target detection result of the cigar to be detected; the second defect detection model is an end-to-end neural network model trained based on the first defect detection model; the target detection result includes any one of the following: non-defective cigarettes, partly filled cigarettes, cigarettes with fins, cigarettes with damaged surfaces, and cigarettes without wrappers.

8. A detection device, characterized in that: The method comprises a data processing module, wherein the data processing module comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images