Cigarette packet production detection method and device and storage medium

Abstract

The invention discloses a cigarette packet production detection method and device, and a storage medium. The method comprises the following steps: loading a target detection network; receiving image data acquired from cigarette packets on a production line; in the backbone structure, extracting basic image features of multiple scales from the image data under the condition of enhancing defect perception of the cigarette packet; in the neck structure, interacting the basic image features of the plurality of scales to obtain a plurality of aggregated image features; in each head structure, candidate boxes representing defects of the cigarette packets on the lining paper and / or the frame paper are generated on the dimensions of the shape and the texture according to the aggregated image features; and fusing the plurality of candidate frames into a target frame representing defects of the cigarette packet on the lining paper and / or the frame paper. According to the embodiment, the performance of detecting the quality of the lining paper and the frame paper in the cigarette packet is effectively improved, the situations of misjudgment or missing detection and the like are reduced, and therefore the efficiency of detecting the quality of the lining paper and the frame paper in the cigarette packet is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a production detection method, device and storage medium for cigarette packages. Background Art

[0002] The cigarette pack contains materials such as lining paper and frame paper. The lining paper wraps the cigarettes, and the frame paper is wrapped over the outside of the lining paper of the cigarettes to fix the cigarettes.

[0003] During the production process of cigarette packs, the lining paper and frame paper may have quality problems such as damage and offset. At present, static detection methods are mostly used to detect the quality of the lining paper and frame paper. That is, when the cigarette pack is stationary, one or more sensors (such as photoelectric sensors or mechanical contacts) are used to judge defects in specific positions.

[0004] For the lining paper, an inductive sensor is set at a fixed station to detect whether there is a metal layer (aluminum foil lining paper) on the lining paper. For the frame paper, the color difference of the frame paper is detected by a fiber optic sensor at a fixed position.

[0005] However, the sensor is susceptible to interference such as position deviation of cigarette packs, which can easily lead to misjudgment or missed detection, resulting in low efficiency in detecting the quality of the liner paper and frame paper in cigarette packs. Summary of the Invention

[0006] In view of this, the present invention provides a production inspection method, equipment and storage medium for cigarette packs, so as to improve the efficiency of inspecting the quality of liner paper and frame paper in cigarette packs.

[0007] A first aspect of the present invention provides a production inspection method for cigarette packages, comprising:

[0008] Loading a target detection network; the target detection network includes a backbone structure, a neck structure, and multiple head structures;

[0009] Receiving image data collected from cigarette packs on a production line;

[0010] In the backbone structure, basic image features at multiple scales are extracted from the image data under the condition of enhancing defect perception of the cigarette pack;

[0011] In the neck structure, interacting the basic image features at multiple scales to obtain multiple aggregated image features;

[0012] In each of the head structures, generating a candidate frame representing defects on the liner paper and / or frame paper of the cigarette pack based on the aggregated image features in the dimensions of shape and texture;

[0013] The plurality of candidate frames are merged into a target frame representing defects on the liner paper and / or frame paper of the cigarette pack.

[0014] A second aspect of the present invention provides a cigarette pack production detection device, comprising:

[0015] A target detection network loading module is used to load a target detection network; the target detection network includes a backbone structure, a neck structure and multiple head structures;

[0016] An image data receiving module, configured to receive image data collected from cigarette packs on the production line;

[0017] a basic image feature extraction module, configured to extract basic image features of multiple scales from the image data in the backbone structure while enhancing defect perception of the cigarette pack;

[0018] An aggregated image feature interaction module, configured to interact the basic image features of multiple scales in the neck structure to obtain multiple aggregated image features;

[0019] a candidate frame generation module, configured to generate, in each of the head structures, a candidate frame representing defects on the liner paper and / or frame paper of the cigarette pack based on the aggregated image features in terms of shape and texture;

[0020] The target frame fusion module is used to fuse the multiple candidate frames into a target frame representing defects on the liner paper and / or frame paper of the cigarette package.

[0021] A third aspect of the present invention provides an electronic device, comprising:

[0022] at least one processor; and

[0023] a memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the production detection method for cigarette packages as described in the first aspect above.

[0025] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the cigarette pack production inspection method as described in the first aspect above.

[0026] A fifth aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the production detection method of cigarette packets as described in the first aspect above.

[0027] In this embodiment, a target detection network is loaded; the target detection network includes a backbone structure, a neck structure, and multiple head structures; image data collected from cigarette packs on a production line is received; in the backbone structure, basic image features at multiple scales are extracted from the image data while enhancing defect perception of the cigarette packs; in the neck structure, the basic image features at multiple scales are interacted to obtain multiple aggregated image features; in each head structure, candidate boxes representing defects on the liner paper and / or frame paper of the cigarette pack are generated based on the aggregated image features in the dimensions of shape and texture; and the multiple candidate boxes are fused into target boxes representing defects on the liner paper and / or frame paper of the cigarette pack. In this embodiment, the target detection network is adaptively improved based on the characteristics of the liner paper and frame paper in the cigarette pack. The target detection network is highly resistant to interference such as positional deviation of the cigarette pack, effectively improving the performance of detecting the quality of the liner paper and frame paper in the cigarette pack, reducing misjudgments or missed detections, and thus effectively improving the efficiency of detecting the quality of the liner paper and frame paper in the cigarette pack.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 This is a flow chart of a cigarette pack production inspection method provided in Example 1 of the present invention.

[0031] Figure 2 This is a comparison diagram of a standard cigarette pack and a defective cigarette pack provided in Example 1 of the present invention.

[0032] Figure 3 This is a schematic diagram of a backbone structure provided in Example 1 of the present invention.

[0033] Figure 4 This is a schematic diagram of a material defect enhancement module provided in Example 1 of the present invention.

[0034] Figure 5 This is a schematic diagram of a head structure provided in Example 1 of the present invention.

[0035] Figure 6It is a structural diagram of a cigarette pack production detection device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Example 1

[0039] See also Figure 1 , shows a flow chart of a cigarette pack production detection method provided by the first embodiment of the present invention. The method can be executed by a cigarette pack production detection device. The cigarette pack production detection device can be implemented in the form of hardware and / or software. The cigarette pack production detection device can be configured in an electronic device. The electronic device includes an edge computing unit, which can deploy a target detection network. Figure 1 As shown, the method includes:

[0040] Step 101: Load the target detection network.

[0041] In this embodiment, a target detection network can be constructed and trained for the inner liner paper and frame paper in the cigarette pack, and the target detection network can be used to detect the quality of the inner liner paper and frame paper in the cigarette pack through computer vision.

[0042] In general, if Figure 2 As shown, there is little difference between the inner lining paper and frame paper in a normal cigarette pack (i.e., standard sample) and the frame paper with defects (displacement) and the inner lining paper with defects (wrinkles (scratches)).

[0043] The native object detection network has limited generalization capabilities. During its construction and pre-training process, it does not care about defects in the lining paper and frame paper in cigarette packs, especially small defects. If sample image data of the lining paper and frame paper in cigarette packs is collected and the native object detection network is fine-tuned using the sample image data, and the fine-tuned native object detection network is used to detect the quality of the lining paper and frame paper in cigarette packs, there will be problems such as insufficient defect detection capabilities.

[0044] Using YOLOv11 as an example of an object detection network, we fine-tuned YOLOv11 using sample image data. When using YOLOv11 to inspect the quality of the liner and frame paper in cigarette packs, we found that its defect detection capabilities were insufficient in the following ways:

[0045] 1. For subtle (≤1mm) defects (such as scratches and pinholes) in the lining paper and frame paper of cigarette packs, the mAP (mean average precision) is less than 70%.

[0046] 2. The false detection rate of texture defects (color difference, abnormal reflection) of the lining paper and frame paper of cigarette packs is greater than 25%.

[0047] 3. The recall rate difference of multi-scale defect detection is greater than 15% (the recall rate of large defects is 98%, and the recall rate of small defects is 83%).

[0048] In this embodiment, the native target detection network is adaptively modified according to the characteristics of the inner liner paper and frame paper in the cigarette pack, thereby improving the defect detection capability of the target detection network when detecting the quality of the inner liner paper and frame paper in the cigarette pack.

[0049] Among them, the target detection network includes a backbone structure, a neck structure, and multiple head structures.

[0050] Among them, the backbone structure Backbone is the main part of the target detection network. Through operations such as convolutional layers and pooling layers, it extracts feature maps with high-level semantic information from the input data.

[0051] The neck structure is located between the backbone structure and the head structure. It fuses feature maps from different levels and combines low-level detail information with high-level semantic information.

[0052] The head structure is responsible for the task of target detection. It detects the type and location of the target by inputting the feature map processed by the neck structure.

[0053] Step 102: Receive image data collected from cigarette packs on the production line.

[0054] In this embodiment, industrial cameras (such as high-speed linear array camera arrays) and light sources (such as LED (light-emitting diode) lights) can be deployed on the cigarette production line. The light source provides lighting conditions of appropriate brightness for the cigarette packs on the production line, and the industrial camera collects image data of the cigarette packs on the production line under the lighting conditions.

[0055] Furthermore, multiple pre-processing steps may be performed on the image data to improve the quality of the image data, such as noise suppression, illumination compensation, etc.

[0056] Step 103: In the backbone structure, basic image features of multiple scales are extracted from the image data under the condition of enhancing defect perception of the cigarette package.

[0057] In this embodiment, the backbone structure Backbone is modified to enhance its ability to detect material surface defects, thereby extracting basic image features at multiple scales from image data while enhancing the defect perception of cigarette packages.

[0058] In one embodiment of the present invention, a middle layer in the backbone structure is a material defect enhancement module MDEM, which is responsible for improving the detection capability of material surface defects.

[0059] The input of the material defect enhancement module MDEM is the intermediate image feature F∈R H×W×C (R is a set of real numbers, H is the height, W is the width, and C is the number of channels, such as F∈R 640×640×3 ), the output is defect enhanced image features.

[0060] The material defect enhancement module MDEM includes a first branch structure, a second branch structure, a third branch structure, a fourth branch structure and a branch fusion structure.

[0061] Taking YOLOv11 as an example of the target detection network, its native backbone structure is CSPDarkNet, such as Figure 3 As shown in the figure, the modified backbone structure Backbone includes convolutional layer Conv, convolutional layer Conv, C3K2 module, convolutional layer Conv, C3K2 module, convolutional layer Conv, C3K2 module, convolutional layer Conv, C3K2 module, material defect enhancement module MDEM, spatial pyramid pooling fast (Spatial Pyramid Pooling Fast, SPPF) and C2PSA module in sequence.

[0062] Among them, the C3K2 module provides feature extraction capabilities by combining variable convolution kernels (such as 3×3, 5×5, etc.) and channel separation strategies.

[0063] The C2PSA module combines the CSP (Cross Stage Partial) structure and the PSA (Pyramid SqueezeAttention) attention mechanism to improve the multi-scale feature extraction capability.

[0064] The feature map output by the fourth C3K2 module (i.e., the intermediate image feature) is input into the material defect enhancement module MDEM, and the feature map output by the material defect enhancement module MDEM (i.e., the defect enhanced image feature) is input into the spatial pyramid pooling module SPPF.

[0065] Among them, the second C3K2 module outputs the basic image features of the first scale, wherein the size of the basic image features of the first scale is 80×80×256.

[0066] The third C3K2 module outputs the basic image features of the second scale, where the size of the basic image features of the second scale is 40×40×512.

[0067] The C2PSA module outputs the basic image features of the third scale, where the size of the basic image features of the third scale is 20×20×1024.

[0068] Of course, the positions of the backbone structure Backbone and its material defect enhancement module MDEM are only used as examples. When implementing this embodiment, the positions of other backbone structures Backbone and its material defect enhancement module MDEM can be set according to actual conditions. For example, in the backbone structure Backbone of target detection networks such as YOLOv5 and YOLOv8, the material defect enhancement module MDEM is set before the spatial pyramid pooling module SPPF, etc. This embodiment does not limit this. In addition, in addition to the positions of the backbone structure Backbone and its material defect enhancement module MDEM described above, those skilled in the art can also adopt other backbone structures Backbone and its material defect enhancement module MDEM positions according to actual needs, and this embodiment does not limit this.

[0069] Then, in this embodiment, step 103 may include the following steps:

[0070] Step 1031: In the first branch structure, retain the intermediate image features.

[0071] like Figure 4 As shown, in the first branch structure, the intermediate image feature F is not processed, so the intermediate image feature F is retained.

[0072] Step 1032: In the second branch structure, the deformation characteristics of the cigarette pack are captured by the intermediate image features to obtain deformation image features.

[0073] In the second branch structure, the deformation characteristics of the cigarette pack can be captured by the intermediate image feature F and recorded as the deformation image feature.

[0074] For example, Figure 4 As shown, the second branch structure includes a deformable convolution module DCN. Then, in the second branch structure, the intermediate image feature F can be input into the deformable convolution module DCN, and a variable row convolution operation is performed on the intermediate image feature F in the deformable convolution module DCN to capture the features of the cigarette pack in deformation and obtain the deformed image feature DCN(F).

[0075] Step 1033: In the third branch structure, the spatial features of the cigarette packet defects are enhanced based on the intermediate image features to obtain spatial image features.

[0076] In the third branch structure, the spatial characteristics of the cigarette packet defect can be enhanced for the intermediate image feature F, which is recorded as the spatial image feature.

[0077] For example, Figure 4 As shown, the third branch structure includes a spatial attention module SA. Then, in the third branch structure, the intermediate image feature F can be input into the spatial attention module SA. In the spatial attention module SA, the intermediate image feature F is converted into the attention of the defect of the cigarette package at each spatial position in space (i.e., the spatial attention map) to obtain the spatial image feature SA (F).

[0078] Among them, SA(F)=σ(Conv([MaxPool(F)];[AvgPool(F)])), where σ represents the sigmoid activation function, Conv represents the convolutional layer (size is 7×7), MaxPool represents the maximum pooling operation in the channel dimension, and AvgPool represents the average pooling operation in the channel dimension.

[0079] Step 1034: In the fourth branch structure, the features of the channel that is sensitive to cigarette pack defects are highlighted from the intermediate image features to obtain channel image features.

[0080] In the fourth branch structure, the intermediate image feature F can highlight the characteristics of the channel that is sensitive to the defects of the cigarette package and record it as the channel image feature.

[0081] For example, Figure 4 As shown, the fourth branch structure includes a channel attention module CA. Then, in the fourth branch structure, the intermediate image feature F can be input into the channel attention module CA. In the channel attention module CA, the intermediate image feature F is converted into the attention of the channel that is sensitive to the defects of the cigarette package to obtain the channel image feature CA(F).

[0082] in, Among them, F represents the intermediate image features, σ represents the sigmoid activation function, MLP represents two layers of fully connected layers, which first compresses the number of channels to C / r (C is the number of channels, r is the weight, r is usually 16), and then restores it to C, MaxPool represents the maximum pooling operation in the channel dimension, and AvgPool represents the average pooling operation in the channel dimension. Indicates multiplication.

[0083] Step 1035: In the branch fusion structure, the intermediate image feature F, the deformation image feature, the spatial image feature and the channel image feature are fused into a defect enhancement image feature.

[0084] In the branch fusion structure, the intermediate image feature F, the deformable image feature DCN(F), the spatial image feature SA(F) and the channel image feature CA(F) are interacted and fused into the defect enhancement image feature MDEM(F).

[0085] For example, Figure 4 As shown, the branch fusion structure includes a convolutional layer Conv (size is 1×1), which multiplies the deformable image feature DCN(F) with the spatial image feature SA(F) (element-level multiplication, which belongs to the broadcast mechanism) to obtain the first fused image feature. Then, the first fused image feature represents the spatial attention weighted deformable image feature DCN(F).

[0086] Concat the intermediate image features, the first fused image features and the channel image features into the second image features.

[0087] In the convolution layer Conv, a convolution operation is performed on the second image feature to obtain the defect enhanced image feature MDEM(F), and the number of channels of the defect enhanced image feature MDEM(F) is compressed back to the number of channels of the intermediate image feature F.

[0088] So, in, Indicates multiplication.

[0089] The Material Defect Enhancement Module (MDEM) enhances the characterization capability of material defects in feature maps from three perspectives: original features, deformation-sensitive features, and attention-enhanced features. It provides richer feature representations for different types of defect detection. These features complement each other and can effectively enhance the perception of tiny defects, allowing for greater attention to subtle shape and texture changes during detection.

[0090] Step 104: In the neck structure, basic image features at multiple scales are interacted to obtain multiple aggregated image features.

[0091] In this embodiment, the native neck structure Neck (such as PANet in YOLOv11) can be used to interact basic image features of multiple scales to obtain aggregated image features of multiple scales (such as 80×80×256, 40×40×512, and 20×20×1024).

[0092] Step 105: In each header structure, a candidate frame representing defects on the liner paper and / or frame paper of the cigarette pack is generated based on the aggregated image features in the dimensions of shape and texture.

[0093] In this embodiment, the head structure of the native target detection network can be reconstructed according to the characteristics of the liner paper and frame paper in the cigarette package, so that it can focus on detecting geometric defects and surface texture defects.

[0094] There is a one-to-one correspondence between the aggregated image features output by the neck structure Neck and the head structure Head. The aggregated image features output by the neck structure Neck are input into the corresponding head structure Head. In the head structure Head, a candidate box representing the defects of the cigarette package on the lining paper and / or frame paper is generated based on the aggregated image features in the dimensions of shape and texture.

[0095] In one embodiment of the present invention, each head structure includes a shape path structure ShapePath, a texture path structure TexturePath and a path fusion structure, wherein the shape path structure ShapePath focuses on detecting defects in geometric shapes, and the texture path structure TexturePath focuses on detecting defects in surface textures.

[0096] Then, in this embodiment, step 105 may include the following steps:

[0097] Step 1051: In the shape path structure, shape image features representing geometric defects in the liner paper and / or frame paper of the cigarette pack are detected based on the aggregated image features.

[0098] The aggregated image features are processed in the shape path structure ShapePath to generate shape image features representing geometric defects in the liner paper and / or frame paper of the cigarette pack.

[0099] Among them, the shape image features have the coordinates of the bounding box and the probability of the shape category.

[0100] In the specific implementation, Figure 5 As shown in the figure, from the direction of forward propagation, the shape path structure ShapePath includes a deformable convolution module DCN, a convolution layer Conv (size is 3×3) and a convolution layer Conv (size is 3×3) in sequence.

[0101] In the deformable convolution module DCN, a deformable convolution operation is performed on the aggregated image features P to obtain shape-deformed image features.

[0102] The aggregated image feature P is connected to the residual of the shape deformation image feature, and the aggregated image feature P is added to the shape deformation image feature to obtain the weighted deformation image feature.

[0103] The convolutional layer Conv and the convolutional layer Conv are called in sequence to perform convolution operations on the weighted deformable image features. The first convolutional layer Conv extracts primary shape features, and the second convolutional layer Conv further extracts advanced shape features to obtain the shape image feature ShapePath(P) representing the geometric defects of the liner paper and / or frame paper of the cigarette package.

[0104] Then, ShapePath(P)=Conv(Conv(P+DCN(P))).

[0105] The ShapePath architecture uses deformable convolution to capture the geometric features of irregular shapes, particularly edges, corners, and deformed regions, which are key characteristics of shape defects such as damage and wrinkles. Two stacked convolutional layers expand the receptive field and extract more abstract shape features.

[0106] Step 1052: In the texture path structure, texture image features representing surface defects of the liner paper and / or frame paper of the cigarette pack are detected based on the aggregated image features.

[0107] The aggregated image features are processed in the texture path structure TexturePath to generate texture image features representing defects on the surface of the liner paper and / or frame paper of the cigarette pack.

[0108] Among them, the texture image feature has confidence and probability of texture category.

[0109] In the specific implementation, Figure 5 As shown in the figure, from the direction of forward propagation, the texture path structure TexturePath includes the convolution block attention module CBAM, the convolution layer Conv (size is 1×1) and the convolution layer Conv (size is 3×3) in sequence.

[0110] In the convolutional block attention module CBAM, an attention enhancement operation is performed on the aggregated image features P to obtain texture attention image features.

[0111] The aggregated image feature P is residually connected to the texture attention image feature, and the aggregated image feature P is added to the texture attention image feature to obtain the weighted texture image feature.

[0112] The convolutional layer Conv and the convolutional layer Conv are called in sequence to perform convolution operations on the weighted texture image features. The first convolutional layer Conv extracts primary shape features, and the second convolutional layer Conv further extracts advanced shape features to obtain the texture image features TexturePath(P) representing the defects on the surface of the liner paper and / or frame paper of the cigarette package.

[0113] Then, TexturePath(P)=Conv(Conv(P+CBAM(P))).

[0114] The TexturePath architecture uses the attention of convolutional blocks to capture irregular texture features, particularly wavy patterns, speckling / graininess, stripes / streaks, and show-through / show-through—the primary characteristics of texture defects. Two stacked convolutional layers expand the receptive field and extract more abstract shape features.

[0115] Step 1053: In the path fusion structure, shape image features and texture image features are combined to generate candidate frames representing defects on the liner paper and / or frame paper of the cigarette pack.

[0116] In the path fusion structure, shape image features and texture image features are interacted and fused to generate candidate frames representing defects on the liner paper and / or frame paper of the cigarette pack, wherein the candidate frames have information such as position, category and confidence.

[0117] In the specific implementation, Figure 5 As shown, the path fusion structure includes a convolutional layer Conv (size is 1×1).

[0118] Then, the shape image features and texture image features are concat- ed into shape and texture image features.

[0119] In the convolution layer Conv, a convolution operation is performed on the shape and pattern image features to obtain a candidate box representing the defects of the cigarette package on the liner paper and / or frame paper.

[0120] Step 106: Merge multiple candidate frames into a target frame representing defects on the liner paper and / or frame paper of the cigarette package.

[0121] In this embodiment, the candidate frames output by multiple head structures are compared and fused into target frames representing defects on the lining paper and / or frame paper of the cigarette package using methods such as IOU (Intersection Over Union) and NMS (Non-Maximum Suppression).

[0122] During the testing phase, image data of frame paper defects (i.e., frame paper defect map) and image data of lining paper defects (i.e., lining paper defect map) were produced respectively. The frame paper defect map and lining paper defect map were used to test the target detection network, and various indicators of the target detection network were calculated based on the recognition results.

[0123] Among them, the detection accuracy of wrinkled frame paper / missing lining paper is 99.9%, the detection accuracy of position offset / scratches is 96%, the comprehensive detection accuracy is 97.8%, and the false detection rate is 1.2%, which effectively reduces the difference in recall rate of multi-scale defect detection.

[0124] If it is confirmed that there are defects on the lining paper and / or frame paper of the cigarette pack, the PLC (programmable logic controller) controller on the production line can be notified. The encoder in the PLC controller feeds back information such as the speed and phase of the packaging machine. Based on this information, the cigarette pack with defects on the lining paper and / or frame paper can be located on the production line, the actuator can be controlled to remove the defective cigarette pack, and the alarm system can be controlled to change the status indication to prompt that the cigarette pack has a defect.

[0125] In this embodiment, a target detection network is loaded; the target detection network includes a backbone structure, a neck structure, and multiple head structures; image data collected from cigarette packs on a production line is received; in the backbone structure, basic image features at multiple scales are extracted from the image data while enhancing defect perception of the cigarette packs; in the neck structure, the basic image features at multiple scales are interacted to obtain multiple aggregated image features; in each head structure, candidate boxes representing defects on the liner paper and / or frame paper of the cigarette pack are generated based on the aggregated image features in the dimensions of shape and texture; and the multiple candidate boxes are fused into target boxes representing defects on the liner paper and / or frame paper of the cigarette pack. In this embodiment, the target detection network is adaptively improved based on the characteristics of the liner paper and frame paper in the cigarette pack. The target detection network is highly resistant to interference such as positional deviation of the cigarette pack, effectively improving the performance of detecting the quality of the liner paper and frame paper in the cigarette pack, reducing misjudgments or missed detections, and thus effectively improving the efficiency of detecting the quality of the liner paper and frame paper in the cigarette pack.

[0126] Example 2

[0127] See also Figure 6 , shows a schematic structural diagram of a cigarette pack production detection device provided by the second embodiment of the present invention. Figure 6 As shown, the device includes:

[0128] The target detection network loading module 601 is used to load the target detection network; the target detection network includes a backbone structure, a neck structure and multiple head structures;

[0129] An image data receiving module 602 is used to receive image data collected from cigarette packs on the production line;

[0130] A basic image feature extraction module 603 is configured to extract basic image features of multiple scales from the image data in the backbone structure while enhancing defect perception of the cigarette pack;

[0131] An aggregated image feature interaction module 604 is configured to interact the basic image features at multiple scales in the neck structure to obtain multiple aggregated image features;

[0132] A candidate frame generation module 605 is configured to generate, in each of the head structures, a candidate frame representing defects on the liner paper and / or frame paper of the cigarette pack based on the aggregated image features in terms of shape and texture;

[0133] The target frame fusion module 606 is used to fuse the multiple candidate frames into a target frame representing the defects of the cigarette package on the liner paper and / or frame paper.

[0134] In one embodiment of the present invention, a middle layer in the backbone structure is a material defect enhancement module, the input of the material defect enhancement module is the intermediate image feature, and the output is the defect enhanced image feature. The material defect enhancement module includes a first branch structure, a second branch structure, a third branch structure, a fourth branch structure and a branch fusion structure;

[0135] The basic image feature extraction module 603 includes:

[0136] A first branch processing module, configured to retain the intermediate image features in the first branch structure;

[0137] a second branch processing module configured to capture deformation features of the cigarette packet from the intermediate image features in the second branch structure to obtain deformation image features;

[0138] a third branch processing module configured to enhance, in the third branch structure, the spatial features of the defects of the cigarette pack based on the intermediate image features to obtain spatial image features;

[0139] a fourth branch processing module, configured to, in the fourth branch structure, highlight features of a channel that is sensitive to defects of the cigarette packet from the intermediate image features to obtain channel image features;

[0140] A branch fusion module is used to fuse the intermediate image features, the deformation image features, the spatial image features and the channel image features into defect enhancement image features in the branch fusion structure.

[0141] In one embodiment of the present invention, the second branch structure includes a deformable convolution module, the third branch structure includes a spatial attention module, the fourth branch structure includes a channel attention module, and the branch fusion structure includes a convolution layer;

[0142] The second branch processing module is further configured to:

[0143] performing a variable row convolution operation on the intermediate image features in the deformable convolution module to capture the features of the cigarette pack in deformation and obtain deformed image features;

[0144] The third branch processing module is further configured to:

[0145] In the spatial attention module, the intermediate image features are converted into spatial attention of the defects of the cigarette pack to obtain spatial image features;

[0146] The fourth branch processing module is further configured to:

[0147] In the channel attention module, the intermediate image features are converted into attention of a channel that is sensitive to defects of the cigarette pack to obtain a channel image feature;

[0148] The branch fusion module is further configured to:

[0149] multiplying the deformation image feature and the spatial image feature to obtain a first fused image feature;

[0150] splicing the intermediate image feature, the first fused image feature and the channel image feature into a second image feature;

[0151] In the convolution layer, a convolution operation is performed on the second image feature to obtain a defect enhanced image feature.

[0152] In one embodiment of the present invention, the backbone structure includes, in sequence, a convolutional layer, a C3K2 module, a convolutional layer, a C3K2 module, a convolutional layer, a C3K2 module, a convolutional layer, a C3K2 module, a convolutional layer, a C3K2 module, a material defect enhancement module, a spatial pyramid pooling module, and a C2PSA module;

[0153] The second C3K2 module outputs basic image features of the first scale;

[0154] The third C3K2 module outputs the basic image features of the second scale;

[0155] The C2PSA module outputs basic image features at the third scale.

[0156] In one embodiment of the present invention, each of the head structures includes a shape path structure, a texture path structure and a path fusion structure;

[0157] The candidate frame generation module 605 includes:

[0158] a shape path processing module configured to detect, in the shape path structure, shape image features representing geometric defects in the liner paper and / or frame paper of the cigarette pack based on the aggregated image features; wherein the shape image features include coordinates of a bounding box and a probability of a shape category;

[0159] a texture path processing module, configured to detect, in the texture path structure, texture image features representing surface defects of the liner paper and / or frame paper of the cigarette pack based on the aggregated image features; wherein the texture image features have a confidence level and a probability of a texture category;

[0160] A path fusion module is used to generate a candidate frame representing defects of the cigarette package on the liner paper and / or frame paper by combining the shape image features and the texture image features in the path fusion structure.

[0161] In one embodiment of the present invention, the shape path structure includes a deformable convolution module, a convolution layer, and a convolution layer;

[0162] The shape path processing module is further configured to:

[0163] performing a deformable convolution operation on the aggregated image features in the deformable convolution module to obtain shape-deformed image features;

[0164] Adding the aggregated image feature to the shape deformation image feature to obtain a weighted deformation image feature;

[0165] The convolution layer and the convolution layer are called in sequence to perform convolution operations on the weighted deformation image features to obtain shape image features representing defects in the geometric shape of the liner paper and / or frame paper of the cigarette package.

[0166] In one embodiment of the present invention, the texture path structure includes a convolutional block attention module, a convolutional layer and a convolutional layer;

[0167] The texture path processing module is further configured to:

[0168] performing an attention enhancement operation on the aggregated image features in the convolutional block attention module to obtain texture attention image features;

[0169] Adding the aggregated image feature to the texture attention image feature to obtain a weighted texture image feature;

[0170] The convolution layer and the convolution layer are called in sequence to perform convolution operations on the weighted texture image features to obtain texture image features representing defects on the surface of the liner paper and / or frame paper of the cigarette pack.

[0171] In one embodiment of the present invention, the path fusion structure includes a convolutional layer;

[0172] The path fusion module is further used for:

[0173] splicing the shape image feature and the texture image feature into a shape-texture image feature;

[0174] In the convolution layer, a convolution operation is performed on the shape image features to obtain candidate frames representing defects on the liner paper and / or frame paper of the cigarette package.

[0175] The cigarette pack production inspection device provided in the embodiment of the present invention can execute the cigarette pack production inspection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the cigarette pack production inspection method.

[0176] Example 3

[0177] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit implementations of the inventions described and / or claimed herein.

[0178] The electronic device includes at least one processor and a memory connected to the at least one processor, such as a read-only memory (ROM), a random access memory (RAM), etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) or the computer program loaded from the storage unit into the random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device can also be stored. The processor, ROM and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0179] Many components in an electronic device are connected to the I / O interface, including: input units, such as a keyboard and mouse; output units, such as various types of displays and speakers; storage units, such as magnetic disks and optical disks; and communication units, such as network cards, modems, and wireless communication transceivers. The communication unit allows the electronic device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0180] The processor can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The processor executes the various methods and processes described above, such as the cigarette pack production detection method.

[0181] In some embodiments, the production detection method of cigarette packs can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via a ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the production detection method of cigarette packs described above can be performed. Alternatively, in other embodiments, the processor can be configured to execute the production detection method of cigarette packs in any other appropriate manner (for example, by means of firmware).

[0182] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] Example 4

[0186] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the production detection method of cigarette packages provided in any embodiment of the present invention.

[0187] The computer program product may be implemented by writing computer program code for performing the operations of the present invention in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0189] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A production inspection method for cigarette packages, characterized in that: include: Load the target detection network; The target detection network includes a backbone structure, a neck structure and multiple head structures; Receiving image data collected from cigarette packs on a production line; In the backbone structure, basic image features at multiple scales are extracted from the image data under the condition of enhancing defect perception of the cigarette pack; In the neck structure, interacting the basic image features at multiple scales to obtain multiple aggregated image features; In each of the head structures, generating a candidate frame representing defects on the liner paper and / or frame paper of the cigarette pack based on the aggregated image features in the dimensions of shape and texture; The plurality of candidate frames are merged into a target frame representing defects on the liner paper and / or frame paper of the cigarette pack.

2. The method according to claim 1, characterized in that An intermediate layer in the backbone structure is a material defect enhancement module, the input of the material defect enhancement module is the intermediate image feature, and the output is the defect enhanced image feature. The material defect enhancement module includes a first branch structure, a second branch structure, a third branch structure, a fourth branch structure and a branch fusion structure; In the backbone structure, basic image features at multiple scales are extracted from the image data under the condition of enhancing defect perception of the cigarette pack, including: In the first branch structure, retaining the intermediate image features; In the second branch structure, the deformation characteristics of the cigarette pack are captured by the intermediate image features to obtain deformation image features; In the third branch structure, the spatial features of the defects of the cigarette pack are enhanced with respect to the intermediate image features to obtain spatial image features; In the fourth branch structure, the features of the channel sensitive to the defects of the cigarette pack are highlighted from the intermediate image features to obtain channel image features; In the branch fusion structure, the intermediate image features, the deformation image features, the spatial image features and the channel image features are fused into defect enhancement image features.

3. The method according to claim 2, characterized in that The second branch structure includes a deformable convolution module, the third branch structure includes a spatial attention module, the fourth branch structure includes a channel attention module, and the branch fusion structure includes a convolution layer; In the second branch structure, capturing the deformation features of the cigarette pack from the intermediate image features to obtain deformation image features includes: performing a variable row convolution operation on the intermediate image features in the deformable convolution module to capture the features of the cigarette pack in deformation and obtain deformed image features; In the third branch structure, enhancing the spatial features of the cigarette packet defect with respect to the intermediate image features to obtain spatial image features includes: In the spatial attention module, the intermediate image features are converted into spatial attention of the defects of the cigarette pack to obtain spatial image features; In the fourth branch structure, the intermediate image features are used to highlight the features of the channel that is sensitive to the defects of the cigarette pack to obtain the channel image features, including: In the channel attention module, the intermediate image features are converted into attention of a channel that is sensitive to defects of the cigarette pack to obtain a channel image feature; In the branch fusion structure, fusing the intermediate image feature, the deformation image feature, the spatial image feature, and the channel image feature into a defect enhancement image feature includes: multiplying the deformation image feature and the spatial image feature to obtain a first fused image feature; splicing the intermediate image feature, the first fused image feature and the channel image feature into a second image feature; In the convolution layer, a convolution operation is performed on the second image feature to obtain a defect enhanced image feature.

4. The method according to claim 2, characterized in that The backbone structure includes convolutional layer, convolutional layer, C3K2 module, convolutional layer, C3K2 module, convolutional layer, C3K2 module, convolutional layer, C3K2 module, material defect enhancement module, spatial pyramid pooling module and C2PSA module in sequence; The second C3K2 module outputs basic image features of the first scale; The third C3K2 module outputs the basic image features of the second scale; The C2PSA module outputs basic image features at the third scale.

5. The method according to any one of claims 1 to 4, characterized in that Each of the head structures includes a shape path structure, a texture path structure and a path fusion structure; Generating, in each of the head structures, candidate frames representing defects on the liner paper and / or frame paper of the cigarette pack based on the aggregated image features in the dimensions of shape and texture, includes: In the shape path structure, shape image features representing geometric defects in the liner paper and / or frame paper of the cigarette pack are detected based on the aggregated image features; wherein the shape image features include coordinates of a bounding box and a probability of a shape category; In the texture path structure, texture image features representing surface defects of the liner paper and / or frame paper of the cigarette pack are detected based on the aggregated image features; wherein the texture image features have a confidence level and a probability of a texture category; In the path fusion structure, the shape image features and the texture image features are combined to generate candidate frames representing defects on the liner paper and / or frame paper of the cigarette pack.

6. The method according to claim 5, characterized in that The shape path structure includes a deformable convolution module, a convolution layer and a convolution layer; In the shape path structure, detecting shape image features representing geometric defects of the liner paper and / or frame paper of the cigarette pack based on the aggregated image features includes: performing a deformable convolution operation on the aggregated image features in the deformable convolution module to obtain shape-deformed image features; Adding the aggregated image feature to the shape deformation image feature to obtain a weighted deformation image feature; The convolution layer and the convolution layer are called in sequence to perform convolution operations on the weighted deformation image features to obtain shape image features representing defects in the geometric shape of the liner paper and / or frame paper of the cigarette package.

7. The method according to claim 5, characterized in that The texture path structure includes a convolutional block attention module, a convolutional layer and a convolutional layer; In the texture path structure, detecting texture image features representing surface defects of the liner paper and / or frame paper of the cigarette pack based on the aggregated image features includes: performing an attention enhancement operation on the aggregated image features in the convolutional block attention module to obtain texture attention image features; Adding the aggregated image feature to the texture attention image feature to obtain a weighted texture image feature; The convolution layer and the convolution layer are called in sequence to perform convolution operations on the weighted texture image features to obtain texture image features representing defects on the surface of the liner paper and / or frame paper of the cigarette pack.

8. The method according to claim 5, characterized in that The path fusion structure includes a convolutional layer; In the path fusion structure, generating a candidate frame representing a defect on the liner paper and / or frame paper of the cigarette pack by combining the shape image feature and the texture image feature includes: splicing the shape image feature and the texture image feature into a shape-texture image feature; In the convolution layer, a convolution operation is performed on the shape image features to obtain candidate frames representing defects on the liner paper and / or frame paper of the cigarette package.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the production detection method for cigarette packages according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the production detection method for cigarette packages according to any one of claims 1 to 8 is implemented.

Images