Cigarette case boxing defect detection method, device, equipment, medium and program product

Abstract

The application relates to a cigarette case boxing defect detection method, device, equipment, medium and program product. The method comprises the following steps: acquiring cigarette case boxing images of a plurality of cigarette cases after boxing; calling a local detection model to perform local defect detection on the cigarette case boxing images to obtain a local defect detection result; the local defect detection result comprises a local defect type and a confidence of a local defect; calling a global detection model to perform global defect perception on the cigarette case boxing images to obtain a global defect perception result; the global defect perception result comprises a probability of a target defect type corresponding to a defect existing in the cigarette case boxing images; the target defect type comprises a global defect type and a local defect type; and the local defect detection result is corrected based on the global defect perception result to obtain a cigarette case boxing defect detection result. The method can improve the recognition accuracy of cigarette case boxing defects.

Description

Technical Field

[0001] This application relates to the field of testing technology, and in particular to a method, apparatus, equipment, medium, and procedure for detecting defects in cigarette box packaging. Background Technology

[0002] In the tobacco manufacturing industry, the automated packing process of cigarette boxes is subject to the combined effects of mechanical factors such as equipment impact, uneven pushing pressure, and conveying positioning deviation, as well as human factors such as improper manual straightening and replenishment operations. This can easily lead to various packing defects such as crushing, missing strips, skewing, misalignment, and box deformation, which directly affect the quality of finished cigarette packaging and cause product loss and quality risks.

[0003] In related technologies, the identification of packing defects mainly relies on manual sampling or traditional target detection models. Among them, manual sampling is highly dependent on human experience and visual perception, resulting in low detection efficiency, strong subjectivity, and high rates of missed and false detections. Traditional target detection models only identify isolated defects and are insufficient in the face of complex defect scenarios, leading to low accuracy in identifying packing defects. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, equipment, medium, and program product for detecting cigarette box packaging defects that can improve the accuracy of identifying defects in cigarette box packaging, in order to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for detecting defects in cigarette box packaging, including:

[0006] Acquire multiple images of cigarette boxes after they have been packed into a box;

[0007] The local detection model is invoked to perform local defect detection on the cigarette box packaging image, and the local defect detection results are obtained; the local defect detection results include the local defect type and confidence level of the local defect;

[0008] A global detection model is invoked to perform global defect perception on the cigarette box packaging image, and a global defect perception result is obtained; the global defect perception result includes the probability that a defect corresponding to the target defect type exists in the cigarette box packaging image; the target defect type includes the global defect type and the local defect type;

[0009] Based on the global defect perception results, the local defect detection results are corrected to obtain the cigarette box packaging defect detection results.

[0010] Secondly, this application also provides a cigarette box packaging defect detection device, comprising:

[0011] The acquisition module is used to acquire images of multiple cigarette boxes after they have been packed into boxes.

[0012] The local defect detection module is used to call the local detection model to perform local defect detection on the cigarette box packaging image and obtain local defect detection results; the local defect detection results include the local defect type and confidence level of the local defect;

[0013] The global defect detection module is used to call the global detection model to perform global defect perception on the cigarette box packaging image and obtain the global defect perception result; the global defect perception result includes the probability that there is a defect corresponding to the target defect type in the cigarette box packaging image; the target defect type includes the global defect type and the local defect type;

[0014] The output module is used to correct the local defect detection results based on the global defect perception results to obtain the cigarette box packaging defect detection results.

[0015] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above method.

[0016] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0017] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0018] The aforementioned cigarette box packaging defect detection method, apparatus, equipment, medium, and program products, targeting the packaged cigarette box images, call a local detection model to perform local defect detection. This local defect detection effectively captures subtle texture changes in the image. Simultaneously, a global detection model is called for global defect perception. This global defect perception effectively perceives the overall arrangement of the entire box of cigarette boxes. Finally, the local defect detection results are corrected based on the global defect perception results to obtain the cigarette box packaging defect detection result. In this process, independent local and global detection models are used to detect defects from different dimensions, and the detection results from both dimensions are fused through result-level correction. This approach balances the recognition of local details and the identification of global arrangement patterns, improving the accuracy of cigarette box packaging defect recognition in complex defect scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a diagram illustrating the application environment of a cigarette box packaging defect detection method in one embodiment.

[0021] Figure 2 This is a flowchart illustrating a method for detecting defects in cigarette box packaging in one embodiment;

[0022] Figure 3 This is a schematic diagram of the correction process for the detection results in one embodiment;

[0023] Figure 4 This is a structural block diagram of a cigarette box packaging defect detection device in one embodiment;

[0024] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0027] The cigarette box packaging defect detection method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is illustrated. Terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on another network server.

[0028] Inspectors can initiate a cigarette box packaging defect detection task through terminal 101. Server 102 responds to the task by acquiring multiple images of the cigarette boxes after packaging. It then uses a local detection model to perform local defect detection on the images, obtaining local defect detection results. These results include the local defect type and confidence level. Next, a global detection model is used to perform global defect perception on the images, obtaining global defect perception results. These results include the probability that a defect of the target defect type exists in the images. The target defect type includes both global and local defect types. Based on the global defect perception results, the local defect detection results are corrected to obtain the final cigarette box packaging defect detection result. Finally, server 102 returns the cigarette box packaging defect detection result to terminal 101, allowing terminal 101 to display the result to the inspectors.

[0029] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0030] In one exemplary embodiment, such as Figure 2 As shown, a method for detecting defects in cigarette box packaging is provided, which is then applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 201 to 204. Wherein:

[0031] Step 201: Obtain images of multiple cigarette boxes after they have been packed.

[0032] Among them, the cigarette box packing image is an on-site imaging image of stacked cigarette boxes, which is collected at the packing station; by performing defect identification on the image, various packing defects generated during the cigarette box packing process can be identified.

[0033] Furthermore, packing defects can be divided into local defects and global defects. Local defects refer to defects that appear on the cigarette box itself, such as crushing (partial damage to the shape of the cigarette box), missing cigarette boxes, or cigarette boxes being obstructed by foreign objects. Global defects are overall assembly abnormalities that occur in the stacking and arrangement of the entire box of cigarette boxes, such as not being packed flat or having excessive gaps.

[0034] In some embodiments, the server can establish a communication connection with the image acquisition device at the packing station, and obtain the packing images of multiple cigarette boxes after they have been packed by calling the image acquisition device to acquire images in real time or reading the image data cached locally by the image acquisition device.

[0035] Step 202: Call the local detection model to perform local defect detection on the cigarette box packaging image and obtain the local defect detection results; the local defect detection results include the local defect type and confidence level of the local defect.

[0036] Among them, the local detection model is used to identify defect targets in a small local area in the image, and to realize defect category discrimination and defect feature extraction; for example, the local detection model can be the YOLO (You Only Look Once) model.

[0037] The YOLO model can be the YOLOv8 model, which consists of a CSP-Darknet (Cross-Stage-Partial Darknet) backbone, a PA Net (Path-Aggregation Network), and a Decoupled Head. Furthermore, the CSP-Darknet is used to extract features at different scales, and the PA Net is used for bidirectional fusion from top to bottom and bottom to top. Finally, the decoupled head outputs the bounding box coordinates, class probabilities, and confidence scores. The YOLOv8 model is jointly optimized using intersection-union loss, classification loss, and distribution focus loss.

[0038] The confidence level is used to characterize the degree of confidence that the local detection model determines that the current detected area belongs to the corresponding local defect type. Furthermore, the confidence level is determined by the local detection model through forward inference, based on the degree of matching between the identified defect features and the defect sample features learned by the model.

[0039] In some embodiments, the local defect detection result further includes a positioning box for locating the local defect. The positioning box marks the pixel area where the local defect is located in the cigarette box packaging image, thereby realizing the spatial location of the defect. For example, the local defect detection result may include a first local defect with the coordinates of the positioning box being (x1, y1, x2, y2), the local defect type being "crushed", and a confidence level of 0.8; and a second local defect with the pixel coordinates of the positioning box being (x3, y3, x4, y4), the local defect type being "missing strip", and a confidence level of 0.6.

[0040] In some embodiments, the training process of the local detection model includes: acquiring sample cigarette box packaging images and sample annotation information of the sample cigarette box packaging images; the sample annotation information includes the location of the ground truth bounding box of the local defect in the sample cigarette box packaging image and the local defect type; the target detection model to be trained performs local defect detection on the sample cigarette box packaging image to obtain the sample local defect prediction result; the sample local defect prediction result includes the location of the predicted bounding box of the local defect in the sample cigarette box packaging image and the local defect prediction type; based on the location of the predicted bounding box and the location of the ground truth bounding box, determine the cross-union ratio loss; based on the local defect prediction type and the local defect type, determine the classification loss; based on the difference between the discrete probability distribution of each side of the predicted bounding box and the endpoints of the discrete intervals where each side of the ground truth bounding box is located, determine the distribution focus loss; based on the cross-union ratio loss, the classification loss, and the distribution focus loss, train the target detection model to be trained to obtain the local detection model.

[0041] The sample labeling information is obtained by manually or semi-automatically labeling the images of sample cigarette boxes.

[0042] In some embodiments, the intersection-union loss, classification loss, and distribution focus loss can be weighted and summed to obtain the target loss. The target loss is then used to train the target detection model to be trained, resulting in a local detection model. Specifically, the network weight parameters of the target detection model to be trained are iteratively updated in the direction of minimizing the target loss until the training termination condition is met, resulting in a trained local detection model. The training termination condition can be that the model has reached a preset maximum number of training iterations, or the model detection accuracy on the validation set meets a preset accuracy threshold, or the target loss value on the validation set decreases to a preset loss threshold.

[0043] In some embodiments, the target loss can be obtained by the following formula:

[0044] .

[0045] in, To compare the losses, For classifying losses, For the distribution focus loss, , and These are the weighted weighting coefficients for each type of loss.

[0046] Furthermore, the loss is compared to the intersection. .

[0047] in, This represents the position of the predicted bounding box output by the model. This represents the location of the ground truth bounding box labeled in the sample. Represents the predicted bounding box With the actual annotation box The crossover ratio between them This represents the square of the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. The length of the diagonal of the smallest bounding rectangle that can simultaneously enclose both the predicted bounding box and the ground truth bounding box. An aspect ratio penalty term used to measure the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. This is the balance weight coefficient corresponding to the aspect ratio penalty term.

[0048] Classification loss .

[0049] in, is the number of bounding boxes in the positive samples (i.e., samples containing local defects); c represents the defect category index. In this embodiment, there are three types of local defects (crushing, missing strips, and occlusion). This represents the true probability (0 or 1) of the i-th box in class c. This represents the predicted probability of the i-th box in the c-th local defect type.

[0050] Distribution focus loss The expression is as follows:

[0051] .

[0052] in, This represents the number of bounding boxes in positive samples (i.e., samples containing local defects). The four sides of the bounding box are respectively These correspond to the top boundary, left boundary, bottom boundary, and right boundary, respectively. This represents the actual label of the i-th box corresponding to the s-th edge; and For the true value Two adjacent discrete interval endpoints; , This represents the predicted discrete probability distribution for the corresponding discrete interval output by the network.

[0053] Understandably, the intersection-union loss is used to optimize the spatial localization accuracy of the bounding box, taking into account the overlapping area, center point distance, and aspect ratio consistency. The classification loss independently optimizes the prediction probability of three types of local defects (crushing, missing strips, and occlusion) through the meta-cross-entropy loss. The distribution focus loss improves localization robustness by modeling each side of the bounding box as a discrete probability distribution and forcing the network to learn the probability distribution of adjacent intervals of the true value, replacing the traditional direct coordinate regression. Through the synergistic optimization of the three, the local detection model achieves high accuracy in the spatial localization and classification of local defects.

[0054] Step 203: Call the global detection model to perform global defect perception on the cigarette box packaging image and obtain the global defect perception result; the global defect perception result includes the probability that there is a defect corresponding to the target defect type in the cigarette box packaging image; the target defect type includes global defect type and local defect type.

[0055] Among them, the global detection model refers to the model used to perform image-level multi-label defect classification on the entire cigarette box packaging image. It outputs the probability of the existence of various defects on the entire image, but does not output the location bounding box of specific defects. In other words, global defect perception refers to judging the existence of various defects from the perspective of the entire image and completing image-level defect recognition.

[0056] Understandably, compared to local defect detection which requires outputting the bounding box location of the defect, global defect perception determines whether a corresponding defect appears in the image from a global perspective, without locating the specific coordinates of the defect. This allows it to focus on capturing the global arrangement pattern of the cigarettes in the entire box, overcoming the limitations of local detection.

[0057] In some embodiments, the training process of the global detection model includes: iteratively training the global detection model using cigarette box packaging image samples and corresponding image-level multi-label annotation information, updating network parameters in the direction of minimizing global loss, until a preset training termination condition is met, and obtaining a trained global detection model.

[0058] The global loss can be a multi-label classification loss. For example, the global loss can be calculated using the following formula: .

[0059] in, Representing the The true label of class defects Representing the The predicted probability of a type of defect, where N represents the total number of target defect types.

[0060] Step 204: Correct the local defect detection results based on the global defect perception results to obtain the cigarette box packaging defect detection results.

[0061] In some embodiments, the cigarette box packaging defect detection results include the defect type, defect location, and confidence level of the current cigarette box packaging image; of course, for global defects, the defect location is the entire image, and the defect location may not be output.

[0062] In some embodiments, the confidence level in the local defect detection results can be corrected using the global defect perception results. Based on the probability of the existence of various defects in the global detection output, the confidence level value of the corresponding defect type in the local defect detection results can be increased or decreased. False defects with a corrected confidence level lower than a preset confidence threshold are filtered out to obtain the cigarette box packaging defect detection results.

[0063] In other embodiments, the global defect perception results can be used to supplement and verify the local defect detection results: when the global defect perception results determine that there is a certain type of defect in the cigarette box packaging image, but the local defect detection results do not detect the same type of defect, the defect is added to the cigarette box packaging defect detection results; when the local defect detection results detect a certain type of defect, and the global defect perception results output that the probability of the existence of the same type of defect is lower than a preset probability threshold, the local defect detection results are removed to obtain the cigarette box packaging defect detection results.

[0064] In the aforementioned cigarette box packaging defect detection method, a local detection model is invoked to perform local defect detection on the packaged cigarette box image. This local defect detection can effectively capture subtle texture changes in the image. Simultaneously, a global detection model is invoked to perform global defect perception. This global defect perception can effectively perceive the overall arrangement of the entire box of cigarette boxes. Finally, the local defect detection results are corrected based on the global defect perception results to obtain the cigarette box packaging defect detection results. In the above process, defect detection is performed from different dimensions through independent local and global detection models, and the detection results from the two dimensions are fused through result-level correction. This approach takes into account both local detail recognition and global arrangement pattern recognition, improving the recognition accuracy of cigarette box packaging defects in complex defect scenarios.

[0065] In one exemplary embodiment, such as Figure 3 As shown, Figure 3 The process for confirming the inspection results of cigarette box packaging defects is shown, namely, step 204 includes steps 301 to 303. Wherein:

[0066] Step 301: Update the confidence of the local defects located in the local defect detection results based on the probability of the target defect type in the global defect perception results.

[0067] Specifically, since the global defect perception result contains both global and local defects, while the local defect detection result only contains local defects, the confidence level of the local defects located in the local defect detection result can be updated based on the probability of local defects in the global defect perception result.

[0068] For example, if the probability of a certain local defect in the global defect perception result is high, it means that there is a high probability that such a local defect exists in the entire cigarette box packaging image. In this case, the confidence of the local defect located in the local defect detection result can be increased. Furthermore, the corresponding confidence adjustment coefficient can be determined according to the specific probability interval in which the probability of the local defect in the global defect perception result is located. Different probability intervals correspond to different confidence adjustment coefficients. Then, the adjustment coefficient is multiplied by the original confidence of the local defect in the local defect detection result to complete the update of the confidence of the local defect.

[0069] For example, if the probability of a certain local defect in the global defect perception result is low, it means that the possibility of such a local defect existing in the entire cigarette box packaging image is small. In this case, the confidence of the local defect located in the local defect detection result can be reduced.

[0070] Step 302: Based on the global defect perception results, supplement the defects not included in the local defect detection results.

[0071] For example, when the global defect perception result determines that a certain type of target defect exists in the cigarette box packaging image, but the local defect detection result does not output a detection record for this type of defect, then the defect type is added to the defect set. Furthermore, for global defects such as not being packed flat or having excessive gaps, there is no need to add bounding box position information; only the defect type and the defect probability output by the global model are recorded. For local defects such as crushing or missing strips, the global model only confirms that the image has the defect, but cannot provide specific defect coordinates. Therefore, the defect type can be marked and awaited for subsequent manual review.

[0072] In some embodiments, if the probability of a certain type of local defect does not exist in the global defect perception result, but the local defect detection includes the detection result of that type of local defect, the confidence of the local defect can be downweighted. If the confidence after downweighting is lower than a preset confidence threshold, the local defect detection result is removed to filter out false defects generated by local detection. If the confidence after downweighting is still not lower than the preset confidence threshold, the local defect is marked as pending review.

[0073] Step 303: Based on the updated and / or supplementary results, obtain the defect detection results for cigarette box packaging.

[0074] In some embodiments, the local defects updated with confidence and the defects supplemented by the global defect perception results are merged, defect items with confidence levels lower than a preset output threshold are filtered out, and the defect type, defect location and final confidence level are retained to output complete cigarette box packaging defect detection results.

[0075] In other embodiments, after merging the local defects with the global supplementary defects after the confidence level is updated, the defect entries can be deduplicated: if the same defect type has both local detection entries and global supplementary entries, the local defect entries with location coordinates are retained, and the global supplementary entries containing only the defect type are discarded, and finally the cigarette box packaging defect detection results are obtained.

[0076] In this embodiment, the two output results of global defect perception and local defect detection are combined. On the one hand, the confidence of local defects is updated using image-level global probability, which can effectively suppress the false detection problem caused by texture and noise interference in local detection. On the other hand, global defect perception is used to supplement the defects missed by local detection, making up for the shortcomings of local detection which is limited by the limited receptive field, making it difficult to identify global defects in the overall arrangement of cigarette boxes and easy to miss small-scale defects. Thus, it can take into account both the defect location ability and the global semantic judgment ability of the whole image, reduce the defect missed detection rate and false detection rate, and improve the overall accuracy of cigarette box packaging defect detection.

[0077] In some embodiments, the confidence level of the local defect located in the local defect detection result is updated based on the probability of the target defect type in the global defect perception result, including:

[0078] Identify overlapping defect types detected in both local and global defect perception results; when the probability of overlapping defect types in global defect perception results is lower than the first probability threshold, reduce the confidence level of overlapping defect types in local detection results and mark the defects corresponding to the overlapping defect types as defects to be re-inspected.

[0079] For example, the first probability threshold is set to 0.3. If the local defect detection result detects a crushing defect, but the global defect perception result outputs a crushing defect probability of only 0.2, which is less than the first probability threshold of 0.3, then the original confidence of the crushing defect is lowered, and the defect is marked as a defect to be re-inspected, instead of being directly determined as a confirmed defect, and is left for manual review and confirmation.

[0080] In the above embodiments, by cross-validating the results of local detection and global perception, defects are not directly deleted when there is insufficient evidence in the global dimension. Instead, the confidence level is lowered and the defects are marked for re-inspection. This avoids the risk of missed detection caused by directly eliminating the detection results. It can filter out high-probability local false detections and retain doubtful defects for manual review, thus balancing the detection accuracy and fault tolerance of cigarette box packaging defect detection.

[0081] In some embodiments, based on the global defect perception results, the defects included in the local defect detection results are supplemented, including:

[0082] Identify candidate defect types with a probability higher than the second probability threshold in the global defect perception results; when the local defect detection results do not contain candidate defect types, identify the defects corresponding to the candidate defect types as supplementary defects and generate frameless alarms for the candidate defect types.

[0083] For example, the second probability threshold is set to 0.7. If the probability of the global defect perception result outputting the unflattened defect is 0.82, which is higher than the second probability threshold of 0.7, but the unflattened defect type is not detected in the local defect detection result, then the unflattened defect is identified as a supplementary defect, a frameless alarm is generated, only the defect type and the global prediction probability are recorded, and the bounding box coordinates are not output.

[0084] In some embodiments, the generated frameless alarm can be output along with the cigarette box packaging defect detection results to alert the inspector that the image has such a defect, but the algorithm fails to locate the specific location of the defect, and the entire image needs to be manually reviewed for confirmation.

[0085] In the above embodiments, high-probability candidate defect types are screened by global defect perception, and frameless alarms are generated to supplement local detection when defects are missed. This can make up for the defects that are easily missed by local detection due to the limitation of the receptive field, such as global defects and some small local defects. There is no need to force the output of defect location. The early warning of missed defects is achieved without relying on the positioning box, thereby reducing the overall missed detection rate of cigarette box packaging defects.

[0086] In some embodiments, the global detection model includes a multi-scale feature extraction network, a bidirectional feature pyramid network, a state-space sequence modeling network, and a multi-label classification head; the global detection model is invoked to perform global defect perception on the cigarette box packaging image, and the global defect perception result is obtained, including:

[0087] A multi-scale feature extraction network is used to extract multi-scale features from cigarette box packaging images, resulting in a multi-scale spatial feature map. This multi-scale spatial feature map includes feature maps at multiple levels, corresponding to detailed texture features, structural arrangement features, and global semantic features, respectively. A bidirectional feature pyramid network is used to perform bidirectional cross-scale fusion of the multi-scale spatial feature map, resulting in a fused feature. A state-space sequence modeling network expands the fused feature into a sequence along the spatial dimension and performs state transfer and global context aggregation along the sequence direction, resulting in a sequence aggregation feature that represents the global arrangement state. A multi-label classification head performs multi-label classification prediction based on the sequence aggregation feature, resulting in a global defect perception result.

[0088] In some embodiments, the multi-scale feature extraction network achieves multi-scale feature extraction through a composite scaling mechanism. This design can maximize feature representation under computational resource constraints. Specifically, it captures detailed textures such as cigarette strip edges and gaps through shallow features and determines the overall structural and semantic features of the box through deep feature encoding, thereby supplementing the shortcomings of local defect identification in small-scale defect feature extraction.

[0089] For example, multi-scale feature extraction networks can employ EfficientNet-B7, ResNet (Residual Network), VGG (Visual Geometry Group), etc.

[0090] Taking EfficientNet-B7 as an example, its input is a preprocessed original cigarette box RGB (Red-Green-Blue) image tensor. The model is loaded in feature extraction mode, and the output is a multi-scale spatial feature map of the last three layers. The spatial resolution of the three feature maps is halved in turn, and the number of channels increases layer by layer, corresponding to shallow detailed texture features, middle-layer structural arrangement features, and deep global semantic features, respectively. Its core is to balance feature expression ability and computational efficiency by adjusting the depth, width and resolution of the network.

[0091] Among them, the composite scaling mechanism uses a composite scaling factor. network depth ,width and resolution Scaling is performed using the following formula:

[0092] .

[0093] in , , These are constant coefficients. By optimizing these coefficients, the optimal network scaling ratio can be found under given computational resource constraints to achieve the best feature extraction performance.

[0094] In some embodiments, in order to facilitate multi-scale feature fusion in the subsequent bidirectional feature pyramid network, after obtaining the multi-scale features, the three different channel dimensions can be uniformly mapped to 256 dimensions through their respective independent 1×1 convolutional projection layers, which can then be used as the input to the bidirectional feature pyramid network.

[0095] In some embodiments, the Bidirectional Feature Pyramid Network (BiFPN) dynamically fuses feature maps from different levels through a bidirectional path, both top-down and bottom-up. Its core function is to highlight key information through learnable weight allocation, enhancing the model's ability to detect global layout defects such as "uneven mounting" and "excessive gaps."

[0096] Specifically, BiFPN receives three layers of 256-dimensional multi-scale features aligned by 1×1 convolutional projection. It then uses nearest-neighbor interpolation to perform layer-by-layer upsampling along a top-down direction to achieve resolution alignment, injecting deep global semantic information into shallow local detail features. Simultaneously, it performs downsampling fusion along a bottom-up direction to achieve bidirectional cross-scale path propagation, using learnable normalized weights. Dynamically balance the feature contributions of each layer to achieve adaptive weighted fusion of multi-scale features.

[0097] The adaptive weighted fusion of multi-scale features can be expressed as the following calculation formula:

[0098] .

[0099] Where j represents the index variable of the fusion path; This represents the sum of the weights over all fusion paths (i.e., the total weights). This represents the feature map input to the i-th fusion path; These are learnable weights used to adjust the importance of each input feature. It is a small constant used for numerical stability. It is used to fuse features at different resolutions multiple times through a bidirectional path to enhance the model's ability to perceive global and local features.

[0100] In some embodiments, the state-space sequence modeling network is also referred to as the state-space model (SSM).

[0101] For example, a state-space sequence modeling network can be a Mamba (a state-space architecture) network, whose core relies on one-dimensional sequence scanning and gating mechanisms to dynamically aggregate long-distance dependency information.

[0102] In the scenario of detecting defects in cigarette pack packaging, the Mamba network relies on its long sequence modeling capability to capture the global context information of the entire pack of cigarettes, overcoming the shortcomings of traditional convolutional neural networks in modeling long-distance dependencies. It is suitable for identifying global defects such as "not packed flat" and "excessive gaps".

[0103] Specifically, Mamba serializes the two-dimensional enhanced feature map output by BiFPN fusion: through tensor permutation, such as permute(0,2,3,1), and reshaping operations, such as reshape(b,h×w,c), the two-dimensional feature map is flattened into a one-dimensional spatial sequence, where b is the batch size, h×w is the sequence length, and c=256 is the channel dimension. Then, for the spatial sequence, its long-range dependency modeling capability with linear complexity O(N) is used to capture the global arrangement pattern of the whole box of cigarettes, thereby capturing the global arrangement context of the whole box of cigarettes.

[0104] For example, a two-dimensional cigarette box image Convert to pseudo-sequence data in row / column order , where L=H×W is the sequence length.

[0105] The state-space model describes information transfer through the following equations:

[0106] .

[0107] in, The input sequence at time... The value, It is in a hidden state. It is the output. , , , It is a learnable parameter matrix. Mamba introduces a gating mechanism on this basis to dynamically control the aggregation and transmission of information in order to better capture long-distance dependencies. The formula can be extended to include gating units. Format:

[0108] .

[0109] in This indicates element-wise multiplication. It is calculated through a gating function and is used to adjust the degree to which the input information updates the hidden state.

[0110] Finally, the multi-label classification head performs multi-label classification prediction based on sequence aggregation features, outputting the probability of each defect category. ,in, It is the first multi-label classification head The raw score of each output unit.

[0111] In the above embodiments, the model structure of a multi-scale feature extraction network, a bidirectional feature pyramid network, a state-space sequence modeling network, and a multi-label classification head can accurately capture detailed texture features, structural arrangement features, and global semantic features through the multi-scale feature extraction network, taking into account both local detailed features and global arrangement features. Then, the bidirectional feature pyramid network adaptively fuses multi-scale features to compensate for the insufficient expression of single-scale features. The state-space sequence modeling network effectively models the long-distance spatial dependencies of the image and explores the overall arrangement pattern of the entire cigarette box. Finally, the multi-label classification head is combined to achieve simultaneous discrimination of multiple types of defects, effectively making up for the shortcomings of the single local detection model, which has a limited field of view and weak global defect recognition ability, and improving the completeness and accuracy of detecting various defects in cigarette box packaging.

[0112] In some embodiments, before inputting images of cigarette packs into a model (including local detection models and global detection models), it is usually necessary to normalize the image data, scaling the pixel values ​​to a specific range, in order to accelerate the convergence speed of model training and improve stability.

[0113] For example, a normalization method can be used to divide each pixel value of the image by 255, changing its range from [0, 255] to [0, 1]. ,in, These are the original pixel values. It is the normalized pixel value.

[0114] In other embodiments, normalization can also be performed using the mean and standard deviation to convert the image data into a distribution with a mean of 0 and a standard deviation of 1, i.e., for each channel of the image. Normalized pixel values ,in, It is the image number The pixel values ​​of each channel. It is the mean of that channel. It is the standard deviation of the channel, usually. =[0.485,0.456,0.406], =[0.229,0.224,0.225].

[0115] In some embodiments, when training a neural network model, gradient descent is typically used to update the model's parameters in order to minimize the loss function.

[0116] Examples of gradient descent algorithms include Adam (Adaptive Moment Estimation), Adam W (Adaptive Moment Estimation Weight Decay), and SGD (Stochastic Gradient Descent).

[0117] Taking SGD as an example, the parameter update formula is:

[0118]

[0119] in, It is the first Model parameters at the next iteration It's the learning rate. It is a loss function Regarding parameters In the The gradient at the next iteration.

[0120] Taking Adam as an example, it includes updates to the first-order moment estimate and the second-order moment estimate, as well as parameter updates, as shown in the following formula:

[0121] First-order moment estimation: ;

[0122] Second-order moment estimation: ;

[0123] Bias correction: , ;

[0124] Parameter update: .

[0125] in, and These are the first-order moment estimate and the second-order moment estimate, respectively. and It is the attenuation rate. It is the first The gradient at the next iteration It is a small constant used for numerical stability.

[0126] In some embodiments, the performance of the model can be determined by evaluation metrics, thereby determining whether the training termination condition has been met; the evaluation metrics of the model (including local detection models and global detection models) may include accuracy, precision, recall and F1 score.

[0127] This application addresses the common shortcomings of existing cigarette box packaging defect detection technologies, including the separation of global and local information, high false negative rates for small-scale defects, weak ability to identify global layout defects, conflicts in multi-model feature fusion, non-standard data annotation, and difficulty in balancing accuracy and efficiency. It constructs an architecture combining multi-scale feature extraction, bidirectional feature fusion, and global feature extraction via a state-space network. This architecture achieves global defect detection, while local target detection accurately locates local defects. Global and local defect detection are completely decoupled and independently optimized during the training phase, fundamentally avoiding gradient conflicts, feature entanglement, and training instability caused by multi-task joint training. The advantages of heterogeneous outputs are only achieved through a post-processing fusion strategy during the inference phase. Leveraging Mamba's powerful linear complexity long sequence modeling capabilities, this application accurately captures the overall arrangement trend and global spatial distribution patterns of the box, significantly improving the identification ability of global layout defects without fixed boundaries, such as "uneven packing" and "excessive gaps." Simultaneously, it utilizes the advantages of local defect detection to quickly and accurately output the bounding box positions and confidence scores of local defects such as "crushed," "missing strips," and "occlusion," achieving a balance between global semantic perception and precise local localization, significantly improving efficiency. This improves the comprehensiveness of defect detection. Simultaneously, EfficientNet-B7 utilizes a composite scaling mechanism to mine multi-level, multi-scale feature information under limited computational resources, fully preserving shallow details and textures such as cigarette edges and minor damage, effectively capturing small-scale weak defects such as slight crushing, thus overcoming the insufficient feature extraction capability of existing models for minute defects, significantly reducing the false negative rate of small-scale defects, and improving the accuracy of full-scale defect detection. Finally, this application establishes a two-layer conflict resolution mechanism of "local spatial localization × global existence verification," strictly adhering to the principle of spatial localization priority, and ensuring the accuracy of defect coordinates. The results are entirely based on local defect detection results. When the probability of the corresponding defect in the global defect perception result is lower than the threshold, an inhibition coefficient is applied to the confidence of the local detection box and it is marked as needing re-inspection. This effectively suppresses local false detections caused by lighting, wrinkles, and texture interference. When the probability of the defect in the global defect perception result is higher than the threshold but the local model has no detection results, it is judged as a distributed weak defect and a no-box alarm is output. This makes up for the missed detection problem caused by the threshold limitation and field of view limitation of the local model. At the same time, it supports multi-label joint decision output, realizing bidirectional offsetting of false detection and missed detection, and further improving the detection and judgment accuracy.

[0128] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0129] Based on the same inventive concept, this application also provides a cigarette box packaging defect detection device for implementing the cigarette box packaging defect detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the cigarette box packaging defect detection device provided below can be found in the limitations of the cigarette box packaging defect detection method described above, and will not be repeated here.

[0130] In one exemplary embodiment, such as Figure 4 As shown, a cigarette box packaging defect detection device 400 is provided, comprising:

[0131] The acquisition module 401 is used to acquire images of multiple cigarette boxes after they have been packed.

[0132] The local defect detection module 402 is used to call the local detection model to perform local defect detection on the cigarette box packaging image and obtain the local defect detection result; the local defect detection result includes the local defect type and confidence level of the local defect.

[0133] The global defect detection module 403 is used to call the global detection model to perform global defect perception on the cigarette box packaging image and obtain the global defect perception result. The global defect perception result includes the probability that there is a defect corresponding to the target defect type in the cigarette box packaging image. The target defect type includes global defect type and local defect type.

[0134] Output module 404 is used to correct the local defect detection results based on the global defect perception results to obtain the cigarette box packaging defect detection results.

[0135] In some embodiments, the output module 404 is used to update the confidence level of the local defects located in the local defect detection results based on the probability of the target defect type in the global defect perception results, and / or to supplement the local defect detection results with defects not included in the local defect detection results based on the global defect perception results; and to obtain the cigarette box packaging defect detection results based on the updated results and / or the supplemented results.

[0136] In some embodiments, the output module 404 is used to determine the overlapping defect types detected in both the local detection results and the global defect perception results; when the probability of the overlapping defect type in the global defect perception results is lower than the first probability threshold, the confidence of the overlapping defect type in the local detection results is reduced, and the defect corresponding to the overlapping defect type is marked as a defect to be re-inspected.

[0137] In some embodiments, the output module 404 is used to determine candidate defect types with a probability higher than a second probability threshold in the global defect perception results; when the local defect detection results do not contain candidate defect types, the defect corresponding to the candidate defect type is determined as a supplementary defect, and a frameless alarm for the candidate defect type is generated.

[0138] In some embodiments, the global defect detection model includes a multi-scale feature extraction network, a bidirectional feature pyramid network, a state-space sequence modeling network, and a multi-label classification head; the global defect detection module 403 is used to extract multi-scale features from the cigarette box packaging image by the multi-scale feature extraction network to obtain a multi-scale spatial feature map; the multi-scale spatial feature map includes feature maps of multiple levels corresponding to detail texture features, structural arrangement features, and global semantic features respectively; the bidirectional feature pyramid network performs bidirectional cross-scale fusion of the multi-scale spatial feature map to obtain fused features; the state-space sequence modeling network expands the fused features into a sequence along the spatial dimension, and performs state transfer and global context aggregation along the sequence direction to obtain a sequence aggregation feature used to represent the global arrangement state; the multi-label classification head performs multi-label classification prediction based on the sequence aggregation feature to obtain the global defect perception result.

[0139] In some embodiments, the cigarette box packaging defect detection device 400 further includes a training module for acquiring sample cigarette box packaging images and sample annotation information of the sample cigarette box packaging images; the sample annotation information includes the position of the ground truth bounding box of a local defect in the sample cigarette box packaging image and the ground truth type of the local defect; the target detection model to be trained performs local defect detection on the sample cigarette box packaging image to obtain the sample local defect prediction result; the sample local defect prediction result includes the position of the predicted bounding box of the local defect in the sample cigarette box packaging image and the local defect prediction type; based on the position of the predicted bounding box and the position of the ground truth bounding box, the intersection-over-union (IoU) loss is determined; based on the local defect prediction type and the local defect ground truth type, the classification loss is determined; based on the difference between the discrete probability distribution of each side of the predicted bounding box and the endpoints of the discrete intervals where each side of the ground truth bounding box is located, the distribution focus loss is determined; based on the IoU loss, the classification loss, and the distribution focus loss, the target detection model to be trained is trained to obtain the local detection model.

[0140] Each module in the aforementioned cigarette box packaging defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0141] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data related to cigarette box packaging defect detection. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a cigarette box packaging defect detection method.

[0142] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0143] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0144] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0145] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0146] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0147] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0148] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0149] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting defects in cigarette box packaging, characterized in that, The method includes: Acquire multiple images of cigarette boxes after they have been packed into a box; The local detection model is invoked to perform local defect detection on the cigarette box packaging image, and the local defect detection results are obtained; the local defect detection results include the local defect type and confidence level of the local defect; A global detection model is invoked to perform global defect perception on the cigarette box packaging image, and a global defect perception result is obtained; the global defect perception result includes the probability that a defect corresponding to the target defect type exists in the cigarette box packaging image; the target defect type includes the global defect type and the local defect type; Based on the global defect perception results, the local defect detection results are corrected to obtain the cigarette box packaging defect detection results.

2. The method according to claim 1, characterized in that, The step of correcting the local defect detection results based on the global defect perception results to obtain the cigarette box packaging defect detection results includes: Based on the probability of the target defect type in the global defect perception result, the confidence level of the local defect located in the local defect detection result is updated, and / or, Based on the global defect perception results, defects not included in the local defect detection results are supplemented; Based on the updated and / or supplementary results, the defect detection results of the cigarette box packaging are obtained.

3. The method according to claim 2, characterized in that, The step of updating the confidence level of the local defect located in the local defect detection result based on the probability of the target defect type in the global defect perception result includes: Determine the types of overlapping defects detected in both the local detection results and the global defect perception results; When the probability of the overlapping defect type in the global defect perception result is lower than the first probability threshold, the confidence of the overlapping defect type in the local detection result is reduced, and the defect corresponding to the overlapping defect type is marked as a defect to be re-inspected.

4. The method according to claim 2, characterized in that, The step of supplementing the defects contained in the local defect detection results based on the global defect perception results includes: Identify candidate defect types with a probability higher than a second probability threshold from the global defect perception results; When the local defect detection result does not include the candidate defect type, the defect corresponding to the candidate defect type is identified as a supplementary defect, and a frameless alarm for the candidate defect type is generated.

5. The method according to claim 1, characterized in that, The global detection model includes a multi-scale feature extraction network, a bidirectional feature pyramid network, a state-space sequence modeling network, and a multi-label classification head; The step of calling the global detection model to perform global defect perception on the cigarette box packaging image, and obtaining the global defect perception result, includes: The multi-scale feature extraction network is used to extract multi-scale features from the cigarette box packaging image to obtain a multi-scale spatial feature map; the multi-scale spatial feature map includes feature maps of multiple levels corresponding to detailed texture features, structural arrangement features and global semantic features respectively; The bidirectional feature pyramid network performs bidirectional cross-scale fusion of the multi-scale spatial feature map to obtain fused features; The state-space sequence modeling network expands the fused features into a sequence along the spatial dimension, and performs state transfer and global context aggregation along the sequence direction to obtain a sequence aggregation feature that represents the global permutation state. The multi-label classification head performs multi-label classification prediction based on the sequence aggregation features to obtain the global defect perception result.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain a sample cigarette box packaging image and sample annotation information of the sample cigarette box packaging image; the sample annotation information includes the position of the ground truth bounding box of the local defect in the sample cigarette box packaging image and the ground truth type of the local defect; The target detection model to be trained performs local defect detection on the sample cigarette box packaging image to obtain the sample local defect prediction result; the sample local defect prediction result includes the position of the prediction box of the local defect in the sample cigarette box packaging image and the local defect prediction type; Based on the positions of the predicted bounding boxes and the ground truth bounding boxes, the cross-union loss is determined; Based on the predicted local defect type and the actual local defect type, determine the classification loss; Based on the difference between the discrete probability distribution of each side of the predicted box and the endpoints of the discrete intervals where each side of the true box is located, the distribution focus loss is determined. The target detection model to be trained is trained based on the intersection-union loss, the classification loss, and the distribution focus loss to obtain the local detection model.

7. A device for detecting defects in cigarette box packaging, characterized in that, The device includes: The acquisition module is used to acquire images of multiple cigarette boxes after they have been packed into boxes. The local defect detection module is used to call the local detection model to perform local defect detection on the cigarette box packaging image and obtain local defect detection results; the local defect detection results include the local defect type and confidence level of the local defect; The global defect detection module is used to call the global detection model to perform global defect perception on the cigarette box packaging image and obtain the global defect perception result; the global defect perception result includes the probability that there is a defect corresponding to the target defect type in the cigarette box packaging image; the target defect type includes the global defect type and the local defect type; The output module is used to correct the local defect detection results based on the global defect perception results to obtain the cigarette box packaging defect detection results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images