Intelligent labeling detection method and system
Through intelligent labeling detection method, combined with visual detection and laser code recognition, automatic detection and removal of labeling and laser codes is achieved, solving the problems of low manual detection efficiency and poor accuracy in the existing technology, and improving the automation of the production line level.
Patent Information
- Application Number
- CN202411052298.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-02
AI Technical Summary
The prior art has the risk of missed and missed detection in labeling and laser code detection, and cannot automatically identify subtle defects and wrong engraving, resulting in large workload and low efficiency.
Using intelligent labeling detection method, the labeling image and laser code image are jointly identified through visual detection equipment, laser code recognition model and RFID equipment, and the multi-defect joint recognition model and indefinite length laser code recognition model are used to achieve defects such as offset and wrinkle Automatic detection and accurate identification of laser codes generate a culling signal to automatically eliminate unqualified products.
It improves the accuracy of labeling and laser code detection, reduces manual errors, realizes automatic removal of NG products, and is suitable for product packaging and production needs with anti-counterfeiting requirements.
Smart Images

Figure CN118577517B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of mechanical automation technology, and more specifically, to an intelligent labeling detection method and system. Background Art
[0002] Labeling machines and laser machines are widely used in the packaging industry, but they also have many shortcomings. For example, when labels are pasted, there are defects such as up and down, left and right deviations, and slight wrinkles on the labels, which have a high risk of missed detection and wrong detection; in traditional laser engraving applications, the characters to be engraved are manually copied to the laser machine in advance, and then the laser machine automatically engraves the characters, which is prone to the risk of duplicate codes caused by manual copying errors and repeated engraving by the laser machine; products with unqualified labels and failed laser code engraving need to be eliminated.
[0003] At present, products with unqualified labels and unqualified laser codes are mainly removed manually. When labeling and coding are automated, the workload of quality inspectors is huge. It is difficult to accurately identify minor defects such as offset and wrinkles in the labeling, and it is impossible to quickly determine the misprinting of the laser code.
[0004] Based on this, the present application provides an intelligent labeling detection method and system to solve the above problems. Summary of the invention
[0005] The purpose of this application is to provide an intelligent labeling detection method and system to solve the problem that the existing labeling detection cannot automatically reject products with unqualified labels and unqualified laser codes; to propose an integrated device, which jointly generates a rejection signal based on the labeling detection results and the laser code detection results, realizes labeling defect detection and laser code character recognition, and realizes the linkage control of visual inspection equipment and sorting equipment.
[0006] The present application first provides an intelligent labeling detection method, including: obtaining a product labeling image collected by a first visual inspection device, a product laser code image collected by a second visual inspection device, and a product RFID label collected by a dual-frequency RFID device; detecting the labeling image based on a multi-defect joint recognition model to obtain a degree of offset and a degree of wrinkle, and comparing the degree of offset and the degree of wrinkle with a set degree of offset and a set degree of wrinkle to obtain a labeling detection result; detecting the laser code image based on a variable-length laser code recognition model to obtain a laser code recognition result, and comparing the laser code recognition result with the product RFID label to obtain a laser code detection result; obtaining a joint control instruction according to the labeling detection result and the laser code detection result, the joint control instruction including a rejection instruction and a pass instruction, and the joint control instruction is used to control a rejection device to perform a rejection or pass operation.
[0007] By adopting the above technical solution, labeling detection and laser code detection are used to realize automatic rejection of NG products; the common defects such as offset, rotation, and wrinkles of the labeling image are identified through the trained multi-defect joint recognition model to improve the accuracy of labeling image defect detection; the laser code in the laser code image is identified and compared through the variable-length laser code recognition model to reduce the risk of misprinting of the laser code. This solution can be widely applied to the production needs of various products / product packaging with anti-counterfeiting requirements.
[0008] In a possible implementation, the labeled image is detected based on a multi-defect joint recognition model to obtain a degree of deviation and a degree of wrinkle, including: preprocessing the labeled image, the preprocessing including: grayscale processing and filtering processing; using a grayscale difference statistics method to calculate the contrast of the labeled image, extracting feature points from the labeled image according to the contrast of the labeled image, querying a corresponding standard labeled image in a template library according to the feature points, the template library containing a plurality of standard labeled images and corresponding feature points; bringing the standard labeled image and the labeled image into a trained multi-defect joint recognition model to obtain a degree of deviation and a degree of wrinkle of the label.
[0009] In a possible implementation, a grayscale difference statistical method is used to calculate the contrast of the labeled image, feature points are extracted from the labeled image according to the contrast of the labeled image, and a corresponding standard labeled image is queried in a template library according to the feature points, including: taking a pixel point in the labeled image as the center pixel point, constructing a 16-pixel Bresenham circle, calculating the average grayscale value of the 16 pixels on the Bresenham circle, calculating the difference between the grayscale value of the 16 pixels and the average grayscale value, calculating the pixel distribution probability of the difference, and calculating Bres according to the pixel distribution probability. The method comprises the following steps: calculating the corner point threshold of the central pixel point according to the contrast, selecting 4 pixel points with equal spacing from the Bresenham circle, calculating the gray value difference between the 4 pixel points and the central pixel point, and when there are more than 3 gray value differences that are greater than or less than the corner point threshold, regarding the central pixel point as a feature point, traversing each pixel point of the labeled image until all feature points of the labeled image are obtained; calculating the BRIEF descriptors of all feature points of the labeled image, and querying the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
[0010] In a possible implementation, a standard labeling image, a labeling image and their corresponding feature points are brought into a trained multi-defect joint recognition model to obtain the offset degree and wrinkle degree of the labeling, wherein the multi-defect joint recognition model includes: an input module for receiving a labeling image and a standard labeling image; an offset prediction module for predicting an offset and a rotation angle based on the labeling image and the standard labeling image; a wrinkle prediction module for correcting the labeling image according to the offset and the rotation angle, and predicting the wrinkle area based on the corrected labeling image and the standard labeling image; an output module for comparing the offset, the rotation angle and the wrinkle area with a preset offset threshold, a rotation angle threshold and a wrinkle area threshold to obtain a labeling image detection result, including: a labeling image detection passed and a labeling image detection failed.
[0011] In a possible implementation, the laser code image is detected based on a variable-length laser code recognition model to obtain a laser code recognition result, and the laser code recognition result is compared with a product RFID tag to obtain a laser code detection result, including: preprocessing the laser code image, the preprocessing including grayscale processing and filtering processing; inputting the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, the CRNN neural network including a recognition layer for extracting a sequence from the laser code image, and a prediction layer for predicting the sequence; comparing the laser code recognition result with the product RFID tag to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
[0012] The present application also provides an intelligent labeling detection system, including: a data acquisition unit, used to acquire a product labeling image collected by a first visual inspection device, a product laser code image collected by a second visual inspection device, and a product RFID label collected by a dual-frequency RFID device; a labeling detection unit, used to detect the labeling image based on a multi-defect joint recognition model to obtain a degree of offset and a degree of wrinkle, and compare the degree of offset and the degree of wrinkle with a set degree of offset and a set degree of wrinkle to obtain a labeling detection result; a laser code detection unit, used to detect the laser code image based on a variable-length laser code recognition model to obtain a laser code recognition result, and compare the laser code recognition result with the product RFID label to obtain a laser code detection result; an NG control unit, used to obtain a joint control instruction based on the labeling detection result and the laser code detection result, the joint control instruction including a rejection instruction and a pass instruction, and the joint control instruction is used to control a rejection device to perform a rejection or pass operation.
[0013] In a possible implementation, the labeling detection unit specifically includes: a labeling image preprocessing unit, used to preprocess the labeling image, the preprocessing including: grayscale processing and filtering processing; a feature point extraction unit, used to calculate the contrast of the labeling image by using a grayscale difference statistics method, extract feature points from the labeling image according to the contrast of the labeling image, and query the corresponding standard labeling image in a template library according to the feature points, the template library contains multiple standard labeling images and corresponding feature points; a labeling defect analysis unit, used to bring the standard labeling image and the labeling image into a trained multi-defect joint recognition model to obtain the degree of offset and wrinkle of the label.
[0014] In a possible implementation, the feature point extraction unit specifically includes: a gray value statistics unit, which is used to construct a 16-pixel Bresenham circle with a pixel point in the labeled image as the central pixel point, calculate the average gray value of the 16 pixels on the Bresenham circle, calculate the difference between the gray value of the 16 pixels and the average gray value, calculate the pixel distribution probability of the difference, and calculate the contrast of the Bresenham circle area according to the pixel distribution probability; a feature point analysis unit, which is used to calculate the corner point threshold of the central pixel point according to the contrast, select 4 pixels with equal spacing from the Bresenham circle, calculate the gray value difference between the 4 pixels and the central pixel point, and when there are more than 3 gray value differences that are greater than or less than the corner point threshold, the central pixel point is regarded as a feature point, and each pixel point of the labeled image is traversed until all feature points of the labeled image are obtained; a standard labeled image extraction unit, which is used to calculate the BRIEF descriptors of all feature points of the labeled image, and query the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
[0015] In a possible implementation, the multi-defect joint recognition model in the labeling defect analysis unit includes: an input module for receiving a labeling image and a standard labeling image; an offset prediction module for predicting an offset and a rotation angle based on the labeling image and the standard labeling image; a wrinkle prediction module for correcting the labeling image according to the offset and the rotation angle, and predicting the wrinkle area based on the corrected labeling image and the standard labeling image; an output module for comparing the offset, the rotation angle and the wrinkle area with a preset offset threshold, a rotation angle threshold and a wrinkle area threshold to obtain a labeling image detection result, including: a labeling image detection passed and a labeling image detection failed.
[0016] In a possible implementation, the laser code detection unit specifically includes: a laser code image preprocessing unit, used to preprocess the laser code image, wherein the preprocessing includes grayscale processing and filtering processing; a laser code recognition unit, used to input the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, wherein the CRNN neural network includes a recognition layer for extracting a sequence from the laser code image and a prediction layer for predicting the sequence; a laser code detection unit, used to compare the laser code recognition result with the product RFID label to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
[0017] Compared with the prior art, this application has the following beneficial effects: label detection and laser code detection are used to automatically remove NG products, and it is applicable to existing label and laser code anti-counterfeiting product production lines. The trained multi-defect joint recognition model is used to identify common defects such as offset, rotation, and wrinkles in label images, thereby improving the accuracy of label image defect detection. The variable-length laser code recognition model is used to identify and compare the laser code in the laser code image, thereby reducing the risk of misprinting the laser code. This solution can be widely applied to the production needs of various products / product packaging with anti-counterfeiting requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0019] Figure 1 A schematic diagram of a flow chart of an intelligent labeling detection method provided in an embodiment of the present application;
[0020] Figure 2 This is a schematic diagram of the structure of a 16-pixel Bresenham circle;
[0021] Figure 3 A schematic diagram of the structure of an intelligent labeling detection system provided in an embodiment of the present application;
[0022] Figure 4 The figure is a flow chart of the linkage control of the first visual inspection device and the second visual inspection device for the rejection device. DETAILED DESCRIPTION
[0023] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.
[0024] See also Figure 1 As shown, Figure 1A flow chart of an intelligent labeling detection method provided in an embodiment of the present application. The method includes: S100, obtaining a product labeling image collected by a first visual inspection device, a product laser code image collected by a second visual inspection device, and a product RFID label collected by a dual-frequency RFID device; S200, detecting the labeling image based on a multi-defect joint recognition model to obtain a degree of offset and a degree of wrinkle, and comparing the degree of offset and the degree of wrinkle with a set degree of offset and a set degree of wrinkle to obtain a labeling detection result; S300, detecting the laser code image based on a variable-length laser code recognition model to obtain a laser code recognition result, and comparing the laser code recognition result with the product RFID label to obtain a laser code detection result; S400, obtaining a joint control instruction according to the labeling detection result and the laser code detection result, the joint control instruction including a rejection instruction and a pass instruction, and the joint control instruction is used to control the rejection device to perform a rejection or pass operation.
[0025] Specifically, this solution collects the product label image through the first visual inspection device, collects the product laser code image through the second visual inspection device, and collects the product RFID tag built into the product label image through the dual-frequency RFID device. The label detection results and laser code detection results are given through the deep learning intelligent algorithm, and the joint control instructions are generated through the two results. The joint control instructions are sent to the rejection device plc, and the rejection device plc controls the magnetic levitation motor to perform the rejection action, so as to retain qualified products and reject NG (no good) products; through the cooperation of label detection, laser code detection and rejection equipment, the real-time rejection of problematic products is dynamically completed to prevent label defects and laser code error products from entering the market.
[0026] The improvement of this application is that labeling detection and laser code detection are used to realize the automatic rejection of NG products. Compared with the traditional manual rejection, or the operation mode of controlling the rejection of NG products by labeling detection or laser code detection alone, this solution combines the three to realize the automatic rejection of NG products based on labeling detection and laser code detection, which is suitable for existing labeling and laser code anti-counterfeiting product production lines. Furthermore, through the trained multi-defect joint recognition model, the common defects such as offset, rotation, and wrinkles of the labeling image are identified to improve the accuracy of labeling image defect detection. The laser code in the laser code image is identified and compared through the variable-length laser code recognition model to reduce the risk of misprinting of the laser code. This solution can be widely applied to the production needs of various products / product packaging with anti-counterfeiting requirements.
[0027] Step S100 is to obtain the product label image collected by the first visual inspection device, the product laser code image collected by the second visual inspection device, and the product RFID tag collected by the dual-frequency RFID device. When the product passes through the first visual inspection device, the second visual inspection device, and the dual-frequency RFID device, the product label image, the product laser code image, and the product RFID tag are collected.
[0028] Specifically, in terms of visual inspection equipment, the pixels that match the field of view can be calculated based on the field of view of the label image and the laser code image, and a suitable industrial camera can be selected based on the pixels. The camera lens can be selected based on the distance between the inspected position and the industrial camera. The light source color, lamp bead combination, lamp bead arrangement, etc. can be selected based on the product / product packaging appearance color, flatness, and the environment. According to the position of the label image and laser code image on the product packaging, a suitable lighting position and angle can be selected, such as top-down lighting, back lighting, 45-degree side lighting, etc. Finally, the angle and type of the light source can be adjusted according to the camera recognition effect to complete the preparation of the visual inspection equipment.
[0029] In addition, for laser labeling, a composite transparent material imaging system can be used to solve the problem of unclear edge contour extraction during laser labeling; to improve measurement accuracy, a dual telecentric system can be used in conjunction with a large-target high-resolution industrial camera to achieve sub-millimeter imaging resolution. The dual telecentric system can solve many problems that affect measurement, such as imaging blur caused by objects not being in the same measurement plane and different magnifications. The field of view presented by a large-target high-resolution industrial camera is greater than 100mm×100mm, and the depth of field is 10mm, which meets the needs of actual production lines. To address the problem of blur caused by high-speed product movement, a high-power industrial LED light source can be customized to achieve high-speed synchronous acquisition between the light source and the industrial camera, thereby improving the robustness against motion blur.
[0030] In the dual-frequency RFID equipment method, a suitable dual-frequency RFID reader is selected according to the performance parameters of the RFID tags (dual-frequency tags, UHF tags, HF tags) contained in the product labeling image, and the product RFID tags are read based on the dual-frequency RFID reader for subsequent laser code detection.
[0031] In a possible implementation, step S200 detects the labeled image based on the multi-defect joint recognition model to obtain the degree of offset and the degree of wrinkle, including: S210, preprocessing the labeled image, the preprocessing includes: grayscale processing and filtering processing; S220, using grayscale difference statistics to calculate the contrast of the labeled image, extracting feature points from the labeled image according to the contrast of the labeled image, and querying the corresponding standard labeled image in the template library according to the feature points, the template library contains multiple standard labeled images and corresponding feature points; S230, bringing the standard labeled image and the labeled image into the trained multi-defect joint recognition model to obtain the degree of offset and the degree of wrinkle of the label.
[0032] Specifically, this solution uses a combination of machine vision and deep learning to detect label image defects, mainly to detect the degree of deviation and wrinkle of the label image. In terms of deviation detection, this solution proposes a feature point matching-based detection method, specifically extracting feature points from the label image, querying the closest standard label image through feature points, and predicting the degree of deviation and wrinkle through the label image and the standard label image.
[0033] Further, step S220, using grayscale difference statistics to calculate the contrast of the labeled image, extracting feature points from the labeled image according to the contrast of the labeled image, and searching the template library for the corresponding standard labeled image according to the feature points, including: S221, taking a pixel point in the labeled image as the center pixel point, constructing a 16-pixel Bresenham circle, calculating the average grayscale value of the 16 pixels on the Bresenham circle, calculating the difference between the grayscale value of the 16 pixels and the average grayscale value, calculating the pixel distribution probability of the difference, and calculating the Bresenham according to the pixel distribution probability. The contrast of the circular area; S222, calculate the corner point threshold of the central pixel according to the contrast, select 4 pixels with equal spacing from the Bresenham circle, calculate the gray value difference between the 4 pixels and the central pixel, when there are more than 3 gray value differences that are greater than or less than the corner point threshold, regard the central pixel as a feature point, traverse each pixel of the labeled image until all the feature points of the labeled image are obtained; S223, calculate the BRIEF descriptors of all feature points of the labeled image, and query the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
[0034] Specifically, step S220 is to extract feature points from the labeled image and query the standard labeled image through the feature points. In order to enhance the accuracy of feature point extraction of the labeled image, especially the accuracy of feature point extraction of the weak texture labeled image, this solution proposes contrast-based feature point extraction instead of traditional fixed threshold extraction. Take any pixel point in the labeled image as the center pixel point, and determine whether it is a feature point through the following steps: construct a 16-pixel Bresenham circle with the center pixel point as the center, and calculate the average grayscale value of the 16 pixels on the Bresenham circle:
[0035] ;
[0036] in, is the average gray value, For the The gray value of each pixel;
[0037] Calculate the difference between the grayscale value of 16 pixels and the average grayscale value:
[0038] ;
[0039] in, For the The difference between the pixel and the average gray value;
[0040] Calculate the pixel distribution probability of the difference:
[0041] ;
[0042] in, Is the difference The pixel distribution probability is Is the difference The number of times it appears in the 16 differences;
[0043] Calculate the contrast of the Bresenham circle area based on the pixel distribution probability:
[0044] ;
[0045] Among them, G is the contrast, and D is the difference set obtained after removing the duplicates of the 16 differences;
[0046] Calculate the corner threshold of the center pixel based on the contrast:
[0047] ;
[0048] Among them, K is the corner threshold, It is an adjustment parameter and takes a value between 0 and 1.
[0049] After determining the corner point threshold of the center pixel through the above steps, select 4 pixels with equal spacing from the Bresenham circle, such as Figure 2 As shown, take P1, P5, P9 and P13 to calculate the gray value difference between the four pixels and the central pixel. When there are more than three gray value differences that are greater than or less than the corner threshold, the central pixel is regarded as a feature point.
[0050] It should be noted that this solution adopts the method of collecting pixels at equal intervals, which can quickly determine the difference between the central pixel and the surrounding pixels, thereby determining whether the central pixel is a corner point. In addition to the above selection method, you can also choose P2, P6, P10 and P14, or P3, P7, P11 and P15, etc.
[0051] Traverse each pixel in the labeled image until all feature points of the labeled image are obtained, and calculate the BRIEF descriptors of all feature points. Use the feature point BRIEF descriptors to query the standard labeled image with the highest similarity from the template library.
[0052] The template library stores multiple standard label images. The standard label images here refer to label images without offset, rotation, or wrinkles, which can be used as templates for detection. Feature points and feature point BRIEF descriptors are extracted from each standard label image by the above method. By executing the feature point matching algorithm, the feature point BRIEF descriptor of the labeled image is matched to the standard label image with the closest feature point BRIEF descriptor, and the standard label image corresponding to the labeled image is obtained.
[0053] Furthermore, step S230 is to bring the standard labeling image and the labeling image into the trained multi-defect joint recognition model to obtain the offset degree and wrinkle degree of the labeling, wherein the multi-defect joint recognition model includes: an input module for receiving the labeling image and the standard labeling image; an offset prediction module for predicting the offset and rotation angle based on the labeling image and the standard labeling image; a wrinkle prediction module for correcting the labeling image according to the offset and rotation angle, and predicting the wrinkle area based on the corrected labeling image and the standard labeling image; an output module for comparing the offset, rotation angle and wrinkle area with the preset offset threshold, rotation angle threshold and wrinkle area threshold to obtain the labeling image detection result, including: labeling image detection passed and labeling image detection failed.
[0054] Specifically, a multi-defect joint recognition model can be constructed based on a combination of two convolutional neural networks. The multi-defect joint recognition model can include: an input module, an offset prediction module, a wrinkle prediction module and an output module.
[0055] Among them, the offset prediction module is used to estimate the spatial transformation between the labeled image and the standard labeled image (including the translation in the x and y directions and the rotation angle); the offset prediction module includes: an input layer for receiving the labeled image and the standard labeled image; a feature extraction layer, including two convolutional layer branches and a pooling layer, for extracting features from the labeled image and the standard labeled image; a feature pyramid network layer, for constructing a feature pyramid to perform multi-scale feature extraction from the output of the feature extraction layer; a feature matching layer, for matching the feature points of the labeled image and the standard labeled image through correlation operations; a transformation estimation layer, including two fully connected layers, for estimating the offset (x, y) and rotation angle (θ) between images based on the matched feature points; an output layer, including three neurons, for outputting the offset in the x direction, the offset in the y direction and the rotation angle;
[0056] Among them, the wrinkle prediction module is used to estimate the wrinkle degree (wrinkle area) of the labeled image compared with the standard labeled image; the wrinkle prediction module includes: a preprocessing layer, which is used to correct the labeled image according to the offset and rotation angle output by the offset prediction module; an input layer is used to access the corrected labeled image and the standard labeled image; a feature extraction layer, which includes two convolutional layer branches and a pooling layer, which are used to extract image features; an encoding layer, which is used to further extract features; a feature fusion layer, which is used to fuse the feature maps of the two images; a segmentation layer, which is used to generate a segmentation map of the wrinkle area; an upsampling layer, which is used to upsample the segmentation map to the original image size; an output layer, which is used to represent the position of the wrinkle area in the labeled image through a binary mask; and an area calculation layer, which is used to accumulate each pixel in the binary mask to obtain the total area of the wrinkle area.
[0057] The output module is used to compare the offset output by the offset prediction module, the rotation angle and the wrinkle area output by the wrinkle prediction module with the preset offset threshold, rotation angle threshold and wrinkle area threshold. When any of the offset, rotation angle and wrinkle area exceeds the threshold range, it is classified as failing the labeling image detection; when the offset, rotation angle and wrinkle area are all within the threshold range, it is classified as passing the labeling image detection.
[0058] It should be noted that when training the multi-defect joint recognition model, the weighted sum of the offset loss, rotation loss and area loss can be used as the loss function, and the Adam optimization algorithm can be used to update the weights. The model is trained using image pairs with labeled offset, rotation angle and wrinkle area, and the network performance is evaluated through an independent validation set.
[0059] It can be understood that this solution optimizes the input of the wrinkle prediction module by offsetting the output of the prediction module, reducing the introduction of errors, so that the wrinkle prediction module can learn wrinkle features more attentively.
[0060] In a possible implementation, step S300, based on a variable-length laser code recognition model, detects the laser code image to obtain a laser code recognition result, and compares the laser code recognition result with the product RFID tag to obtain a laser code detection result, including: S310, preprocessing the laser code image, the preprocessing includes grayscale processing and filtering processing; S320, inputting the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, the CRNN neural network includes a recognition layer for extracting a sequence from the laser code image, and a prediction layer for predicting the sequence; S330, comparing the laser code recognition result with the product RFID tag to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
[0061] Specifically, the task of the indefinite-length laser code recognition model is to recognize the laser code of uncertain length in the laser code image. The indefinite-length laser code recognition model may include: an input layer for accessing the laser code image; a feature extraction layer, including two convolutional layers and one pooling layer, for extracting the features of the laser code image; a deep feature extraction layer, also including two convolutional layers and one pooling layer, for further extracting the deep-level features of the laser code image; a coding feature extraction layer, including two convolutional layers, for extracting coding features; a sequence model layer, including two recurrent layers, for sequence modeling based on the first GRU unit, and for refining the coding features based on the second GRU unit; a sequence output layer, including a fully connected layer, for converting the output of the recurrent layer into a potential representation of the code; and a coding output layer, for outputting the laser code prediction result.
[0062] It should be noted that when training the variable-length laser code recognition model, the difference between the predicted laser code and the real laser code (product RFID tag) can be used as the loss function, and the Adam optimization algorithm is used to update the weights. The model is trained using image pairs with annotated laser codes, and the network performance is evaluated through an independent validation set.
[0063] Step S400 is to obtain a joint control instruction based on the above-mentioned labeling detection result and laser code detection result. When both the labeling detection result and the laser code detection result are qualified, the joint control instruction is to pass. When either the labeling detection result or the laser code detection result is unqualified, the joint control instruction is to reject. The rejection device is controlled by the joint control instruction to perform a rejection or pass operation.
[0064] See also Figure 3 As shown, Figure 3A structural schematic diagram of an intelligent labeling detection system provided for an embodiment of the present application. The system includes: a data acquisition unit, which is used to acquire a product labeling image acquired by a first visual inspection device, a product laser code image acquired by a second visual inspection device, and a product RFID label acquired by a dual-frequency RFID device; a labeling detection unit, which is used to detect the labeling image based on a multi-defect joint recognition model, obtain the degree of offset and the degree of wrinkle, and compare the degree of offset and the degree of wrinkle with the set degree of offset and the set degree of wrinkle to obtain a labeling detection result; a laser code detection unit, which is used to detect the laser code image based on an indefinite length laser code recognition model, obtain a laser code recognition result, and compare the laser code recognition result with the product RFID label to obtain a laser code detection result; an NG control unit, which is used to obtain a joint control instruction according to the labeling detection result and the laser code detection result, the joint control instruction includes a rejection instruction and a pass instruction, and the joint control instruction is used to control the rejection device to perform a rejection or pass operation.
[0065] In a possible implementation, the labeling detection unit specifically includes: a labeling image preprocessing unit, which is used to preprocess the labeling image, and the preprocessing includes: grayscale processing and filtering processing; a feature point extraction unit, which is used to calculate the contrast of the labeling image using a grayscale difference statistics method, extract feature points from the labeling image according to the contrast of the labeling image, and query the corresponding standard labeling image in a template library according to the feature points, and the template library contains multiple standard labeling images and corresponding feature points; a labeling defect analysis unit, which is used to bring the standard labeling image and the labeling image into a trained multi-defect joint recognition model to obtain the degree of offset and wrinkle of the label.
[0066] In a possible implementation, the feature point extraction unit specifically includes: a gray value statistics unit, which is used to construct a 16-pixel Bresenham circle with a pixel point in the labeled image as the central pixel point, calculate the average gray value of the 16 pixels on the Bresenham circle, calculate the difference between the gray value of the 16 pixels and the average gray value, calculate the pixel distribution probability of the difference, and calculate the contrast of the Bresenham circle area according to the pixel distribution probability; a feature point analysis unit, which is used to calculate the corner point threshold of the central pixel point according to the contrast, select 4 pixels with equal spacing from the Bresenham circle, calculate the gray value difference between the 4 pixels and the central pixel point, and when there are more than 3 gray value differences that are greater than or less than the corner point threshold, the central pixel point is regarded as a feature point, and each pixel point of the labeled image is traversed until all feature points of the labeled image are obtained; a standard labeled image extraction unit, which is used to calculate the BRIEF descriptors of all feature points of the labeled image, and query the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
[0067] In a possible implementation, the multi-defect joint recognition model in the labeling defect analysis unit includes: an input module for receiving a labeling image and a standard labeling image; an offset prediction module for predicting an offset and a rotation angle based on the labeling image and the standard labeling image; a wrinkle prediction module for correcting the labeling image according to the offset and the rotation angle, and predicting the wrinkle area based on the corrected labeling image and the standard labeling image; an output module for comparing the offset, the rotation angle and the wrinkle area with a preset offset threshold, a rotation angle threshold and a wrinkle area threshold to obtain a labeling image detection result, including: a labeling image detection passed and a labeling image detection failed.
[0068] In a possible implementation, the laser code detection unit specifically includes: a laser code image preprocessing unit, which is used to preprocess the laser code image, and the preprocessing includes grayscale processing and filtering processing; a laser code recognition unit, which is used to input the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, and the CRNN neural network includes a recognition layer for extracting a sequence from the laser code image and a prediction layer for predicting the sequence; a laser code detection unit, which is used to compare the laser code recognition result with the product RFID label to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
[0069] It should be noted that the intelligent labeling detection method and system provided by this solution can be found in Figure 4 As shown. Labeling detection and laser code detection are used to automatically remove NG products, which is suitable for existing labeling and laser code anti-counterfeiting product production lines. The trained multi-defect joint recognition model is used to identify common defects such as offset, rotation, and wrinkles in the labeling image to improve the accuracy of labeling image defect detection. The variable-length laser code recognition model is used to identify and compare the laser code in the laser code image to reduce the risk of misprinting the laser code. This solution can be widely applied to the production needs of various products / product packaging with anti-counterfeiting requirements.
[0070] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A smart labeling detection method, characterized in that: include: Acquire a product label image acquired by a first visual inspection device, a product laser code image acquired by a second visual inspection device, and a product RFID label acquired by a dual-frequency RFID device; Based on the multi-defect joint recognition model, the labeling image is detected to obtain the degree of deviation and the degree of wrinkle, and the degree of deviation and the degree of wrinkle are compared with the set degree of deviation and the set degree of wrinkle to obtain the labeling detection result; Detecting the laser code image based on a variable-length laser code recognition model to obtain a laser code recognition result, and comparing the laser code recognition result with the product RFID tag to obtain a laser code detection result; A joint control instruction is obtained according to the label detection result and the laser code detection result, wherein the joint control instruction includes a rejection instruction and a pass instruction, and the joint control instruction is used to control the rejection device to perform a rejection or pass operation; Among them, the labeled image is detected based on the multi-defect joint recognition model to obtain the degree of deviation and the degree of wrinkle, including: preprocessing the labeled image, the preprocessing includes: grayscale processing and filtering processing; using grayscale difference statistics to calculate the contrast of the labeled image, extracting feature points from the labeled image according to the contrast of the labeled image, and querying the corresponding standard labeled image in the template library according to the feature points, the template library contains multiple standard labeled images and corresponding feature points; bringing the standard labeled image and the labeled image into the trained multi-defect joint recognition model to obtain the degree of deviation and the degree of wrinkle of the label; The contrast of the labeled image is calculated by using a grayscale difference statistics method, feature points are extracted from the labeled image according to the contrast of the labeled image, and the corresponding standard labeled image is queried in a template library according to the feature points, including: taking a pixel point in the labeled image as a central pixel point, constructing a 16-pixel Bresenham circle, calculating the average grayscale value of the 16 pixels on the Bresenham circle, calculating the difference between the grayscale value of the 16 pixels and the average grayscale value, calculating the pixel distribution probability of the difference, and calculating the Bresenham circle according to the pixel distribution probability. The method comprises the following steps: calculating the corner point threshold of the central pixel according to the contrast, selecting four pixels with equal spacing from the Bresenham circle, calculating the gray value difference between the four pixels and the central pixel, and when three or more gray value differences are greater than or less than the corner point threshold, regarding the central pixel as a feature point, traversing each pixel of the labeled image until all feature points of the labeled image are obtained; calculating the BRIEF descriptors of all feature points of the labeled image, and querying the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
2. According to the intelligent labeling detection method of claim 1, the standard labeling image, the labeling image and the corresponding feature points are brought into the trained multi-defect joint recognition model to obtain the offset degree and wrinkle degree of the labeling, wherein: The multi-defect joint identification model includes: An input module, used for receiving a labeled image and a standard labeled image; An offset prediction module is used to predict the offset and rotation angle based on the labeled image and the standard labeled image; A wrinkle prediction module, used to correct the labeled image according to the offset and the rotation angle, and predict the wrinkle area according to the corrected labeled image and the standard labeled image; The output module is used to compare the offset, rotation angle and wrinkle area with the preset offset threshold, rotation angle threshold and wrinkle area threshold to obtain the labeling image detection result, including: the labeling image detection passes and the labeling image detection fails.
3. According to the intelligent labeling detection method of claim 1, the laser code image is detected based on the variable-length laser code recognition model to obtain a laser code recognition result, and the laser code recognition result is compared with the product RFID label to obtain a laser code detection result, including: Preprocessing the laser code image, wherein the preprocessing includes grayscale processing and filtering processing; Inputting the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, wherein the CRNN neural network includes a recognition layer for extracting a sequence from the laser code image and a prediction layer for predicting the sequence; The laser code recognition result is compared with the product RFID tag to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
4. An intelligent labeling detection system, characterized in that: include: A data acquisition unit, used to acquire the product labeling image acquired by the first visual inspection device, the product laser code image acquired by the second visual inspection device, and the product RFID label acquired by the dual-frequency RFID device; A labeling detection unit, used to detect the labeling image based on a multi-defect joint recognition model to obtain a degree of deviation and a degree of wrinkle, and compare the degree of deviation and the degree of wrinkle with a set degree of deviation and a set degree of wrinkle to obtain a labeling detection result; A laser code detection unit, used to detect the laser code image based on a variable-length laser code recognition model to obtain a laser code recognition result, and compare the laser code recognition result with a product RFID tag to obtain a laser code detection result; An NG control unit, used for obtaining a joint control instruction according to the label detection result and the laser code detection result, wherein the joint control instruction includes a rejection instruction and a pass instruction, and the joint control instruction is used for controlling the rejection device to perform a rejection or pass operation; The labeling detection unit specifically includes: a labeling image preprocessing unit, which is used to preprocess the labeling image, and the preprocessing includes: grayscale processing and filtering processing; a feature point extraction unit, which is used to calculate the contrast of the labeling image by using a grayscale difference statistical method, extract feature points from the labeling image according to the contrast of the labeling image, and query the corresponding standard labeling image in the template library according to the feature points, and the template library contains multiple standard labeling images and corresponding feature points; a labeling defect analysis unit, which is used to bring the standard labeling image and the labeling image into the trained multi-defect joint recognition model to obtain the offset degree and wrinkle degree of the label; The feature point extraction unit specifically includes: a gray value statistics unit, which is used to construct a 16-pixel Bresenham circle with a pixel point in the labeled image as the central pixel point, calculate the average gray value of the 16 pixels on the Bresenham circle, calculate the difference between the gray value of the 16 pixels and the average gray value, calculate the pixel distribution probability of the difference, and calculate the contrast of the Bresenham circle area according to the pixel distribution probability; a feature point analysis unit, which is used to calculate the corner point threshold of the central pixel point according to the contrast, select 4 pixels with equal spacing from the Bresenham circle, calculate the gray value difference between the 4 pixels and the central pixel point, and when there are more than 3 gray value differences that are greater than or less than the corner point threshold, the central pixel point is regarded as a feature point, and each pixel point of the labeled image is traversed until all feature points of the labeled image are obtained; a standard labeled image extraction unit, which is used to calculate the BRIEF descriptors of all feature points of the labeled image, and query the standard labeled image with the highest similarity from the template library through the feature point BRIEF descriptors.
5. The intelligent labeling detection system according to claim 4, characterized in that: The multi-defect joint recognition model in the labeling defect analysis unit includes: An input module, used for receiving a labeled image and a standard labeled image; An offset prediction module is used to predict the offset and rotation angle based on the labeled image and the standard labeled image; A wrinkle prediction module, used to correct the labeled image according to the offset and the rotation angle, and predict the wrinkle area according to the corrected labeled image and the standard labeled image; The output module is used to compare the offset, rotation angle and wrinkle area with the preset offset threshold, rotation angle threshold and wrinkle area threshold to obtain the labeling image detection result, including: the labeling image detection passes and the labeling image detection fails.
6. The intelligent labeling detection system according to claim 4, characterized in that: The laser code detection unit specifically comprises: A laser code image preprocessing unit, used for preprocessing the laser code image, wherein the preprocessing includes grayscale processing and filtering processing; A laser code recognition unit, used for inputting the preprocessed laser code image into a trained CRNN neural network to obtain a laser code recognition result, wherein the CRNN neural network includes a recognition layer for extracting a sequence from the laser code image and a prediction layer for predicting the sequence; The laser code detection unit is used to compare the laser code recognition result with the product RFID tag to obtain a laser code detection result, including: laser code image detection passed and laser code image detection failed.
