Classification detection method and system for detecting two-dimensional code label defects based on lightweight convolution

Abstract

The embodiment of the invention discloses a classification detection method and system for detecting two-dimensional code label defects based on lightweight convolution. The efficiency and accuracy of two-dimensional code label printing defect detection can be improved. The method comprises the following steps: S1, acquiring a two-dimensional code label in a label printing area by using a high-resolution camera to obtain a camera rough image; s2, in combination with HSV color recognition and threshold segmentation, taking the two-dimensional code label as an ROI region to perform feature extraction on the rough image of the camera to obtain a fine contour image of the two-dimensional code label; s3, a lightweight convolution module LRB is used for extracting defect features of the two-dimensional code label in the fine contour image, the lightweight convolution module LRB is constructed based on a ghost module GM and a lightweight residual block, and the lightweight residual block is designed to be of an inverted residual block structure composed of linear bottleneck structures; using a ghost module GM to replace the inverted residual structure and a depth separable convolutional layer in the two-dimensional convolutional layer; and S4, using a Softmax classification layer to predict the defect features of the two-dimensional code labels and outputting a defect classification result.

Description

Technical Field

[0001] The present invention relates to the field of image processing and machine technology, and in particular to a classification detection method and a system thereof for detecting defects of two-dimensional code labels based on lightweight convolution. Background Art

[0002] Currently, researchers are mainly concerned about how to improve the printing quality of QR codes to ensure that they can be accurately recognized in different environments. Some of the latest research focuses on the optimization of printing parameters, such as ink type, printing resolution, and printing speed.

[0003] Some studies have focused on improving the error tolerance of QR codes to cope with poor or damaged printing. These methods mainly focus on improving QR code design and error correction coding.

[0004] In industrial applications, there are also some studies dedicated to improving the accuracy of QR code recognition on high-speed production lines to meet the needs of real-time detection and tracking.

[0005] In general, the research field of QR code label printing and detection and recognition technology continues to develop and innovate, but there are few studies on quality control based on QR label printing. However, the completeness and clarity of QR code label printing information are of great significance to product quality control. However, in the process of printing labels, due to the small number of defective samples, small defects and low contrast, a large number of misjudgments are prone to occur. In the mature modern automated production process, a large number of printed labels need to be subjected to real-time defect recognition tasks to ensure product label quality control. Summary of the invention

[0006] In order to improve the efficiency and accuracy of two-dimensional code label printing defect detection, an embodiment of the present invention provides a classification detection method and a system for detecting two-dimensional code printing defects based on lightweight convolution.

[0007] In a first aspect, an embodiment of the present invention provides a classification detection method for detecting defects in a two-dimensional code label based on lightweight convolution, characterized in that it includes the following steps:

[0008] S1: Use a high-resolution camera to capture the QR code label in the label printing area to obtain a rough camera image;

[0009] S2: Combining HSV color recognition and threshold segmentation, the QR code label is used as the ROI area to extract features from the rough camera image to obtain a fine contour image of the QR code label;

[0010] S3: Use lightweight convolution module LRB to extract defect features of QR code labels in fine contour images, where lightweight convolution module LRB is constructed based on ghost module GM and lightweight residual block. The lightweight residual block is designed as an inverted residual block structure composed of linear bottleneck structure, and ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer;

[0011] S4: Use the Softmax classification layer to predict the defect features of the QR code label and output the defect classification results.

[0012] Optionally, in a possible implementation of the first aspect, the above S3: using a lightweight convolution module LRB to extract defect features of the two-dimensional code label in the fine contour image includes:

[0013] S3.1: First, a linear operation is performed on the defect features of the QR code label in the GM fine contour image through a ghost module with an N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3;

[0014] S3.2: Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions;

[0015] S3.3: Finally, the fine contour image is processed by the ghost module GM with a 1×1 convolution kernel to reduce the dimension and obtain the defect features of the QR code label in the fine contour image.

[0016] Optionally, in a possible implementation of the first aspect, the above S2: combining HSV color recognition and threshold segmentation, taking the QR code label as the ROI area to extract features from the camera rough image to obtain a fine contour image includes:

[0017] S2.1: Convert the camera rough image from RGB color space to HSV color space to obtain an HSV image;

[0018] S2.2: Perform color threshold segmentation on the HSV image;

[0019] S2.3: Perform ROI extraction processing on the segmented image to obtain a ROI area image;

[0020] S2.4: Perform fine contour extraction on the ROI area image to obtain a fine contour image of the QR code label.

[0021] Optionally, in a possible implementation of the first aspect, the linear operation in S3.1 above adopts the following formula:

[0022]

[0023] In the formula, zi represents the i-th internal feature map generated by convolution, W i,j To generate the unit linear mapping operation of the intrinsic feature map, z ij Represents the ghost feature map obtained after j linear operations.

[0024] Optionally, in a possible implementation of the first aspect, the value of N is 3, and the linear operation is used in each individual channel of the ghost module of the 3×3 convolution kernel to generate linear processing of the feature map, thereby realizing convolution mapping.

[0025] In a second aspect, an embodiment of the present invention provides a classification detection system for detecting defects in a two-dimensional code label based on lightweight convolution, which is characterized by comprising:

[0026] fine contour extractors, feature extractors, and predictors;

[0027] The fine contour extractor is used to: combine HSV color recognition and threshold segmentation, take the QR code label as the ROI area to extract features from the camera rough image to obtain the fine contour image of the QR code label. The camera rough image includes the QR code label in the label printing area captured by a high-resolution camera;

[0028] The feature extractor is used to: extract defect features of the QR code label in the fine contour image using a lightweight convolution module LRB, wherein the lightweight convolution module LRB is constructed based on a ghost module GM and a lightweight residual block, the lightweight residual block is designed as an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer;

[0029] The predictor is used to: use the Softmax classification layer to predict defect features and output defect classification results.

[0030] Optionally, in a possible implementation of the second aspect, the feature extractor is specifically used to perform the following steps when extracting defect features of the QR code label:

[0031] Firstly, the defect features of the QR code label in the GM fine contour image are linearly operated through the ghost module with N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3;

[0032] Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions;

[0033] Finally, the fine contour image is subjected to dimensionality reduction processing by the ghost module GM with a 1×1 convolution kernel to obtain the defect features of the QR code label in the fine contour image.

[0034] Optionally, in a possible implementation of the second aspect, the fine contour extractor is specifically used to perform the following steps when extracting the fine contour image of the QR code label:

[0035] Convert the camera rough image from RGB color space to HSV color space to obtain HSV image;

[0036] Perform color threshold segmentation on HSV images;

[0037] Perform ROI extraction processing on the segmented image to obtain a ROI area image;

[0038] The ROI area image is refined and contour extracted to obtain a fine contour image of the QR code label.

[0039] In a third aspect, another aspect of an embodiment of the present invention provides a computer device, including:

[0040] at least one processor; and

[0041] a memory communicatively coupled to the at least one processor;

[0042] Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described in any one of the first aspect and its implementation methods above.

[0043] In the fourth aspect, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method as described in any one of the first aspect and its implementation methods is implemented.

[0044] In the fifth aspect, another aspect of an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the method as described in any one of the first aspect and its implementation methods.

[0045] The embodiments of the present invention adopt the above technical solution to achieve the following technical effects:

[0046] The present invention is based on a classification detection method for detecting defects in two-dimensional code labels using lightweight convolution. First, the two-dimensional code label is used as the ROI area to extract features from the rough camera image and obtain a fine contour image of the two-dimensional code label. Secondly, the defect features of the two-dimensional code label in the fine contour image are extracted using the lightweight convolution module LRB. Finally, the defect features of the two-dimensional code label are predicted and output using the Softmax classification layer to output the defect classification result, thereby realizing a defect recognition method based on the combination of traditional image processing and deep learning. Furthermore, since the lightweight convolution module LRB is constructed based on the ghost module GM and the lightweight residual block, the lightweight residual block is designed to have an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer, the method can quickly calculate and process defect features, improve the detection efficiency of two-dimensional code defect classification detection, and accurately extract defect features through the newly designed lightweight convolution module combined with HSV color recognition and threshold segmentation technology, thereby improving the accuracy of two-dimensional code defect classification detection. . BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0048] Figure 1 A schematic flow chart of a classification detection method for detecting defects in a two-dimensional code label based on lightweight convolution in an embodiment of the present invention;

[0049] Figure 2 It is the complete implementation process of the QR printing defect network in the present invention in industry;

[0050] Figure 3 This is the framework diagram of the QR label printing defect network model;

[0051] Figure 4 Schematic diagram of the detailed ROI extraction process;

[0052] Figure 5 Comparison diagram of different ROI extraction methods;

[0053] Figure 6 A schematic diagram comparing the three methods of deep separable convolution, point convolution, and ghost module with different basic blocks;

[0054] Figure 7 It is a structural schematic diagram of a classification detection system for detecting defects of two-dimensional code labels based on lightweight convolution in an embodiment of the present invention;

[0055] Figure 8 FIG. 4 is a schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] In the description of the present invention, it should be understood that the numerical labels before the steps do not identify the order of executing the steps, but are only used to facilitate the description of the present invention and distinguish each step, and therefore cannot be understood as a limitation of the present invention.

[0059] First, the following terms are explained:

[0060] GM refers to a lightweight convolution operation, and specifically, a ghost module Ghost Module performs the above lightweight convolution operation;

[0061] HSV is a relational representation of points in the RGB color space. They attempt to describe perceptual color relationships more accurately than RGB, while still remaining computationally simple. HSV Represents hue, saturation and value;

[0062] QR, Quick Response is a two-dimensional code. Correspondingly, QR label refers to the printing and detection and recognition technology of two-dimensional code labels, which plays an important role in the industrial and commercial fields;

[0063] ROI, which stands for region of interest, is a method introduced into image processing by simulating the visual attention mechanism of the human eye, which promotes the improvement of image computing efficiency.

[0064] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided by the embodiments of the present invention, the relevant technologies are described below:

[0065] Embodiment 1

[0066] The classification detection method for detecting defects of two-dimensional code labels based on lightweight convolution in the embodiment of the present invention is described in detail. Figure 1 As shown, including:

[0067] S1: Use a high-resolution camera to capture the QR code label in the label printing area and obtain a rough camera image.

[0068] Use a high-resolution camera (such as an industrial camera) to capture images of the QR code label to ensure the clarity and integrity of the image. During the acquisition process, uniform illumination must be ensured to avoid image quality degradation due to uneven illumination. The camera resolution is 2448×2408. The acquisition device is shown in Figure 2 The acquired image is shown in .

[0069] Furthermore, the camera rough image may be obtained by performing image preprocessing on the captured image. Specifically, the specific operations of image preprocessing include:

[0070] 1. Grayscale: refers to the formula for converting a color image into a grayscale image. The weighted average method is usually used to merge the three channels of the RGB color image into a grayscale value. The commonly used formula is as follows: Y = 0.2989R + 0.5870G + 0.1140B, where Y is the grayscale value, and R, G, and B are the pixel values ​​of the red, green, and blue channels respectively;

[0071] 2. Filtering: Specifically, it can be Gaussian filtering. Gaussian filtering is a commonly used image smoothing technology used to remove noise in images. Its core idea is to use a Gaussian function to weight neighborhood pixels to smooth the image. The formula of the Gaussian filter is: Among them, x and y represent the distance between the current pixel and the filter center in the horizontal and vertical directions respectively;

[0072] When performing convolution, the convolution of image I and Gaussian kernel G can be expressed as:

[0073]

[0074] Where σ is the standard deviation of the Gaussian distribution, k is the size of the kernel, and I fitered(x, y) is the pixel value after filtering, I(x+i, y+j) represents the pixel value of the original image I at position (x+i, y+j), i and j are the offsets of the convolution kernel, indicating the position of the currently processed pixel relative to the center pixel (x, y); G(i, j) is the value of the Gaussian kernel at position (i, j), indicating the weight of that position.

[0075] 3. Binarization: The purpose of binarization is to convert a grayscale image into an image containing only black and white colors. A common threshold method is to use a global threshold segmentation method. Set a threshold T, and use the following formula for binarization:

[0076]

[0077] Among them, B(x,y) is the pixel value in a binary image, with a value of 1 representing white (or foreground) and a value of 0 representing black (or background), while I(x,y) is the pixel value in a grayscale image.

[0078] S2: Combining HSV color recognition and threshold segmentation, the QR code label is used as the ROI area to extract features from the camera's rough image to obtain a fine contour image of the QR code label.

[0079] ROI detection is a method introduced into image processing by simulating the visual attention mechanism of the human eye. It promotes the computational efficiency of images. HSV is a color space constructed based on the intuitive characteristics of color. It is often used for color region segmentation. The HSV color space describes the color through three components: color H, saturation S, and brightness V. Saturation S is a proportional value ranging from 0 to 1. When S is 0, it is colorless. In this paper, we combine grayscale image threshold segmentation with HSV, extract the target rectangular area ROI, and perform shape and size screening in parallel to achieve accurate selection of the ROI area of ​​the target label and obtain the final fine image. The detailed extraction process is as follows Figure 4 shown.

[0080] Specifically, in a possible implementation of the embodiment of the present invention, the above S2: combining HSV color recognition and threshold segmentation, taking the QR code label as the ROI area to extract features from the camera rough image to obtain a fine contour image includes:

[0081] S2.1: Convert the camera rough image from RGB color space to HSV color space to obtain an HSV image;

[0082] S2.2: Perform color threshold segmentation on the HSV image;

[0083] S2.3: Perform ROI extraction processing on the segmented image to obtain a ROI area image;

[0084] S2.4: Perform fine contour extraction on the ROI area image to obtain a fine contour image of the QR code label.

[0085] S3: Use lightweight convolution module LRB to extract defect features of QR code labels in fine contour images, where the lightweight convolution module LRB is constructed based on the ghost module GM and the lightweight residual block. The lightweight residual block is designed as an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer.

[0086] Different from the structure and operation mode adopted by MobileNetV2, the LRB proposed in this invention adopts GM to replace the inverse residual bottleneck and the depth-separable convolution layer in Conv2d. First, GM performs linear operation on the input feature dimension information to ensure the dimension information and make the dimension richer. Then, BN and ReLU6 linear activation functions are used to retain the feature information. Then, the feature information is input into GM and the 1×1 convolution kernel is used to reduce the dimension of the features increased by the previous module. The detailed model structure can be found in Figure 6 As shown in part b.

[0087] Specifically, optionally, in one implementation of an embodiment of the present invention, the above S3: the specific steps of using the lightweight convolution module LRB to extract the defect features of the QR code label in the fine contour image include: S3.1: First, a linear operation is performed on the defect features of the QR code label in the GM fine contour image through a ghost module with an N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3; S3.2: Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions; S3.3: Finally, the fine contour image is subjected to dimensionality reduction processing through the ghost module GM with a 1×1 convolution kernel to obtain the defect features of the QR code label in the fine contour image.

[0088] Further optionally, the linear operation in S3.1 above adopts the following formula:

[0089]

[0090] In the formula, z i represents the i-th internal feature map generated by convolution, W i,j To generate the unit linear mapping operation of the intrinsic feature map, z ij Represents the ghost feature map obtained after j linear operations.

[0091] Specifically, optionally, the value of N is 3, and the linear operation is used in each individual channel of the ghost module of the 3×3 convolution kernel to generate linear processing of the feature map, thereby realizing convolution mapping.

[0092] S4: Use the Softmax classification layer to predict the defect features of the QR code label and output the defect classification results.

[0093] The detailed process of outputting defect category labels through the Softmax layer (i.e., Softmax classifier) ​​is shown in Figure 3 As shown in the prediction section.

[0094] In an embodiment of the present invention, the present invention is based on a classification detection method for detecting defects in a two-dimensional code label by lightweight convolution. First, in combination with HSV color recognition and threshold segmentation, the two-dimensional code label is used as the ROI area to extract features of the rough camera image to obtain a fine contour image of the two-dimensional code label. Secondly, the defect features of the two-dimensional code label in the fine contour image are extracted using a lightweight convolution module LRB. Finally, the defect features of the two-dimensional code label are predicted and output as defect classification results using a Softmax classification layer, thereby realizing a defect recognition method based on the combination of traditional image processing and deep learning. Furthermore, since the lightweight convolution module LRB is constructed based on a ghost module GM and a lightweight residual block, the lightweight residual block is designed to be an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer, the method can quickly calculate and process defect features, thereby improving the detection efficiency of two-dimensional code defect classification detection, and through the newly designed lightweight convolution module combined with HSV color recognition and threshold segmentation technology, the defect features can be accurately extracted, thereby improving the accuracy of two-dimensional code defect classification detection.

[0095] Finally, combine Figure 2-6 The QR label printing defect network model, ROI detailed extraction process, comparison of different ROI extraction methods, and comparison of different basic blocks involved in the first embodiment of the present invention are described in detail as follows:

[0096] like Figure 3 As shown, the network model of QR label printing defects in the embodiment of the present invention is explained, which mainly includes three parts: fine contour, feature extraction and prediction. The fine contour is mainly composed of a refined contour extractor to complete the refined contour extraction of the collected coarse image. The output image is first input into different lightweight residual blocks (LRB) for feature extraction, and finally label prediction is performed to achieve real-time classification of QR label printing defects.

[0097] like Figure 4 As shown in the figure, the detailed ROI extraction process is described:

[0098] For the input coarse image f(x, y) | RGB, the three channel values ​​can be obtained as R, G, and Y, with a value range of 0 to 255. Then, each channel value is normalized using formula (1) to obtain the chromaticity coordinate value R′0, G′0, Y′0. Then, the chromaticity extreme value ΔY of different channels is calculated using formulas (2) and (3). The RGB color space parameters can be converted and calculated to obtain the HSV value. The detailed conversion calculation process is shown in formula (4). The specific calculation process is as follows:

[0099]

[0100] Y max =max(R′0,G′0,B′0)

[0101] Y min =min(R′0,G′0,B′0) (2)

[0102] ΔY=Y max -Y min (3)

[0103] V=Y max

[0104]

[0105] Where, H∈[0°, 360°]; S∈[0, 1]; V∈[0, 1]. Formula (4) is used to calculate the values ​​of H, S, and V respectively and output the final f(x, y)| HSV Figure. The processed HSV image is then subjected to color threshold processing, and the threshold is set to [(0,0,255), (255,0,255)]. Finally, the binary image after threshold segmentation is subjected to morphological processing and rectangular area screening. When the contour length is greater than 800, it meets the target frame selection area. Finally, the obtained rectangular frame image is adjusted to a uniform size to complete the acquisition of the ROI area. In order to compare the differences between different ROI extraction methods and illustrate the effectiveness of our method, we have made an intuitive comparison of the results obtained by different extraction methods. Figure 5 ,The method based on the maximum rectangle in the figure is easy to select the non-target area, and the template matching cannot completely locate the target and some parts are missing. Therefore, through comparison, it can be seen that the method we proposed is the most complete and accurate in extracting the target area.

[0106] Next, the lightweight data network construction, model training and prediction, and experimental verification in the present invention are described in detail:

[0107] 1. Lightweight volume network construction

[0108] 1) Lightweight convolution operation GM

[0109] Compared with the cross-channel processing features of existing convolutions and the use of deep convolutions to process deep spatial information, cheap operations use cheap linear operations based on the use of ordinary convolutions to generate inherent feature maps, which can better exchange different channel feature information. While maintaining the inherent features, it improves the diversity of different channel features after generation, and can effectively reduce the amount of calculation compared to ordinary convolutions. Therefore, this efficient convolution method is introduced into the feature extraction network. Some standard convolution layers in similar deep separable convolutions still have residual 1×1 convolution layers, which will take up a lot of memory and FLOPs (Floating-point Operations). At the same time, the standard CNN calculation process will have redundancy in the intermediate feature maps. When compared with the GM method, the convolution operation used to generate the same feature map is only half of the standard convolution. For the input image X∈R c×h×w (where h and w represent the height and width of the image) After convolution processing, m feature maps can be generated, feature X∈R c×h×w :

[0110] Y=X*f0 (5)

[0111] where f0∈R c×k×k×m Represents the output matrix with a convolution kernel size of k×k and a number of channels of c. After the standard convolution mapping, n internal feature maps can be generated, and it is known that m≤n. Then, after the cheap operation of identity transformation is performed on each feature map that passes through the convolution, m×s Ghost feature maps can be obtained after each mapping in Y. This ensures that the output of the standard convolution layer is consistent with the number of features processed by GM. The cheap operation in this process is expressed as:

[0112]

[0113] In the formula, z i represents the i-th internal feature map generated by convolution, W i,j To generate the unit linear mapping operation of the intrinsic feature map, z ij It represents the ghost feature map obtained after j linear operations. According to Formula 6, we can get Y′=[y 11 ,y 12 ,…,y ms ], a total of m×s output feature maps, where our GM uses a 3×3 convolution kernel. Because each individual channel uses a cheap operation (linear transformation) to generate a feature map linear processing for convolution mapping, the computational cost is much lower than the standard convolution method.

[0114] 2) Lightweight convolution module LRB

[0115] In the structural design of the network, in order to solve the problem that the difference between defects in QR printing is small, resulting in low contrast, making it difficult for the network to accurately identify. To this end, we used GM and residual blocks to construct different LRBs during the design process, and increased the depth of the network to enhance the ability to extract low-contrast features. LBR is composed of basic blocks of different GM operations and connection methods, and the basic blocks are combined to construct the final feature extraction network. The construction idea is similar to MobileNetV2. By optimizing the traditional residual block, it is changed to an inverted residual block structure with a linear bottleneck structure, such as Figure 5 As shown in (a), the structure is exactly the opposite of the 1×1 residual of ResNet. When the input feature map X is of size h×w×c, a 1×1 convolution kernel is first used to increase the dimension of the input feature to obtain the expanded feature map X. expand , the size is h×w×(c×t), where t∈(1,6) is the expansion coefficient; then use the 3×3 convolution kernel for feature processing to obtain the deep convolution feature map X depth , the size is s is the step size; finally, the feature map X is obtained by dimensionality reduction through a 1×1 convolution kernel down , the size is c out is the number of output feature channels. This forms a bottleneck structure with small ends and a large middle. Different from the structure and operation mode adopted by MobileNetV2, the proposed LRB uses GM to replace the inverse residual bottleneck and the depth-separable convolution layer in Conv2d. First, GM performs linear operations on the input feature dimension information to ensure the dimensional information and make the dimension richer. Then, BN and ReLU6 linear activation functions are used to retain the feature information. Then, the GM is used to reduce the dimension of the features increased in the previous module using a 1×1 convolution kernel. For the detailed model structure, see Figure 5 As shown in part b.

[0116] At the same time, in order to further enhance the classification model's acquisition of depth and dimensional information for feature extraction, the GM operation is stacked again on the basis of LBR to restore the dimension to the input size. Then the input dimensional features and output dimensions are shortcut-connected to further expand the output feature information, so that the model can better extract deep spatial information and improve the model's information fusion ability. Among them, the shortcut connection is adopted when the stride is 1. If the stride is s = 1 and the number of input and output channels is the same, the feature map after the shortcut connection is Y1 = X + X down , otherwise the reduced dimension feature map Y0=X is output.

[0117] 3) Model structure

[0118] The framework structure of the QR label printing defect network model is shown in Figure 3, mainly includes three parts: fine contour, feature extraction and model prediction. The fine contour is mainly composed of a refined contour extractor, which completes the fine contour extraction of the collected coarse image. The output image is input into the feature extractor and predictor composed of different LRBs (Lightweight Residual Blocks) for model training and prediction, realizing real-time classification of QR label printing defects. The LRB adopts a lightweight residual block structure, which improves the model's reasoning speed and feature representation ability by reducing the number of parameters and calculations. The lightweight classification network part of printing defect detection consists of multiple layers of LRBs and inverted residual blocks. By extracting features layer by layer, it finally realizes the classification detection of QR code defects.

[0119] 2. Model training and prediction

[0120] 1) Dataset construction: Build a dataset containing various types of QR code defects for model training and verification.

[0121] There are obvious "intra-class" changes between normal images and abnormal defective images of QR codes. At the same time, the samples of defective images have high visual similarity. There is an imbalance in the number of different defect categories and samples. In order to better eliminate the influence of these factors and realize the sorting of defective images, the data set is divided into Good and NG categories. Both categories of data are not labeled and are used as anomaly detection binary classification data sets for training and testing. The Good class represents a total of 710 pieces of normal printed label data, and the NG class represents printed images that are abnormal from normal images. The abnormal label data mainly includes various QR label printing defects such as dirt, peripheral dirty spots, blur, and offset. There are a total of 679 defective images.

[0122] 2) Model training: Use the optimization algorithm Adam to train the model and adjust the network parameters to achieve optimal performance on the training data set.

[0123] 3) Model prediction: Apply the trained model to new images for real-time defect detection and classification. Output defect category labels through the Softmax layer (Softmax classifier). See the detailed process for details. Figure 3 .

[0124] 3. Experimental Verification

[0125] 1) Experimental setup: The experiment was conducted on the constructed QR code defect dataset to evaluate the accuracy and recognition speed of the model. The proposed defect detection network is based on the Pytorch framework and implemented on NVIDIA GeForce RTX 4070Ti. During the experiment, in order to reduce the impact of the imbalance of QR code data categories and the differences between dataset images on the results, and to more objectively evaluate the model results, a 5-fold cross-validation strategy was adopted in both training and verification. The number of epochs for training was 200, and the momentum factor was 0.9, patience = 10, and ReduceLROnPlateau learning rate scheduler was used. The training, test and validation sets are divided into the ratio of 7:2:1 on the public dataset for comparison. The other parameters of the experiment are set the same, where the initial learning rate of training is set to 0.001 and batch=16. For multi-classification tasks, we evaluate the classification performance of each category separately and use 5 evaluation indicators to evaluate each category: The main evaluation indicators are accuracy (ACC), area under the receiver operating characteristic (ROC) curve (AUC), precision (Pre), recall (Recall) and F1 score (F1):

[0126]

[0127] For the i-th category, TN i 、FN i TP i FP i , represent the number of true negatives, false negatives, true positives and false positives respectively.

[0128] To verify the effectiveness of the proposed method, we selected 7 different models for experimental comparison. Considering the lightweight and industrial application of the model, we mainly selected several different models for comparison: ResNet50, EfficientNetv2, GhostNetv2, MobileNetv3 and ConvNext, as well as two ViT-related models: Swin-T and MobileVit. The quantitative results of the experiment on the QR printing defect dataset are compared in Table 1. Our model is higher than the comparison method in Acc, Pre, Recall and F1. Among them, Acc, Recall and F1 of the lightweight MobileVit model are 3.10%, 3.17% and 3.18% higher than those of the lightweight MobileVit model, Pre and Recall are 2.60% and 2.37% higher than those of the ResNet50 network, and Acc is 4.18%, 3.17% and 3.22% higher than those of the lightweight network GhostNetv2, GhostNetv2 and MobileVit models. Due to the imbalance of model data samples, the recognition results of Swin-T and ConvNext are not ideal, which also shows the effectiveness of our method in feature extraction.

[0129] Table 1: Quantitative comparison of different model indicators on QR (%)

[0130]

[0131] Table 2: Complexity analysis of the model

[0132]

[0133] As shown in Table 2, FLOPs is the number of floating-point operations per second. It can be seen that the accuracy of our proposed method is higher than that of other comparison methods. It not only achieves the highest frame rate on GPU devices, but also has a lower number of floating-point operations per second and the number of parameters, which can illustrate the lightweight performance of our method.

[0134] 2) Experimental results: The experimental results show that the network model achieved an accuracy of 96.72% and an efficient recognition speed of 186 frames per second on the QR code defect dataset. All indicators are better than the existing comparison models and have good generalization performance.

[0135] Embodiment 2

[0136] Figure 7 FIG. 1 is a structural diagram of a classification detection system for detecting defects in a two-dimensional code label based on lightweight convolution in an embodiment of the present invention. Figure 7 As shown, the classification detection system 70 includes: a fine contour extractor 701, a feature extractor 702 and a predictor 703;

[0137] The fine contour extractor 701 is used to: combine HSV color recognition and threshold segmentation, take the QR code label as the ROI area to perform feature extraction on the camera rough image to obtain a fine contour image of the QR code label, where the camera rough image includes the QR code label obtained by using a high-resolution camera to capture the label printing area;

[0138] The feature extractor 702 is used to: extract defect features of the two-dimensional code label in the fine contour image using a lightweight convolution module LRB, wherein the lightweight convolution module LRB is constructed based on a ghost module GM and a lightweight residual block, the lightweight residual block is designed to be an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer;

[0139] The predictor 703 is used to use the Softmax classification layer to predict the defect features and output the defect classification results.

[0140] Optionally, in an implementation of the embodiment of the present invention, the feature extractor 702 is specifically used to perform the following steps when extracting the defect features of the two-dimensional code label:

[0141] Firstly, the defect features of the QR code label in the GM fine contour image are linearly operated through the ghost module with N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3;

[0142] Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions;

[0143] Finally, the fine contour image is subjected to dimensionality reduction processing by the ghost module GM with a 1×1 convolution kernel to obtain the defect features of the QR code label in the fine contour image.

[0144] Optionally, in an implementation manner of the embodiment of the present invention, the fine contour extractor 701 is specifically used to perform the following steps when extracting the fine contour image of the two-dimensional code label:

[0145] Convert the camera rough image from RGB color space to HSV color space to obtain HSV image;

[0146] Perform color threshold segmentation on HSV images;

[0147] Perform ROI extraction processing on the segmented image to obtain a ROI area image;

[0148] The ROI area image is refined and contour extracted to obtain a fine contour image of the QR code label.

[0149] The fine contour extractor 701, feature extractor 702 and predictor 703 in the embodiment of the present invention can also be used to execute the corresponding operation steps in the above-mentioned embodiment 1.

[0150] Embodiment 3

[0151] Figure 8 FIG. 4 is a schematic diagram of the structure of a computer device in an embodiment of the present invention.

[0152] In some embodiments, the computer device 80 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 80 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers). Figure 8 As shown, the computer device 80 includes but is not limited to: a memory 801, a processor 802, and a network interface 803 that can communicate with each other through a system bus. Among them:

[0153] The memory 801 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 801 can be an internal storage module of the computer device 80, such as a hard disk or memory of the computer device 80. In other embodiments, the memory 801 can also be an external storage device of the computer device 80, such as a plug-in hard disk equipped on the computer device 80, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 801 can also include both the internal storage module of the computer device 80 and its external storage device. In this embodiment, the memory 801 is generally used to store the operating system and various application software installed on the computer device 80, such as the program code of the classification detection method for detecting defects of two-dimensional code labels based on lightweight convolution. In addition, the memory 801 can also be used to temporarily store various types of data that have been output or are to be output.

[0154] In some embodiments, the processor 802 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 802 is generally used to control the overall operation of the computer device 80, such as performing control and processing related to data interaction or communication with the computer device 80. In this embodiment, the processor 802 is used to run the program code stored in the memory 801 or process data.

[0155] The network interface 803 may include a wireless network interface or a wired network interface, and the network interface 803 is generally used to establish a communication link between the computer device 80 and other computer devices. For example, the network interface 803 is used to connect the computer device 80 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 80 and the external terminal. The network can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0156] It should be pointed out that Figure 8 Only a computer device having components 801 - 803 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0157] In this embodiment, the classification detection method for detecting two-dimensional code label defects based on lightweight convolution stored in the memory 801 can also be divided into one or more program modules and executed by one or more processors (such as processor 802) to complete an embodiment of the present invention.

[0158] Embodiment 4

[0159] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the classification detection method for detecting two-dimensional code label defects based on lightweight convolution in the embodiment are implemented.

[0160] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store the operating system and various application software installed on the computer device, such as the program code of the classification detection method for detecting defects of two-dimensional code labels based on lightweight convolution in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0161] Embodiment 5

[0162] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0163] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, and optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0164] It should be noted that the above are only preferred embodiments of the present invention, and the patent protection scope of the present invention is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A classification detection method for detecting defects in two-dimensional code labels based on lightweight convolution, characterized in that: The following steps are involved: S1: Use a high-resolution camera to capture the QR code label in the label printing area to obtain a rough camera image; S2: Combining HSV color recognition and threshold segmentation, the QR code label is used as the ROI area to extract features from the rough camera image to obtain a fine contour image of the QR code label; S3: Use lightweight convolution module LRB to extract defect features of QR code labels in fine contour images, where lightweight convolution module LRB is constructed based on ghost module GM and lightweight residual block. The lightweight residual block is designed as an inverted residual block structure composed of linear bottleneck structure, and ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer; S4: Use the Softmax classification layer to predict the defect features of the QR code label and output the defect classification results.

2. The method according to claim 1, characterized in that The specific steps of S3: using the lightweight convolution module LRB to extract defect features of the two-dimensional code label in the fine contour image include: S3.1: First, a linear operation is performed on the defect features of the QR code label in the GM fine contour image through a ghost module with an N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3; S3.2: Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions; S3.3: Finally, the fine contour image is processed by the ghost module GM with a 1×1 convolution kernel to reduce the dimension and obtain the defect features of the QR code label in the fine contour image.

3. The method according to claim 1 or 2, characterized in that: The specific steps of S2: combining HSV color recognition and threshold segmentation, taking the QR code label as the ROI area to extract features from the camera rough image to obtain a fine contour image include: S2.1: Convert the camera rough image from RGB color space to HSV color space to obtain an HSV image; S2.2: Perform color threshold segmentation on the HSV image; S2.3: Perform ROI extraction processing on the segmented image to obtain a ROI area image; S2.4: Perform fine contour extraction on the ROI area image to obtain a fine contour image of the QR code label.

4. The method according to claim 2, characterized in that: The linear operation in S3.1 is calculated using the following formula: In the formula, z i represents the i-th internal feature map generated by convolution, W i,j To generate the unit linear mapping operation of the intrinsic feature map, z ij Represents the ghost feature map obtained after j linear operations.

5. The method according to claim 2 or 4, characterized in that: The value of N is 3, and the linear operation is used in each individual channel of the ghost module of the 3×3 convolution kernel to generate linear processing of the feature map, thereby realizing convolution mapping.

6. A classification detection system for detecting defects in two-dimensional code labels based on lightweight convolution, characterized in that: include: fine contour extractors, feature extractors, and predictors; The fine contour extractor is used to: combine HSV color recognition and threshold segmentation, take the QR code label as the ROI area to extract features from the camera rough image to obtain the fine contour image of the QR code label. The camera rough image includes the QR code label in the label printing area captured by a high-resolution camera; The feature extractor is used to: extract defect features of the QR code label in the fine contour image using a lightweight convolution module LRB, wherein the lightweight convolution module LRB is constructed based on a ghost module GM and a lightweight residual block, the lightweight residual block is designed as an inverted residual block structure composed of a linear bottleneck structure, and the ghost module GM is used to replace the inverted residual structure and the depth-separable convolution layer in the two-dimensional convolution layer; The predictor is used to: use the Softmax classification layer to predict defect features and output defect classification results.

7. The system according to claim 6, characterized in that The feature extractor is used to extract the defect features of the QR code label by performing the following steps: Firstly, the defect features of the QR code label in the GM fine contour image are linearly operated through the ghost module with N×N convolution kernel to increase the dimension of the defect features, where N is an integer greater than or equal to 3; Secondly, the defect feature information of the fine contour image is retained through BN and ReLU6 linear activation functions; Finally, the fine contour image is subjected to dimensionality reduction processing by the ghost module GM with a 1×1 convolution kernel to obtain the defect features of the QR code label in the fine contour image.

8. The system according to claim 6 or 7, characterized in that: The fine contour extractor is used to extract the fine contour image of the QR code label by performing the following steps: Convert the camera rough image from RGB color space to HSV color space to obtain HSV image; Perform color threshold segmentation on HSV images; Perform ROI extraction processing on the segmented image to obtain a ROI area image; The ROI area image is refined and contour extracted to obtain a fine contour image of the QR code label.

9. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described in any one of claims 1 to 5 above.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images