Methods, apparatus, electronic equipment and storage media for detecting label defects

Abstract

This invention provides a label defect detection method, apparatus, electronic device, and storage medium, belonging to the field of machine vision technology. The method includes: acquiring a registered and aligned binarized image of a template edge and a binarized image of the edge to be tested; performing dilation processing on the template edge binarized image and finding its intersection with the binarized image of the edge to be tested; subtracting this intersection from the binarized image of the edge to be tested to obtain a first defect region with redundant features; performing dilation processing on the binarized image of the edge to be tested and finding its intersection with the template edge binarized image; subtracting this intersection from the template edge binarized image to obtain a second defect region with missing features; and determining the defect detection result based on the aforementioned defect regions. This invention, through a bidirectional dilation tolerance differential mechanism, effectively accommodates local deformation caused by label wrinkles and positional offset of edge pixels, solving the problem of numerous false positives caused by local deformation, and significantly reducing the false alarm rate while ensuring a high defect detection rate.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting label defects. Background Technology

[0002] In the manufacturing process of home appliances (such as refrigerators, air conditioners, and washing machines), minor defects such as printing errors, ink leaks, dirt, black spots, misalignment, or creases are unavoidable on product labels (including energy efficiency labels, parameter nameplates, and barcodes). The quality of the labels directly affects the product's reputation and compliance. Therefore, achieving efficient and accurate automatic detection of label defects on the production line has become an urgent requirement in intelligent manufacturing.

[0003] To meet the aforementioned requirements for automated detection, existing label defect detection schemes typically employ traditional image difference method (DM) or structural similarity index measure (SSIM) algorithms. Specifically, these schemes primarily use images of standard products acquired by a camera as templates. After performing basic corrections on the image of the label to be tested, a rigorous global comparison is directly performed between the template image and the image to be tested based on the pixel grayscale values. The degree of grayscale difference is used to determine whether the label to be tested has defects.

[0004] However, in actual industrial production, slight wrinkles or incomplete flattening of labels are unavoidable during application and imaging. This inevitably leads to localized deformation or minor spatial displacement in the image under test. Existing differential mechanisms based on strict grayscale comparison are extremely sensitive to such geometric changes. When performing pixel-level matching, their inherent algorithmic characteristics cannot distinguish between genuine defects and slight physical deformations. As a result, these non-defect-related minor deformations and edge shifts are directly amplified by the algorithm and judged as serious differences. Consequently, a large number of normal, qualified labels are misclassified as defective products, resulting in an extremely high false alarm rate and severely restricting the detection efficiency of actual production lines. Summary of the Invention

[0005] This invention provides a label defect detection method, device, electronic device, and storage medium to solve the problem that traditional rigid pixel comparison methods are unable to overcome the interference of false differences caused by the flexible deformation or uneven application of labels, which leads to a high risk of misjudgment in the detection system. It achieves the goal of retaining high sensitivity to detect real printing defects while intelligently filtering out false alarms caused by spatial position offset, thus greatly improving the accuracy and reliability of automatic label quality inspection in industrial production lines.

[0006] This invention provides a method for detecting label defects, comprising the following steps: Obtain the template edge binarized image of the reference label and the test edge binarized image of the label to be tested. The test edge binarized image and the template edge binarized image have been registered and aligned. After performing morphological dilation on the template edge binarized image, the intersection of the template edge binarized image and the edge binarized image to be tested is obtained to obtain a first tolerance intersection image; the difference operation is performed between the edge binarized image to be tested and the first tolerance intersection image to obtain a first defect region characterizing the redundant features of the label to be tested; After performing morphological dilation on the binarized image of the edge to be tested, the intersection of the binarized image of the template edge is obtained to get a second tolerance intersection image; the difference operation is performed on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region characterizing the missing features of the label to be tested; Based on the first defect area and the second defect area, the defect detection result of the label to be tested is determined.

[0007] According to a label defect detection method provided by the present invention, the step of obtaining a template edge binarized image of a reference label and a test edge binarized image of the label to be tested includes: Obtain the template edge feature map of the reference label and the test edge feature map of the label to be tested; Extract the geometric structural features that match each other between the template edge feature map and the edge feature map to be tested, so as to calculate the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map; Based on the spatial transformation mapping relationship, the edge feature map to be tested is transformed in spatial coordinates so that the edge feature map to be tested and the template edge feature map are registered and aligned. Threshold segmentation is performed on the registered and aligned template edge feature map and the edge feature map to be tested, respectively, to generate the template edge binarization map and the edge binarization map to be tested.

[0008] According to a label defect detection method provided by the present invention, the step of extracting the geometric structural features that match between the template edge feature map and the edge feature map to be tested, and calculating the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map, includes: Feature points are extracted from the template edge feature map and the edge feature map to be tested; The feature points of the extracted template edge feature map are matched with the feature points of the edge feature map to be tested to obtain a set of feature point pairs; Based on the coordinate correspondence in the set of feature points, a transformation matrix is ​​calculated to characterize the spatial transformation mapping relationship.

[0009] According to a label defect detection method provided by the present invention, the step of extracting feature points from the template edge feature map and the edge feature map to be tested, and matching the extracted feature points from the template edge feature map with the feature points from the edge feature map to be tested to obtain a set of feature point pairs includes: The template edge feature map and the edge feature map to be tested are input into the feature matching model to obtain the set of feature point pairs output by the feature matching model; The feature matching model is trained based on multiple edge feature image pairs and the true feature matching relationship labels corresponding to each edge feature image pair. The transformation matrix is ​​a homography transformation matrix.

[0010] According to a label defect detection method provided by the present invention, the step of acquiring the template edge feature map of the reference label and the test edge feature map of the label to be tested includes: Obtain the initial template image of the reference label and the initial image to be tested of the label to be tested; Edge-preserving filtering is performed on the initial template image and the initial test image respectively to obtain a template-filtered image and a test-filtered image; The edge gradient intensity values ​​of each pixel in the template filtered image and the test filtered image are calculated respectively to generate the template edge feature map and the test edge feature map containing image geometric structure information.

[0011] According to the label defect detection method provided by the present invention, the edge-preserving filtering process includes bilateral filtering or non-local mean filtering. The step of calculating the edge gradient intensity value of each pixel in the template filtered image and the test filtered image respectively includes: The Sobel edge operator is used to extract the edge gradient intensity values ​​of the template filtered image and the filter image to be tested, respectively.

[0012] According to a label defect detection method provided by the present invention, the initial image of the label to be tested is obtained, including: Obtain a global image of the target image containing the label to be tested; In the global image to be tested, determine the coordinates of local anchor points that match the preset local feature template; Based on the local anchor point coordinates and the preset coordinate offset relationship, the complete label boundary of the label to be tested in the global image to be tested is determined; Based on the complete label boundary, the initial image of the label to be tested is cropped from the global image to be tested.

[0013] According to a label defect detection method provided by the present invention, before acquiring a global image of the label to be tested, the method further includes: Obtain the global reference image of the reference label; In the global reference image, a local texture region is selected as the preset local feature template, and the reference label boundary of the reference label is defined. Calculate the positional offset between the preset local feature template and the reference label boundary, and establish the preset coordinate offset relationship.

[0014] According to a label defect detection method provided by the present invention, determining the defect detection result of the label to be tested based on the first defect region and the second defect region includes: Obtain each connected component in the first defect region and the second defect region; Calculate the area value of each of the connected components; The connected regions whose area values ​​are greater than a preset area threshold are determined as the target defect areas of the label to be tested. Based on all the target defect areas, the defect detection results of the label to be tested are generated.

[0015] According to a label defect detection method provided by the present invention, generating defect detection results of the label under test based on all the target defect areas includes: Obtain the defect pixel coordinates of the target defect region in the binarized image of the edge to be tested; Based on the inverse transformation of the spatial transformation mapping relationship, the defect pixel coordinates are inversely mapped to the local defect coordinates corresponding to the initial image to be tested; Based on the preset coordinate offset relationship, the local defect coordinates are reverse-mapped back to the global image under test to obtain the global defect coordinates of the label under test in the global image under test. The information containing the global defect coordinates will be used as the defect detection result of the label to be tested.

[0016] According to a label defect detection method provided by the present invention, morphological dilation processing is performed on the binary image of the template edge and on the binary image of the edge to be tested, including: The edge contours in the template edge binarization image and the edge contours in the test edge binarization image are respectively expanded outward by a preset number of pixels.

[0017] According to a label defect detection method provided by the present invention, the step of selecting a local texture region in the global reference image as the preset local feature template includes: Within the boundary of the baseline label, extract the saliency and uniqueness index of the texture features of each candidate region; Candidate regions whose texture feature saliency and uniqueness index meet preset conditions are determined as preset local feature templates; The preset local feature template includes at least one of a specific mark area, a specific image area, or a barcode area on the reference label.

[0018] The present invention also provides a label defect detection device, comprising: The edge image acquisition module is used to acquire the template edge binarized image of the reference label and the test edge binarized image of the label to be tested. The test edge binarized image and the template edge binarized image have been registered and aligned. The first defect extraction module is used to perform morphological dilation processing on the template edge binarized image and then find the intersection with the test edge binarized image to obtain a first tolerance intersection image; and to perform a difference operation on the test edge binarized image and the first tolerance intersection image to obtain a first defect region characterizing the redundant features of the test label. The second defect extraction module is used to perform morphological dilation processing on the binarized image of the edge to be tested and then find the intersection with the binarized image of the template edge to obtain a second tolerance intersection image; and to perform a difference operation on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region that represents the missing features of the label to be tested. The result determination module is used to determine the defect detection result of the label to be tested based on the first defect area and the second defect area.

[0019] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the label defect detection method as described above.

[0020] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the label defect detection method as described above.

[0021] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the label defect detection method as described above.

[0022] The label defect detection method, device, electronic device and storage medium provided by the present invention effectively accommodate the local deformation caused by label wrinkles and the positional offset of edge pixels through a bidirectional expansion tolerance differential mechanism, which solves the problem of a large number of false alarms caused by local deformation and significantly reduces the false alarm rate while ensuring a high defect detection rate. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating the label defect detection method provided by the present invention.

[0025] Figure 2 This is a schematic diagram illustrating the process of obtaining the template edge binarization image and the edge binarization image to be tested provided by the present invention.

[0026] Figure 3 This is a schematic diagram illustrating the process of obtaining the template edge feature map and the edge feature map to be tested provided by the present invention.

[0027] Figure 4 This is a schematic diagram of the label defect detection device provided by the present invention.

[0028] Figure 5 This is a schematic diagram of the operation of the label defect detection device provided by the present invention.

[0029] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0032] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0033] In the manufacturing process of home appliances such as refrigerators, air conditioners, and washing machines, product labels (such as energy efficiency labels, parameter nameplates, barcodes, etc.) may have various minor defects such as surface silkscreen printing errors, ink leakage, dirt, black spots, misalignment, creases, missing strokes, and blurriness. The quality of the labels directly affects the reputation and compliance of the products. Currently, the main method of inspection relies on manual inspection, which involves visually comparing the standard qualified benchmark label with the label to be tested to determine whether the label to be tested has silkscreen printing defects.

[0034] However, due to the differences in materials among different labels, using uniform imaging parameters for the camera cannot guarantee the uniformity of brightness in the acquired images. Existing defect detection methods struggle to address the problem of numerous false alarms caused by uneven brightness. Setting parameters individually for each label would be extremely difficult in practice. Furthermore, when multiple labels are present on the release paper, or when a single label has a large area, slight wrinkles or unevenness are unavoidable during imaging, leading to localized distortion in the acquired images. Current matching correction algorithms are based on rigid transformations or perspective transformations, which cannot correct the image of the locally deformed label to be completely identical to the template image of the reference label. This inevitably results in misalignment and offset issues, and existing algorithms cannot effectively solve the problem of false alarms caused by localized distortion.

[0035] This invention creatively proposes a label defect detection method based on a differential defect detection mechanism of edge topology, which greatly solves the problem of a large number of false alarms caused by local image deformation or matching deviation. While ensuring that all defects are detected, it can significantly reduce false alarms.

[0036] The following is combined with Figures 1-6 This invention describes the label defect detection method, apparatus, electronic device, and storage medium provided by the present invention.

[0037] To illustrate the specific implementation of this embodiment in detail and clearly, the execution subject and operating environment that may be involved in this embodiment will first be described. The label defect detection method provided in this embodiment can be executed by various electronic devices with image processing and computing capabilities. In the manufacturing scenario of home appliances (such as refrigerators, air conditioners, washing machines, etc.), the execution subject can specifically be an industrial control computer (ICC), edge computing node, vision controller, or data processing server containing a graphics processing unit (GPU) deployed on the production line. For the sake of brevity and coherence, the ICC will be consistently used as the execution subject of this label defect detection method in the following specific implementation.

[0038] Figure 1 This is a flowchart illustrating the label defect detection method provided by the present invention, as shown below. Figure 1 As shown, including but not limited to the following steps: Step 11: Obtain the template edge binarization image of the reference label and the test edge binarization image of the label to be tested.

[0039] In industrial visual inspection, the benchmark label can be a standard qualified product label that has been pre-confirmed manually or by a system, with complete printing content and no surface defects, and is used as the standard for subsequent inspection and comparison. The label to be tested refers to the actual product label that is currently circulating on the production line and whose surface printing quality needs to be judged. In order to fundamentally eliminate the interference of uneven image brightness caused by different reflectivity of different label materials or changes in production line ambient lighting, this embodiment does not adopt the traditional direct comparison method based on pixel grayscale values, but instead adopts a strategy of extracting the underlying geometric topology of the image.

[0040] As an optional implementation, images of the reference label are pre-acquired using an industrial camera. Edge extraction algorithms, such as the Canny operator, Prewitt operator, or other gradient-based operators, are used to extract the geometric contour boundaries from the image. Subsequently, binarization techniques such as global thresholding (e.g., the Otsu algorithm) or adaptive thresholding are used to generate an image containing only a black background and white outlines, or an image with a white background and black outlines. The resulting image is the template edge binarization image. Similarly, for the label images acquired online in real-time, the same edge extraction and binarization processing can be performed to generate the corresponding edge binarization image.

[0041] It should be noted that the binary image of the edge to be tested and the binary image of the template edge used in this embodiment have been registered and aligned. Since the label to be tested inevitably experiences positional shifts or rotations during the application process, the industrial control computer needs to ensure that the two are aligned in the spatial coordinate system before performing morphological calculations. This state of registration and alignment can be achieved by the industrial control computer using correction algorithms such as feature template matching, contour registration, or spatial geometric transformations based on feature points (such as affine transformations and perspective transformations) to map and align the overall contour of the label to be tested to the coordinate system of the reference label. After registration and alignment, the binary image of the edge to be tested and the binary image of the template edge have achieved a one-to-one correspondence in the macroscopic geometric structures such as patterns, text, and barcodes.

[0042] Step 12: After performing morphological dilation on the binarized image of the template edge, find the intersection with the binarized image of the edge to be tested to obtain a first tolerance intersection image; perform a difference operation on the binarized image of the edge to be tested and the first tolerance intersection image to obtain a first defect region representing the redundant features of the label to be tested.

[0043] In actual production, when there are multiple labels on the release paper or a single label has a large area, it is difficult to completely flatten the label during application, which can easily cause slight wrinkles. This minute physical deformation will be reflected in the image, causing random pixel shifts at local edges. If a rigid pixel-by-pixel comparison is performed, these normal deformation shifts will be falsely reported as defects by the system.

[0044] To eliminate this localized deformation, this embodiment innovatively employs morphological dilation processing, using structural elements of preset size and shape, such as rectangular, cross-shaped, or elliptical kernels, to expand all edge contour lines in the template edge binarization image outward at the pixel level. This operation essentially artificially constructs a tolerance isolation band around the standard contour, allowing for pixel fluctuations. Subsequently, a pixel-by-pixel logical AND operation is performed between the dilated template edge binarization image and the actual test edge binarization image, i.e., the intersection is calculated, thus obtaining the first tolerance intersection image. The resulting first tolerance intersection image accurately extracts all normal edge lines in the test label that fall within the standard tolerance range.

[0045] Finally, by subtracting the first tolerance intersection map from the initial binarized image of the edge to be tested, the reasonable and matching edges in the binarized image of the edge to be tested are completely eliminated. The remaining pixels that cannot be covered by the expanded template edge binarized image must be unexpected content that has appeared out of nowhere on the label to be tested. These redundant features are usually intuitively represented as ink splatter, dirt, black spots, or extra erroneous strokes in the production line printing process. In this embodiment, the areas where these redundant features are located are extracted, recorded, and summarized, thus obtaining the first defect area.

[0046] Step 13: After performing morphological dilation on the binarized image of the edge to be tested, find the intersection with the binarized image of the template edge to obtain a second tolerance intersection image; perform a difference operation on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region that represents the missing features of the label to be tested.

[0047] Step 12 above aims to find extra defects in the label to be tested. Correspondingly, the core purpose of step 13 is to accurately capture the missing elements on the label to be tested. This embodiment will use reverse topological differential logic, that is, perform morphological dilation processing on the current binary image of the edge to be tested, so that its existing edges expand outward, and then find the intersection with the standard template edge binary image to obtain the second tolerance intersection image.

[0048] In the above process, if a stroke of a character on the label under test is broken or a line of a barcode is missing, the severely missing area will still appear as a blank space even if the binarized image of the edge under test is dilated. Therefore, when finding the intersection, the standard line features corresponding to the template edge binarized image will be lost because they cannot find an intersection with the image under test.

[0049] Subsequently, the industrial control computer subtracts the second tolerance intersection map from the binarized image of the template edge. The remaining pixel area represents the portion that was originally present on the reference label but is missing from the label under test due to printing defects. These missing features are typically characterized by defects such as missing strokes, broken lines, or blurred printing causing the edges to disappear. In this embodiment, the area exceeding the subtraction is retained, thus constituting the second defect area.

[0050] Step 14: Determine the defect detection result of the label to be tested based on the first defect area and the second defect area.

[0051] After separating the first defect area and the second defect area, the industrial control computer can combine the data from these two parts to make a final judgment.

[0052] In one alternative implementation, the industrial control computer can perform an image-level logical OR fusion operation on the first and second defect regions to obtain a global defect map containing all types of defects. Further, pixel statistics can be directly performed on this global defect map, or morphological analyses such as connected component extraction and bounding box calculation can be performed. If the extracted defect features, such as the total pixel area of ​​the defect, the maximum connected component area, or the defect length, exceed a preset industrial allowable threshold, the defect detection result of the label under test can be determined as unqualified (defect present); conversely, if there are no defective areas or the defects are extremely small and belong to normal noise, it is judged as qualified.

[0053] Finally, the industrial control computer can output the defect detection result to the programmable logic controller (PLC) of the production line in the form of alarm signals and control commands to drive the robotic arm to remove defective products and display the defect coordinates in a highlighted visual display on the human-machine interface.

[0054] The label defect detection method provided in this embodiment utilizes a bidirectional dilation differential detection mechanism based on edge topology. This mechanism reduces color or grayscale images to edge binarized images, fundamentally eliminating color interference caused by differences in reflectivity between labels of different materials and variations in production line lighting. More importantly, it employs morphological dilation to construct a tolerance buffer zone and, through bidirectional intersection and difference operations between the template label and the label under test, accurately separates genuine ink excess and ink leakage defects. This gives the visual inspection system a strong structural tolerance capability when faced with slight wrinkles or minor local deformations caused by large-area labels that are difficult to flatten. It completely solves the problem of excessively high false alarms caused by the inability of traditional rigid pixel comparison algorithms to distinguish between real defects and reasonable deformations. While ensuring that all printing defects are intercepted and detected, it achieves a significant reduction in the false judgment rate.

[0055] Based on the above embodiments, in actual industrial production line processes, the labels under test often inevitably undergo positional shifts such as translation and rotation when entering the visual inspection station. To achieve accurate image alignment in complex industrial production environments and fundamentally eliminate color interference caused by differences in reflectivity of different label materials and variations in ambient lighting, this embodiment details the specific implementation methods for obtaining the template edge binarization image of the reference label and the test edge binarization image of the label under test.

[0056] Figure 2 This is a schematic diagram illustrating the process of obtaining the template edge binarization map and the test edge binarization map provided by the present invention, as shown below. Figure 2 As shown, obtaining the template edge binarized image of the reference label and the test edge binarized image of the label to be tested includes: Step 21: Obtain the template edge feature map of the reference label and the edge feature map of the label to be tested.

[0057] In this embodiment, an industrial camera can be used to acquire the original reference image of the reference label and the original test image of the label to be tested. Then, a basic edge gradient operator can be called to perform convolution operations.

[0058] In one feasible implementation, the Canny operator, Prewitt operator, or Laplacian operator can be used as the convolution kernel to perform global filtering sliding calculations on the original reference image and the original test image, respectively. By calculating the gray-level difference magnitude between adjacent pixels, the image, which originally contained color and background illumination, is transformed into a feature matrix representing the local gradient intensity distribution, thus generating a template edge feature map and a test edge feature map that retain continuous gray-level transition information, respectively.

[0059] Step 22: Extract the geometric structural features that match between the template edge feature map and the edge feature map to be tested, so as to calculate the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map.

[0060] After obtaining the template edge feature map and the edge feature map to be tested, this embodiment will find the offset pattern between the two in physical space.

[0061] As a feasible implementation method, a feature extraction algorithm based on global contour attributes can be used. Specifically, the geometric centroid coordinates, circumscribed rectangle boundaries, and principal direction angles of the main connected contours in the template edge feature map and the edge feature map to be tested are calculated respectively, and these macroscopic geometric attributes are used as the geometric structure features.

[0062] Subsequently, the two-dimensional translation vector (dx, dy) of the centroid of the edge feature map to be tested relative to the centroid of the template edge feature map is calculated by subtraction, and the principal direction angle deviation (θ) between the two is calculated. Based on this two-dimensional translation vector and the principal direction angle deviation, a two-dimensional affine transformation matrix is ​​constructed through geometric analysis.

[0063] In another feasible implementation, the Normalized Cross-Correlation (NCC) algorithm can be directly used to perform a full-image sliding matching search at the image pyramid scale to find the position offset with the largest correlation coefficient.

[0064] Finally, the two-dimensional affine transformation matrix or position offset calculated above is used as the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map.

[0065] Step 23: Based on the spatial transformation mapping relationship, perform spatial coordinate transformation on the edge feature map to be tested, so that the edge feature map to be tested and the template edge feature map complete the registration and alignment.

[0066] Furthermore, in this embodiment, the calculated spatial transformation mapping relationship (such as a two-dimensional affine transformation matrix) is used to perform matrix multiplication on the two-dimensional coordinates of each pixel in the edge feature map to be tested, and the new target coordinates after remapping are calculated.

[0067] Since the new target coordinates calculated after rotation and translation usually contain a fractional part (i.e., sub-pixel coordinates), they cannot be directly mapped to an integer pixel grid.

[0068] Therefore, as a specific implementation, this embodiment can call the bilinear interpolation algorithm or the bicubic interpolation algorithm to calculate the final gradient value of the sub-pixel coordinate position by performing a distance-weighted summation based on the gradient intensity values ​​of the 4 or 16 neighboring integer pixels around the sub-pixel coordinate. Through this remapping and interpolation operation, the translation and rotation deviations of the edge feature map to be tested can be completely eliminated in the pixel space coordinate system, so that the edge positions of the edge feature map to be tested and the template edge feature map are perfectly coincident, thereby completing the registration and alignment.

[0069] Step 24: Perform threshold segmentation on the registered and aligned template edge feature map and the edge feature map to be tested, respectively, to generate the template edge binarized map and the edge binarized map to be tested.

[0070] After completing the sub-pixel level registration and alignment, the continuous gradient feature matrix can be discretized into deterministic states containing only edges and no edges.

[0071] In one feasible implementation, the Otsu's Method algorithm can be invoked to automatically traverse all gradient intensity levels and calculate and find the optimal threshold that maximizes the inter-class variance between the foreground edge pixel set and the background pixel set.

[0072] Subsequently, using the determined optimal threshold, the registered and aligned template edge feature map and the edge feature map to be tested are compared pixel by pixel. For example, pixels with gradient strength greater than the optimal threshold are forcibly assigned a value of 255 (i.e., white, representing a valid contour), while pixels with gradient strength less than or equal to the optimal threshold are forcibly assigned a value of 0 (i.e., black, representing a background without edges). Through this non-linear numerical truncation operation, the template edge binarized map and the edge binarized map to be tested are finally generated for morphological difference operations.

[0073] Considering that performing binarization before spatial transformation would cause severe jaggedness and blurring (i.e., grayscale transition bands) in the originally clear-cut binarized edges, severely compromising the topological integrity of the edges, this embodiment adopts a calculation process of first performing spatial interpolation registration on the grayscale / gradient feature map, followed by binarization thresholding. This allows the edge feature map to fully utilize continuous gradient intensity information during interpolation, maximizing the preservation of the smoothness and sub-pixel accuracy of the label edges. Thresholding is then performed without any loss in registration and alignment, ensuring that the final generated template edge binarized image and the edge binarized image under test have sharp and perfectly matched contours.

[0074] Based on the above embodiments, in order to clearly illustrate how to obtain the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map with high accuracy and robustness, as an optional embodiment, a specific implementation method is provided for extracting the geometric structural features that match between the template edge feature map and the edge feature map to be tested, so as to calculate the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map. This implementation method mainly includes, but is not limited to: Feature points are extracted from the template edge feature map and the edge feature map to be tested; The feature points of the extracted template edge feature map are matched with the feature points of the edge feature map to be tested to obtain a set of feature point pairs; Based on the coordinate correspondence in the set of feature points, a transformation matrix is ​​calculated to characterize the spatial transformation mapping relationship.

[0075] In one feasible implementation, traditional machine vision feature extraction algorithms can be invoked, such as the Scale-Invariant Feature Transform (SIFT) algorithm, the SpeededUp Robust Features (SURF) algorithm, or the Oriented Fast and Rotated BRIEF (ORB) feature operator. Taking the ORB feature operator as an example, firstly, the FAST corner detection algorithm is used to traverse the template edge feature map and the edge feature map to be tested, identifying local extreme points where pixel grayscale values ​​change drastically as feature points; then, the BRIEF algorithm is used to extract local pixel blocks around the feature points in the template edge feature map and the feature points in the edge feature map to be tested, generating feature descriptors composed of binary codes respectively.

[0076] After obtaining the feature points with feature descriptors, the correspondence between the template edge feature map and the edge feature map to be tested can be further established.

[0077] As a specific implementation, the Brute Force Matcher algorithm or the Fast Library for Approximate Nearest Neighbors (FLANN) algorithm can be invoked. In the embodiment using the Brute Force Matcher algorithm, the Hamming distance or Euclidean distance between the feature descriptors of feature points in the template edge feature map and the feature descriptors of feature points in the test edge feature map is calculated one by one. Then, the feature points with the smallest distance that meet the preset distance threshold are combined as successfully matched nodes, thereby obtaining the set of feature point pairs containing all correctly matched correspondences.

[0078] Furthermore, in industrial settings, the patterns on label surfaces may exhibit highly similar, repetitive textures, making traditional manual feature operators prone to mismatches. To address this issue, this embodiment also provides an optional implementation based on deep learning networks, including but not limited to: The template edge feature map and the edge feature map to be tested are input into the feature matching model to obtain the set of feature point pairs output by the feature matching model; The feature matching model is trained based on multiple edge feature image pairs and the true feature matching relationship label corresponding to each edge feature image pair.

[0079] The industrial control computer (ICC) synchronously inputs the template edge feature map and the edge feature map to be tested into a pre-deployed feature matching model, such as the LightGlue deep learning feature matching network based on an attention mechanism. This feature matching model can extract deep image semantic features through its internal convolutional neural network branches and capture global contextual information using self-attention and cross-attention mechanisms. In this way, the feature matching model can perceive the overall geometric topological coherence of the label and output a set of feature point pairs filtered out of repetitive texture interference to the ICC.

[0080] To enable the feature matching model to possess the aforementioned high-precision matching capability, the feature matching model is trained based on multiple edge feature image pairs and the corresponding true feature matching relationship labels for each edge feature image pair.

[0081] During the offline training phase of the model, a large number of edge feature image pairs containing different lighting variations, rotation angles, and deformation degrees are pre-collected. The true feature matching labels corresponding to each edge feature image pair are obtained through manual calibration or high-precision instrument localization. Then, the loss function is calculated using these true feature matching labels, and the network weights of the feature matching model are continuously updated using the backpropagation algorithm until the loss function converges, thus obtaining the trained feature matching model.

[0082] After obtaining the feature point pair set, the first two-dimensional pixel coordinates of each matching pair in the feature point pair set in the template edge feature map and the second two-dimensional pixel coordinates in the edge feature map to be tested can be extracted to establish the coordinate correspondence.

[0083] To eliminate the interference of a very small number of residual erroneous matching points on the final matrix calculation, this embodiment also calls the Random Sample Consensus (RANSAC) algorithm or the Least Median of Squares (LMeDS) algorithm. For example, the RANSAC algorithm is used to randomly select a minimum number of feature point pairs from the feature point pair set, iteratively calculate and retain the mathematical transformation matrix with the most inliers.

[0084] In the above iterative calculation process, the final transformation matrix obtained is a homography transformation matrix. The homography transformation matrix can generally be a 3x3 matrix. A 3-dimensional projection transformation matrix. Compared to rigid or affine transformation matrices that can only characterize translation and rotation in a two-dimensional plane, this homography transformation matrix can not only accurately characterize the positional offset and angular rotation of the edge feature map under test in a two-dimensional plane, but also describe with great precision the perspective distortion caused by changes in the shooting angle of industrial cameras or the incomplete flatness of the label surface in three-dimensional space.

[0085] The label defect detection method provided in this embodiment, by introducing traditional feature operators or deep learning-based feature matching models and combining them with the RANSAC algorithm to eliminate outliers, ensures that a high-purity and high-accuracy set of feature point pairs can still be extracted even under complex production line lighting changes and label repetitive texture interference. More importantly, by using the homography transformation matrix as a representation of spatial transformation mapping relationships, it can effectively correct the perspective distortion of the label under test in three-dimensional space, providing sub-pixel-level registration accuracy for subsequent image detection, thereby significantly reducing the false alarm rate of morphological difference algorithms.

[0086] Based on the above embodiments, in order to effectively eliminate light and color interference caused by changes in production line ambient lighting or differences in the reflectivity of label materials, this embodiment provides a specific implementation method for obtaining the template edge feature map of the reference label and the test edge feature map of the label to be tested.

[0087] Figure 3 This is a schematic diagram illustrating the process of obtaining the template edge feature map and the edge feature map to be tested provided by the present invention, as shown below. Figure 3 As shown, it mainly includes, but is not limited to: Step 31: Obtain the initial template image of the reference label and the initial image to be tested of the label to be tested.

[0088] In one alternative implementation, raw digital images of standard, qualified reference labels are pre-acquired via an industrial camera interface and stored in memory as initial template images. Subsequently, during normal production line operation, control sensors capture raw digital images in real time of products on the current production line bearing the labels to be tested, using these as the initial images to be tested. The acquired initial template image and the initial images to be tested are typically color images containing complete RGB color information, or single-channel grayscale images containing rich grayscale tonal variations.

[0089] Step 32: Perform edge-preserving filtering on the initial template image and the initial test image respectively to obtain the template-filtered image and the test-filtered image.

[0090] When acquiring initial template images and initial test images in industrial settings, high-frequency noise such as sensor background noise or environmental dust is inevitably introduced. Traditional smoothing filtering algorithms, such as simple mean filtering or standard Gaussian filtering, can blur the edges of label text or barcodes while removing noise, severely affecting the accuracy of subsequent edge extraction. Therefore, this embodiment uses an edge-preserving filtering algorithm for pre-denoising.

[0091] In one alternative implementation, anisotropic diffusion filtering, guided filtering, or other edge-preserving smoothing filtering algorithms can be employed. Taking anisotropic diffusion filtering as an example, isotropic diffusion is performed in flat areas of the image using partial differential equations to smooth noise, while diffusion is actively suppressed in contour areas with large image gradients. This eliminates random surface noise while perfectly preserving the sharpness of boundaries such as labels and text. After filtering, a smooth, denoised, and clearly edge-defined template image and a test image are generated.

[0092] Step 33: Calculate the edge gradient intensity value of each pixel in the template filtered image and the test filtered image respectively, so as to generate the template edge feature map and the test edge feature map containing image geometric structure information.

[0093] After obtaining the smoothed and denoised template image and the image to be tested, the spatial first derivatives of both can be calculated in the horizontal and vertical directions.

[0094] In one alternative implementation, a specific differential kernel, such as the Prewitt edge operator, the Scharr edge operator, or the Roberts crossover operator, can be configured to perform two-dimensional spatial convolution operations on the template-filtered image and the image to be filtered. Through the two-dimensional spatial convolution operation, the gradient components of each pixel in the horizontal and vertical directions are calculated, and then the gradient magnitude (such as the square root of the sum of the squares of the horizontal and vertical components) is obtained. The obtained gradient magnitude is used as the edge gradient intensity value of the current pixel.

[0095] Finally, the edge gradient intensity values ​​of each pixel are re-mapped according to their original spatial coordinates to obtain the template edge feature map and the edge feature map to be tested, which have been stripped of their original color and overall brightness and only retain high-frequency contour lines.

[0096] At this point, the template edge feature map and the edge feature map to be tested encompass the boundary coordinates and shape topology of all core elements such as text, lines and barcodes on the label, that is, they contain the geometric structure information of the image.

[0097] The label defect detection method provided in this embodiment performs edge-preserving filtering and edge gradient strength calculation sequentially on the initial template image and the initial test image. At the underlying processing level, it effectively eliminates random high-frequency noise generated during image acquisition. Since gradient calculation is essentially a mathematical operation to extract high-frequency details of the image and suppress low-frequency background, this process completely removes the brightness and color information caused by different label surface materials and uneven ambient lighting, retaining only the purest label geometric structure information. This gives the visual defect detection system a strong anti-lighting interference capability when facing various reflective labels or production line lighting changes, fundamentally solving the problem of amplifying underlying feature differences due to uneven local brightness, which leads to a large number of false alarms.

[0098] Based on the above embodiments, as an optional embodiment, the edge-preserving filtering process used includes bilateral filtering or non-local mean filtering.

[0099] In one feasible implementation, in order to balance the real-time detection performance of the industrial production line with the image smoothing effect, a bilateral filtering algorithm is invoked for the initial template image and the initial image to be tested.

[0100] The bilateral filtering algorithm used in this embodiment introduces not only pixel spatial distance as Gaussian spatial weights when performing two-dimensional spatial convolution operations, but also pixel grayscale value differences as range weights. Through this dual weight determination mechanism, depth mean smoothing is performed in grayscale flat areas such as label backgrounds or solid color patterns; while at the outline boundaries of label text, logos, or barcodes, due to the extremely large pixel grayscale differences, the calculated range weights approach zero, thereby completely blocking cross-boundary pixel smoothing fusion at the underlying computation level.

[0101] In another feasible implementation, for labels made of special materials with extremely dense and irregular reflective noise on their surface, such as brushed metal nameplates or laser anti-counterfeiting labels, a non-local mean filtering algorithm can be invoked. This non-local mean filtering algorithm is not limited to using neighboring pixels around the target pixel; instead, it searches for structurally similar local pixel blocks within a global search window of the initial template image and the initial test image, and performs a weighted average. This implementation can completely eliminate complex high-frequency noise while perfectly and losslessly preserving the geometric sharpness of the original label edges.

[0102] In one optional embodiment, calculating the edge gradient intensity value of each pixel in the template filtered image and the filter image to be tested may include, but is not limited to: The Sobel edge operator is used to extract the edge gradient intensity values ​​of the template filtered image and the filter image to be tested, respectively.

[0103] After obtaining the template filtered image and the filter image to be tested, which have undergone edge-preserving smoothing and denoising, this embodiment transforms the image into a purely geometric topological representation. In a specific implementation, 3... A 3-dimensional Sobel edge operator, which specifically comprises a horizontal convolutional kernel with horizontal difference weights and a vertical convolutional kernel with vertical difference weights.

[0104] Furthermore, a global two-dimensional spatial sliding convolution operation is performed on the template filtered image and the image to be tested using a horizontal convolution kernel to calculate the horizontal gradient component of each pixel in the horizontal direction; simultaneously, the same global two-dimensional spatial sliding convolution operation is performed using a vertical convolution kernel to calculate the vertical gradient component of each pixel in the vertical direction.

[0105] Subsequently, by adding the absolute values ​​of the horizontal gradient components and the vertical gradient components of the same pixel, or by calculating the square root of the sum of the squares of the horizontal gradient components and the squares of the vertical gradient components, the calculated values ​​are used as the edge gradient intensity values ​​of each pixel in the template filtered image and the image to be tested.

[0106] This embodiment fundamentally overcomes the fatal flaw of traditional Gaussian filtering, which inevitably leads to edge blurring during noise removal, by employing bilateral filtering or nonlocal mean filtering algorithms, ensuring that the sharpness of minute strokes and barcode boundaries is not compromised. Simultaneously, it utilizes the Sobel edge operator to extract edge gradient intensity values. Because the Sobel edge operator inherently possesses a weighted smoothing effect on adjacent rows or columns in the difference calculation, it enables secondary suppression of residual minute noise when extracting edge contours in both the template filtered image and the test filtered image. This entire feature extraction pipeline, combining edge-preserving filtering and the Sobel edge operator, achieves extremely high-precision sub-pixel-level edge extraction quality, ensuring that the subsequent output feature map is completely free from interference from material reflections and uneven illumination.

[0107] In real industrial production lines, the field of view of industrial cameras is usually much larger than the size of a single label. Directly performing high-precision geometric registration and morphological processing on the entire panoramic image would not only consume a significant amount of system computing power but would also be highly susceptible to interference from the complex background of the production line. Therefore, this embodiment employs a hierarchical, rapid positioning method from local to global perspectives. In one optional embodiment, acquiring the initial image of the label to be tested may include, but is not limited to: Obtain a global image of the target image containing the label to be tested; In the global image to be tested, determine the coordinates of local anchor points that match the preset local feature template; Based on the local anchor point coordinates and the preset coordinate offset relationship, the complete label boundary of the label to be tested in the global image to be tested is determined; Based on the complete label boundary, the initial image of the label to be tested is cropped from the global image to be tested.

[0108] In this embodiment, the industrial control computer triggers a photoelectric sensor via an industrial camera on the production line to capture real-time images, obtaining a macroscopic field-of-view image of the product currently flowing through the vision inspection station, encompassing the entire surface of the product casing or a large area of ​​release paper background. This macroscopic field-of-view image includes not only the label to be inspected but also a large number of irrelevant conveyor belts or background structures. The acquired macroscopic field-of-view image is used as the global image to be inspected and loaded completely into the system's running memory.

[0109] To achieve fast and interference-resistant positioning, a preset local feature template is pre-loaded in the system memory. In one feasible implementation, the preset local feature template does not include the entire content of the label, but rather a small area image with strong texture saliency and absolute uniqueness, pre-selected manually or by algorithm, such as a specific brand logo pattern, a complex anti-counterfeiting mark, or a specific positioning angle of a barcode.

[0110] Furthermore, a two-dimensional sliding window search is performed in the global image to be tested by invoking a fast template matching algorithm. In specific implementation, algorithms such as Normalized Cross-Correlation (NCC), Sum of Absolute Differences (SAD), or Shape-Based Matching based on edge topology can be used to calculate the similarity score matrix between the image data within the sliding window and the preset local feature template one by one.

[0111] Subsequently, the industrial control computer extracts the global extreme value from the similarity score matrix and accurately locates the center position of the pixel with the highest similarity score and greater than the preset similarity threshold as the local anchor point coordinates.

[0112] After obtaining the precise coordinates of the local anchor point, the preset coordinate offset relationship that has been pre-calibrated and stored by the system can be invoked.

[0113] In a specific embodiment, the preset coordinate offset relationship is represented as a two-dimensional spatial geometric vector set containing constant distances in the horizontal and vertical directions. It accurately records the relative pixel offsets from the local anchor point coordinates to the real physical boundary of the label to be tested, such as the upper left and lower right corners of the label outline.

[0114] Furthermore, this embodiment can directly and accurately calculate the absolute pixel coordinates of each vertex of the edge of the label under test by performing vector addition or affine translation matrix operations on the obtained local anchor point coordinates. Then, a closed rectangular bounding box or polygonal outline is constructed using the calculated absolute pixel coordinates of each vertex of the edge of the label under test, thereby clearly defining the complete label boundary of the label under test in the global image under test within a large field of view.

[0115] After determining the coordinates of the complete label boundary, a hard cropping operation of the Region of Interest (ROI) can be performed at the underlying data matrix level. Specifically, based on the row and column coordinate index sequence formed by the complete label boundary, the underlying pixel data tensor corresponding to the global image under test can be sliced ​​and cropped, independently extracting the pixel data belonging to the complete label boundary, thereby generating an initial image under test containing only the label itself. Finally, this clean initial image under test is sent to the subsequent edge-preserving filtering process.

[0116] This embodiment employs a hierarchical localization approach, moving from local to global. It utilizes a pre-defined local feature template with the most prominent texture features to perform rapid initial matching of small regions within the global test image to find local anchor point coordinates. This effectively avoids the massive computational cost and risk of repeated texture mismatches associated with directly performing full-size large-template matching in a global test image containing a large background. Subsequently, a constant pre-defined coordinate offset relationship is cleverly combined to directly deduce the complete label boundary and crop the initial test image, completely eliminating interference from irrelevant environmental pixels such as production line backgrounds, conveyor belts, or product casings on subsequent feature extraction. This dimensionality reduction and cropping mechanism based on local anchor point-to-global boundary mapping not only significantly improves the computational speed of subsequent low-level image processing and high-precision sub-pixel secondary matching but also ensures that the entire algorithm possesses extremely high spatial positioning accuracy and robustness when facing scenarios with multiple signatures on a single document or large-area complex production line backgrounds.

[0117] To initialize the positioning reference and clipping parameters before starting real-time detection on the actual production line, this embodiment provides specific template creation and parameter calibration steps before acquiring the global image of the label to be tested, including but not limited to: Obtain the global reference image of the reference label; In the global reference image, a local texture region is selected as the preset local feature template, and the reference label boundary of the reference label is defined. Calculate the positional offset between the preset local feature template and the reference label boundary, and establish the preset coordinate offset relationship.

[0118] In one alternative implementation, during the offline debugging or template creation phase of the vision inspection system, an industrial camera is controlled to capture a global image of a reference label placed at a standard inspection station on the production line under standard lighting conditions. The acquired wide-field-of-view digital image, which includes the complete production line background and the full view of the reference label, is stored in the system memory and used as the global reference image.

[0119] After acquiring the global reference image, the industrial control machine will further establish the image anchor point reference and the boundary reference for subsequent online initial matching and image cropping.

[0120] In one optional implementation, the industrial control computer displays a global reference image through a human-machine interface and receives the coordinates of a first rectangle and a second rectangle input by the operator via an input device (such as mouse drag-and-drop selection). The industrial control computer extracts a local pixel block enclosed by the first rectangle, containing a corner of a specific brand logo or high-contrast anti-counterfeiting mark, and uses this extracted local pixel block as the local texture region, setting the local texture region as a preset local feature template. Simultaneously, the industrial control computer sets the rectangular coordinate region enclosed by the second rectangle, which exactly encompasses the entire physical edge of the reference label, as the reference label boundary.

[0121] In another alternative implementation, the industrial control computer can call an image saliency detection algorithm to automatically evaluate the texture richness and uniqueness in the global reference image, automatically crop out a small pixel region with the largest information entropy and the most dramatic gradient change as a local texture region and set it as a preset local feature template; then, the maximum connected bounding rectangle of the reference label is automatically extracted using a connected component analysis algorithm to automatically delineate the boundary of the reference label.

[0122] After defining the template and boundaries, this embodiment continues to perform spatial vector distance calculations in the underlying two-dimensional pixel coordinate system. Specifically, the center pixel coordinates (or fixed corner coordinates) of the preset local feature template can be extracted first, and the top-left and bottom-right vertex pixel coordinates of the baseline label boundary can be extracted respectively. Subsequently, the horizontal and vertical absolute distances from the center pixel coordinates to the top-left vertex pixel coordinates, and the horizontal and vertical absolute distances from the center pixel coordinates to the bottom-right vertex pixel coordinates are calculated.

[0123] Finally, the industrial control computer uses the calculated values ​​of a series of horizontal and vertical absolute distances as position offsets, and encapsulates the position offsets as specific spatial geometric vector configuration parameters and stores them in the database, thereby establishing and solidifying the preset coordinate offset relationship at the system bottom layer.

[0124] In this embodiment, the most prominent local texture regions are extracted from the global reference image as preset local feature templates. The positional offset between the preset local feature template and the reference label boundary is calculated by the distance of the underlying pixel coordinates, thereby establishing a constant preset coordinate offset relationship. The originally computationally intensive full-size image matching is reduced to a lightweight local anchor point search with simple constant vector translation. This allows the visual defect detection method to handle scenarios such as multiple signatures on a single document or multiple product line changes without maintaining and comparing a large global label template in memory. It only needs to call a very small preset local feature template and a digitized preset coordinate offset relationship to instantly deduce the complete clipping boundary in the macroscopic view. This not only greatly saves the video memory and memory space occupied by the vision system, but also significantly reduces the computational delay of subsequent pipeline real-time positioning and clipping operations from the source of algorithm initialization.

[0125] To effectively filter out minute interfering pixels caused by image sensor noise or environmental dust, and accurately extract label defects that are truly industrially significant, this embodiment further provides an implementation method for determining the defect detection results of the label under test based on a first defect region and a second defect region, mainly including but not limited to the following steps: Obtain each connected component in the first defect region and the second defect region; Calculate the area value of each of the connected components; The connected regions whose area values ​​are greater than a preset area threshold are determined as the target defect areas of the label to be tested. Based on all the target defect areas, the defect detection results of the label to be tested are generated.

[0126] In the underlying binary image matrix, the first and second defect regions are represented as discrete sets of white (or black) foreground pixels. This embodiment utilizes a connected component labeling algorithm, such as a two-pass algorithm or a seed-filling-based region growing algorithm, to traverse all foreground pixels in the binary image matrix using a preset neighborhood determination template (e.g., 4-connected or 8-connected neighborhood). This clusters interconnected pixels and assigns them the same independent connected component numerical identifier, thereby dividing the discrete pixels into independent connected components with topological connectivity.

[0127] After labeling and clustering connected components, feature statistics are performed on pixel clusters with the same connected component numerical identifier. Specifically, the industrial control computer traverses the underlying image labeling matrix and accumulates the total number of pixels belonging to the same connected component numerical identifier.

[0128] In one alternative implementation, the total number of pure pixels obtained by accumulation is directly used as the area value of each corresponding connected region; or, in combination with the calibration parameters of the industrial camera (such as the actual physical square millimeter size corresponding to a single pixel), the total number of pixels is converted into an absolute value with actual physical units, and the converted absolute value is used as the area value of each connected region.

[0129] After completing the above morphological filtering and defect confirmation, this embodiment will summarize the extracted target defect regions and determine whether the set of all target defect regions is an empty set.

[0130] If the set of all target defect areas is determined to be empty, it means that there are no substantial defects on the surface of the label to be tested that exceed the industrial tolerance range. At this time, a defect detection result indicating "label is qualified" will be generated.

[0131] If the set of all target defect areas is determined to be non-empty, the position coordinates of the bounding rectangle of all target defect areas and the defect classification attributes are extracted. That is, based on whether it comes from the first defect area or the second defect area, the excess ink defect or the ink leakage defect is distinguished, and a defect detection result containing specific coordinates and attribute information is generated to represent "labeling non-conformity".

[0132] Finally, the industrial control computer outputs the defect detection results to the production line's programmable logic controller or human-machine interface to trigger alarms or execute rejection actions.

[0133] The label defect detection method provided in this embodiment introduces a connected component labeling algorithm and patch feature statistical analysis technology to transform scattered defect pixels into connected component objects with independent topological structures. Furthermore, it extracts area values ​​for threshold filtering. This underlying filtering mechanism based on physical connectivity and area values ​​can effectively filter out non-substantial isolated interference noise points caused by camera sensor noise, airborne dust, or normal minute printing fluctuations. This ensures that the finally confirmed target defect area fully conforms to the actual material defect judgment standards of the industrial production line. It not only significantly improves the noise resistance robustness of the visual defect detection system in complex physical environments, but also ensures that no real major defects are missed, while further strongly reducing the false alarm rate from the logical source of the final decision.

[0134] Based on the above embodiments, in order to accurately indicate the physical location of defects to operators in the panoramic view of the original production line during the final output, and to achieve intuitive defect visualization and traceability, this embodiment elaborates in detail the specific coordinate mapping and restoration steps for generating the defect detection results of the label to be tested based on the target defect area, mainly including but not limited to: Based on the inverse transformation of the spatial transformation mapping relationship, the defect pixel coordinates are inversely mapped to the local defect coordinates corresponding to the initial image to be tested; Based on the preset coordinate offset relationship, the local defect coordinates are reverse-mapped back to the global image under test to obtain the global defect coordinates of the label under test in the global image under test. After identifying the target defect region constituting a substantial flaw, this embodiment invokes a contour extraction algorithm, such as the Suzuki 85 contour tracking algorithm or a bounding box generation algorithm. In the two-dimensional matrix coordinate system of the current binary image of the edge to be measured, the coordinates of the circumscribed rectangle vertices or the coordinates of a fine set of boundary points surrounding the target defect region are calculated and extracted. One or more sets of extracted two-dimensional pixel coordinates are recorded as the defect pixel coordinates. At this point, the defect pixel coordinates are in the local coordinate system after cropping and registration alignment.

[0135] Since a positive spatial coordinate transformation was applied to the image in the previous embodiment to align the images, this embodiment needs to undo this geometric transformation to restore the coordinates. In specific implementation, the industrial control computer retrieves the spatial transformation mapping relationship calculated in the previous steps from memory, such as 3. A 3-dimensional homography transformation matrix is ​​generated, and the inverse matrix of the homography transformation matrix is ​​solved through underlying linear algebra operations.

[0136] The linear algebra operations used can be Gaussian-Jordan elimination or singular value decomposition, etc.

[0137] Subsequently, this embodiment uses the solved inverse matrix to perform matrix multiplication on the defect pixel coordinates. Through this inverse spatial geometry deduction, the defect pixel coordinates, originally in the registration coordinate system, are rotated and scaled back to the unaligned space, thereby calculating the local defect coordinates corresponding to the initial test image coordinate system.

[0138] After restoring the registration offset, this embodiment further cancels the image cropping offset, including: retrieving the preset coordinate offset relationship established based on the anchor point in the previous positioning and cropping step, such as the horizontal and vertical translation vectors of the upper left corner of the cropped area relative to the entire large image. These horizontal and vertical translation vectors are then directly added to the local defect coordinates obtained in the previous step, i.e., a simple two-dimensional vector addition operation is performed. After vector addition, the boundary of the local cropping is completely crossed, converting the coordinate values ​​into absolute pixel coordinates corresponding to the original large image coordinate system without any processing. At this point, these absolute coordinates are determined as the global defect coordinates of the label to be tested in the global image under test.

[0139] Finally, this embodiment will encapsulate the obtained global defect coordinates into a standard data structure protocol, such as encapsulating them into a JSON format string, and attach attributes such as the defect type (redundant feature or missing feature) and area size corresponding to the coordinates to generate a complete defect detection result.

[0140] Furthermore, the defect detection results can be directly transmitted to the high-definition display on the production line. Using the global defect coordinates, a red warning box can be superimposed on the displayed global image under test, thereby visually indicating the physical location of the specific defect to the on-site personnel.

[0141] This embodiment constructs a rigorous coordinate backtracking calculation link by introducing the inverse transformation of spatial transformation mapping relationship and the inverse addition mapping of preset coordinate offset relationship. It can efficiently complete complex differential detection in a very small registered image space, and can project the final alarm coordinates back to the original macroscopic field of view without any processing. It solves the pain point that local detection algorithms are difficult to accurately locate defects in the global picture, and provides operators with absolutely accurate real physical space coordinate guidance for production line re-inspection and robotic arms to carry out physical removal.

[0142] To further quantify the tolerance buffer range in morphological processing and ensure that it does not mask real subtle defects while being compatible with physical deformation, this embodiment provides a specific implementation method for performing morphological dilation processing on the binary image of the template edge and on the binary image of the edge to be tested, mainly including: Within the boundary of the baseline label, extract the saliency and uniqueness index of the texture features of each candidate region; Candidate regions whose texture feature saliency and uniqueness index meet preset conditions are determined as preset local feature templates; The preset local feature template includes at least one of a specific mark area, a specific image area, or a barcode area on the reference label.

[0143] After the industrial control computer completes the binarization process, the edge contours in the template edge binarization map and the edge contours in the test edge binarization map are both represented in the computer's underlying data as thin pixel connected bands formed by foreground pixels.

[0144] Optionally, in this embodiment, when performing the operation of expanding the edge contour outward, a morphological structural element kernel of a specific size and shape is pre-constructed in the system memory.

[0145] As an alternative implementation, the structural element core can be set as an isotropic square matrix core, or a cross-shaped structural matrix core constructed to better preserve the sharpness of the label edge contour corners.

[0146] Furthermore, this embodiment utilizes a predefined morphological structuring element kernel to perform two-dimensional sliding maximum filtering operations on both the template edge binarized image and the test edge binarized image, forcibly flipping background pixels within the area covered by the structuring element kernel into foreground contour pixels. This allows the originally thin edge contour to thicken and enlarge uniformly in all directions, thereby constructing a tolerance isolation band of a certain physical thickness for both the template edge binarized image and the test edge binarized image.

[0147] In actual labeling processes on industrial production lines, when multiple labels are present on the release paper simultaneously, or when a single label has a large physical area, uneven adhesion tension can easily lead to slight wrinkles or incomplete flattening of the label surface. These minute undulations and deformations in the three-dimensional physical world, after being imaged in two dimensions by an industrial camera, inevitably cause unpredictable physical positional shifts in the local edge contours within the pixel coordinate system. To accurately and safely absorb these shifts caused by local deformation, this embodiment limits the outward expansion of a predetermined number of pixels to a specific range of 1 to 5 pixels.

[0148] In actual production line deployment, parameters can be flexibly and dynamically configured between 1 and 5 pixels based on the material elasticity properties of the current batch of labels or the optical imaging resolution of the camera. For example, for acrylic labels that are relatively hard, have high flatness, and are not prone to wrinkling, the preset number of pixels can be configured to the lower limit of 1 or 2 pixels (corresponding to 3 pixels at the bottom layer). A 3-dimensional structural element core is used to provide a compact and strict tolerance range; however, for ultra-thin paper labels that are prone to local physical wrinkles, the preset number of pixel widths can be adjusted upwards to 4 or 5 pixels (corresponding to 9 pixels in the underlying layer). 9 or 11 The structure element core of size 11 provides a wider tolerance boundary on the template edge binarization map and the test edge binarization map, ensuring that the test edge contour that has undergone local physical deformation can still be completely covered by the expanded safety tolerance intersection.

[0149] This embodiment establishes a precise flexible tolerance mechanism by numerically quantifying and controlling the width of a preset number of pixels at the underlying mathematical morphology operation level. Strictly defining the expansion width within the critical range of 1 to 5 pixels not only allows the algorithm to readily accommodate legitimate edge pixel offsets induced by local deformation of the label or uneven application, fundamentally eliminating spurious differential interference caused by non-defect physical deformation; it also effectively avoids the risk of incorrect adhesion of originally independent edge contours on the label due to excessively large expansion width settings, or the risk of real minute ink leakage defects being buried by overly coarsened edge contours. Thus, a perfect balance between extremely low false positive rate and extremely high minute defect detection rate is achieved at the underlying algorithm architecture level.

[0150] In order to completely eliminate the subjective arbitrariness caused by manual selection in the offline template creation stage and to ensure the absolute reliability of the anchor point in the initial matching stage of online detection, this embodiment provides a specific automated optimization implementation method for selecting local texture regions in the global reference image as preset local feature templates.

[0151] Specifically, selecting a local texture region in the global reference image as the preset local feature template may include, but is not limited to: Within the boundary of the baseline label, extract the saliency and uniqueness index of the texture features of each candidate region; Candidate regions whose texture feature saliency and uniqueness index meet preset conditions are determined as preset local feature templates; The preset local feature template includes at least one of a specific mark area, a specific image area, or a barcode area on the reference label.

[0152] In one alternative implementation, this embodiment pre-defines the reference label boundary using a fixed size (e.g., set to 64). A two-dimensional sliding window (64 pixels or 1288-128 pixels) is used to perform a global sliding traversal with a specific step size. Each local image pixel block captured by the sliding is used as an independent image unit, thereby extracting all image pixel blocks generated during the traversal to form the candidate regions.

[0153] For each extracted candidate region, this embodiment will further call the image information entropy algorithm or gradient magnitude statistics algorithm, such as the Laplacian operator energy evaluation algorithm.

[0154] Furthermore, in this embodiment, the variance or gradient sum of the pixel gray values ​​in each candidate region is calculated, and the industrial control computer uses the calculated value that characterizes the richness of high-frequency details in the image as the saliency of the texture features.

[0155] Meanwhile, to prevent multiple false matches during online matching, the Normalized Cross Correlation (NCC) algorithm can be used. The current candidate region is used as a self-addressable sub-template, and cross-matching calculations are performed on the remaining non-overlapping regions within the baseline label boundary. After extracting the second-highest matching score, the reciprocal of this score, or the difference between the highest and second-highest matching scores, is used as a unique indicator of how easily the current candidate region is confused with the surrounding background.

[0156] In the actual visual characteristics of industrial labels, after the above-mentioned saliency and uniqueness calculations and screening, the final determined preset local feature templates usually correspond to physical anti-counterfeiting or identification structures with extremely high contrast and strong uniqueness on the label surface. Specifically, the selected preset local feature templates are usually manifested as the specific logo area containing the brand's exclusive logo pattern, the specific image area containing complex and non-reusable illustrations or warning icons, or the barcode area containing dense, high-frequency black and white stripes with intersecting boundaries (such as a corner of a one-dimensional barcode or a specific positioning black frame mark of a QR code), etc.

[0157] This embodiment can automatically perform quantitative screening and investigation of all available pixel blocks within the boundary of the baseline label in the offline stage. The preset condition judgment mechanism, which uses mathematical weighting and deep fusion of texture feature saliency and uniqueness indicators, not only completely eliminates the subjective blindness and experience dependence brought about by manual image selection, but also ensures from the underlying mathematical principles that the extracted preset local feature template must be the region containing the richest edge gradients and absolutely no similar or repetitive textures in the entire label. By using such specific marker regions, specific image regions, or barcode regions with high-frequency unique features as positioning benchmarks, the risk of mismatched local position coordinates caused by similar text layouts or repeated printed patterns on the label during the subsequent online real-time initial matching stage from local to overall is effectively eliminated. This provides accurate image feature anchors for sub-pixel-level capture and positioning of visual defect detection systems on high-speed production lines.

[0158] Figure 4 This is a schematic diagram of the label defect detection device provided by the present invention, as shown below. Figure 4 As shown, it mainly includes, but is not limited to: Edge image acquisition module 41 is used to acquire the template edge binarized image of the reference label and the test edge binarized image of the label to be tested, wherein the test edge binarized image and the template edge binarized image have been registered and aligned. The first defect extraction module 42 is used to perform morphological dilation processing on the template edge binarized image and find the intersection with the test edge binarized image to obtain a first tolerance intersection image; and to perform difference operation on the test edge binarized image and the first tolerance intersection image to obtain a first defect region characterizing the redundant features of the test label. The second defect extraction module 43 is used to perform morphological dilation processing on the binarized image of the edge to be tested and to find the intersection with the binarized image of the template edge to obtain a second tolerance intersection image; and to perform a difference operation on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region that represents the missing features of the label to be tested. The result determination module 44 is used to determine the defect detection result of the label to be tested based on the first defect area and the second defect area.

[0159] It should be noted that the label defect detection device provided in this embodiment can be used to perform the label defect detection device provided in any of the above embodiments, and will not be described in detail here.

[0160] Figure 5 This is a schematic diagram of the operation of the label defect detection device provided by the present invention, as shown below. Figure 5 As shown, after the detection process enters the start node, the edge image acquisition module 41 first reads the global reference image of the standard qualified product from the system memory, and simultaneously drives the industrial camera to acquire the global image to be tested, including the production line background, in real time. In order to quickly remove the background in the huge field of view, the edge image acquisition module 41 inputs the global reference image and the global image to be tested into the coarse matching processing logic.

[0161] In practice, a fast template matching function based on the normalized cross-correlation algorithm is called to search for multiple local anchor points in the global image to be tested, and the underlying tensor slicing operation is performed according to the calculated preset coordinate offset relationship, thereby generating the first initial image to be tested, the second initial image to be tested, and so on up to the nth initial image to be tested in parallel in memory.

[0162] Subsequently, the edge image acquisition module 41 will perform edge-preserving filtering and Sobel edge gradient extraction calculation on the global reference image (or its cropped portion) and the parallel initial test images generated above, thereby generating a template edge feature map containing only high-frequency information of geometric structure, and n test edge feature maps corresponding one-to-one with the n initial test images.

[0163] Next, the edge image acquisition module 41 enters the deep learning matching stage, calling a pre-deployed deep feature matching model (e.g., the LightGlue deep learning network based on a self-attention mechanism) to extract a set of highly robust geometric feature point pairs between the template edge feature map and each test edge feature map, and calculates n independent homography transformation matrices. Using these homography transformation matrices, image correction is performed on each of the n test edge feature maps, so that each image-corrected test edge feature map is perfectly aligned with the template edge feature map in the physical space coordinate system.

[0164] Finally, the edge image acquisition module 41 performs adaptive binarization processing to generate the template edge binarized image by performing adaptive binarization on the template edge feature map; simultaneously, adaptive binarization is performed in parallel on each edge feature map to be tested after image correction, thereby completely transforming the feature map with continuous gradient into n edge binarized images to be tested. At this time, the edge binarized images to be tested and the template edge binarized images have been registered and aligned.

[0165] Furthermore, in the first defect extraction module 42, a specific size (such as 3) is called. 3 or 5 A morphological structuring element kernel (a 5-matrix) is used to expand the edge contours outward in the template edge binarized image at the pixel level, i.e., to perform morphological dilation. Then, the dilated template edge binarized image is subjected to a bitwise logical AND operation with the currently compared test edge binarized image to obtain the intersection, thereby accurately outputting the first tolerance intersection image.

[0166] Finally, the first defect extraction module 42 will perform a bitwise logical difference operation (i.e., subtraction operation) on the original binary image of the edge to be tested and the first tolerance intersection image to completely remove abnormal pixels that exceed the standard contour on the label to be tested, and successfully obtain the first defect area that represents redundant features such as ink splatter and black spots.

[0167] Furthermore, under the same edge difference logic branch, the second defect extraction module 43 performs reverse topological difference, including: using the same morphological structuring element kernel, performing pixel-level expansion of the edge contour in the current test edge binarization image to perform morphological dilation processing. Next, the dilated test edge binarization image and the standard template edge binarization image are subjected to a bitwise logical AND operation to generate the second tolerance intersection image. Subsequently, the template edge binarization image and the second tolerance intersection image are subjected to a bitwise logical difference operation to extract pixels that are present on the template but have disappeared on the test image due to broken lines, missing strokes, etc., successfully obtaining the second defect region representing the missing features of the test label.

[0168] Finally, the result determination module 44 performs connected component extraction and calculates the defect area. This includes: calling a connected component labeling algorithm to cluster the scattered differential foreground pixels in the first and second defect regions, encapsulating them into independent connected component objects with topological connectivity. Subsequently, the pixels within each connected component are traversed to count the absolute number of pixels, thus completing the calculation of the defect area.

[0169] Furthermore, the result determination module 44 compares the calculated defect area values ​​of each connected region with a preset area threshold that is acceptable for industrial production. It rigorously filters out tiny noise interference regions with area values ​​less than or equal to the preset area threshold, and formally identifies connected regions with area values ​​greater than the preset area threshold as target defect areas. Based on the final retained target defect areas, it generates the defect detection result, which includes defect coordinates and acceptance / acceptance status. It outputs a detection signal to guide the robot arm's actions on the production line or to trigger a screen highlight alarm, ultimately progressing to the end of the process.

[0170] The label defect detection device provided in this embodiment effectively accommodates local deformation caused by label wrinkles and positional offset of edge pixels through a bidirectional expansion tolerance differential mechanism, solving the problem of a large number of false alarms caused by local deformation and significantly reducing the false alarm rate while ensuring a high defect detection rate.

[0171] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logic instructions in the memory 630 to execute a label defect detection method. This method includes: acquiring a template edge binarized image of a reference label and a test edge binarized image of a label to be tested, wherein the test edge binarized image and the template edge binarized image have been registered and aligned; performing morphological dilation on the template edge binarized image and finding its intersection with the test edge binarized image to obtain a first tolerance intersection image; performing a difference operation on the test edge binarized image and the first tolerance intersection image to obtain a first defect region representing redundant features of the label to be tested; performing morphological dilation on the test edge binarized image and finding its intersection with the template edge binarized image to obtain a second tolerance intersection image; performing a difference operation on the template edge binarized image and the second tolerance intersection image to obtain a second defect region representing missing features of the label to be tested; and determining the defect detection result of the label to be tested based on the first defect region and the second defect region.

[0172] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the label defect detection method provided in the above embodiments, the method comprising: acquiring a template edge binarized image of a reference label and a test edge binarized image of a label to be tested, wherein the test edge binarized image and the template edge binarized image have been registered and aligned; performing morphological dilation processing on the template edge binarized image and then aligning it with the test edge binarized image. The intersection of the binarized images of the edge to be tested is obtained to obtain a first tolerance intersection image; the difference operation is performed between the binarized images of the edge to be tested and the first tolerance intersection image to obtain a first defect region representing redundant features of the label to be tested; the morphological dilation processing of the binarized images of the edge to be tested is performed and the intersection with the binarized images of the template edge is obtained to obtain a second tolerance intersection image; the difference operation is performed between the binarized images of the template edge and the second tolerance intersection image to obtain a second defect region representing missing features of the label to be tested; based on the first defect region and the second defect region, the defect detection result of the label to be tested is determined.

[0174] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the label defect detection method provided in the above embodiments. The method includes: acquiring a template edge binarized image of a reference label and a test edge binarized image of a label to be tested, wherein the test edge binarized image and the template edge binarized image have been registered and aligned; performing morphological dilation processing on the template edge binarized image and then finding its intersection with the test edge binarized image to obtain a first tolerance. Intersection map; perform difference operation on the binarized image of the edge to be tested and the first tolerance intersection map to obtain a first defect region representing redundant features of the label to be tested; perform morphological dilation processing on the binarized image of the edge to be tested and find its intersection with the binarized image of the template edge to obtain a second tolerance intersection map; perform difference operation on the binarized image of the template edge and the second tolerance intersection map to obtain a second defect region representing missing features of the label to be tested; determine the defect detection result of the label to be tested based on the first defect region and the second defect region.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting label defects, characterized in that, include: Obtain the template edge binarized image of the reference label and the test edge binarized image of the label to be tested. The test edge binarized image and the template edge binarized image have been registered and aligned. After performing morphological dilation on the template edge binarized image, the intersection of the template edge binarized image and the edge binarized image to be tested is obtained to obtain a first tolerance intersection image; the difference operation is performed between the edge binarized image to be tested and the first tolerance intersection image to obtain a first defect region characterizing the redundant features of the label to be tested; After performing morphological dilation on the binarized image of the edge to be tested, the intersection of the binarized image of the template edge is obtained to get a second tolerance intersection image; the difference operation is performed on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region characterizing the missing features of the label to be tested; Based on the first defect area and the second defect area, the defect detection result of the label to be tested is determined.

2. The label defect detection method according to claim 1, characterized in that, The process of obtaining the template edge binarized image of the reference label and the test edge binarized image of the label to be tested includes: Obtain the template edge feature map of the reference label and the test edge feature map of the label to be tested; Extract the geometric structural features that match each other between the template edge feature map and the edge feature map to be tested, so as to calculate the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map; Based on the spatial transformation mapping relationship, the edge feature map to be tested is transformed in spatial coordinates so that the edge feature map to be tested and the template edge feature map are registered and aligned. Threshold segmentation is performed on the registered and aligned template edge feature map and the edge feature map to be tested, respectively, to generate the template edge binarization map and the edge binarization map to be tested.

3. The label defect detection method according to claim 2, characterized in that, The step of extracting the geometric structural features that match the template edge feature map and the edge feature map to be tested, and calculating the spatial transformation mapping relationship between the edge feature map to be tested and the template edge feature map, includes: Feature points are extracted from the template edge feature map and the edge feature map to be tested; The feature points of the extracted template edge feature map are matched with the feature points of the edge feature map to be tested to obtain a set of feature point pairs; Based on the coordinate correspondence in the set of feature points, a transformation matrix is ​​calculated to characterize the spatial transformation mapping relationship.

4. The label defect detection method according to claim 3, characterized in that, The step involves extracting feature points from the template edge feature map and the edge feature map to be tested, and matching the extracted feature points from the template edge feature map with the feature points from the edge feature map to be tested to obtain a set of feature point pairs, including: The template edge feature map and the edge feature map to be tested are input into the feature matching model to obtain the set of feature point pairs output by the feature matching model; The feature matching model is trained based on multiple edge feature image pairs and the true feature matching relationship labels corresponding to each edge feature image pair. The transformation matrix is ​​a homography transformation matrix.

5. The label defect detection method according to claim 2, characterized in that, The step of obtaining the template edge feature map of the reference label and the test edge feature map of the label to be tested includes: Obtain the initial template image of the reference label and the initial image to be tested of the label to be tested; Edge-preserving filtering is performed on the initial template image and the initial test image respectively to obtain a template-filtered image and a test-filtered image; The edge gradient intensity values ​​of each pixel in the template filtered image and the test filtered image are calculated respectively to generate the template edge feature map and the test edge feature map containing image geometric structure information.

6. The label defect detection method according to claim 5, characterized in that, The edge-preserving filtering process includes bilateral filtering or non-local mean filtering. The step of calculating the edge gradient intensity value of each pixel in the template filtered image and the test filtered image respectively includes: The Sobel edge operator is used to extract the edge gradient intensity values ​​of the template filtered image and the filter image to be tested, respectively.

7. The label defect detection method according to claim 5, characterized in that, Obtaining the initial image of the label to be tested includes: Obtain a global image of the target image containing the label to be tested; In the global image to be tested, determine the coordinates of local anchor points that match the preset local feature template; Based on the local anchor point coordinates and the preset coordinate offset relationship, the complete label boundary of the label to be tested in the global image to be tested is determined; Based on the complete label boundary, the initial image of the label to be tested is cropped from the global image to be tested.

8. The label defect detection method according to claim 7, characterized in that, Before acquiring the global image of the target image containing the target label, the method further includes: Obtain the global reference image of the reference label; In the global reference image, a local texture region is selected as the preset local feature template, and the reference label boundary of the reference label is defined. Calculate the positional offset between the preset local feature template and the reference label boundary, and establish the preset coordinate offset relationship.

9. The label defect detection method according to claim 8, characterized in that, The step of determining the defect detection result of the label under test based on the first defect region and the second defect region includes: Obtain each connected component in the first defect region and the second defect region; Calculate the area value of each of the connected components; The connected regions whose area values ​​are greater than a preset area threshold are determined as the target defect areas of the label to be tested. Based on all the target defect areas, the defect detection results of the label to be tested are generated.

10. The label defect detection method according to claim 9, characterized in that, The step of generating defect detection results for the label under test based on all the target defect regions includes: Obtain the defect pixel coordinates of the target defect region in the binarized image of the edge to be tested; Based on the inverse transformation of the spatial transformation mapping relationship, the defect pixel coordinates are inversely mapped to the local defect coordinates corresponding to the initial image to be tested; Based on the preset coordinate offset relationship, the local defect coordinates are reverse-mapped back to the global image under test to obtain the global defect coordinates of the label under test in the global image under test. The information containing the global defect coordinates will be used as the defect detection result of the label to be tested.

11. The label defect detection method according to claim 1, characterized in that, The morphological dilation processing of the template edge binarized image and the test edge binarized image includes: The edge contours in the template edge binarization image and the edge contours in the test edge binarization image are respectively expanded outward by a preset number of pixels.

12. The label defect detection method according to claim 8, characterized in that, The step of selecting a local texture region in the global reference image as the preset local feature template includes: Within the boundary of the baseline label, extract the saliency and uniqueness index of the texture features of each candidate region; Candidate regions whose texture feature saliency and uniqueness index meet preset conditions are determined as preset local feature templates; The preset local feature template includes at least one of a specific mark area, a specific image area, or a barcode area on the reference label.

13. A label defect detection device, characterized in that, include: The edge image acquisition module is used to acquire the template edge binarized image of the reference label and the test edge binarized image of the label to be tested. The test edge binarized image and the template edge binarized image have been registered and aligned. The first defect extraction module is used to perform morphological dilation processing on the template edge binarized image and then find the intersection with the test edge binarized image to obtain a first tolerance intersection image; and to perform a difference operation on the test edge binarized image and the first tolerance intersection image to obtain a first defect region characterizing the redundant features of the test label. The second defect extraction module is used to perform morphological dilation processing on the binarized image of the edge to be tested and then find the intersection with the binarized image of the template edge to obtain a second tolerance intersection image; and to perform a difference operation on the binarized image of the template edge and the second tolerance intersection image to obtain a second defect region that represents the missing features of the label to be tested. The result determination module is used to determine the defect detection result of the label to be tested based on the first defect area and the second defect area.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the label defect detection method as described in any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the label defect detection method as described in any one of claims 1 to 12.

Images