Labeling machine identification quality detection method and system of AI vision technology

Abstract

The present application relates to the technical field of image defect detection, and particularly relates to a labeling machine identification quality detection method and system based on AI vision technology. Firstly, an identification area is extracted; further, according to the similarity between the identification area and a standard identification area, effective edge points are screened out, different areas are transformed into the same coordinate system, and a suspected defect area is obtained; further, according to the similarity of the suspected defect area in adjacent frame images, a matching index is obtained; further, according to the distribution of the matching index and the gray level change in the suspected defect area, a region to be analyzed and a defect region are divided; further, according to the similarity of the region to be analyzed of different articles in the same batch, combined with the fluctuation of the matching index and the gray level change, the defect region is screened out; finally, according to the number of effective edge points and the area of the defect region, a labeling effect evaluation value is obtained, the labeling quality is quantified, and the detection precision and stability are improved.

Description

Technical Field

[0001] The present invention relates to the field of image defect detection technology, and in particular to a labeling machine marking quality detection method and system using AI vision technology. Background Art

[0002] At present, when visual inspection technology is used to perform quality inspection on the labels of labeling machines in industrial inspection, a region of interest is usually set according to the location of the label of the item to be tested. The region is extracted from the acquired image and the image quality is analyzed using machine vision methods such as edge detection and template matching. The defective label is then output and the corresponding product is eliminated.

[0003] If the product labeling surface is an uneven surface such as a curved surface or a special shape, the label edge is prone to wrinkles or tensile deformation at the corners and arcs. Among them, if the curvature of the bonding surface is different, the degree of impact on the label detection is also different (for example, when the label is small and the curvature of the bonding surface is not obvious, the impact on the label quality is small), resulting in false detection of real wrinkle defects and tensile deformation. In addition, since the conveyor belt may shake during the transportation of items, the labeling position of the item may be displaced or rotated to a certain extent, resulting in inaccurate detection of label quality. Summary of the Invention

[0004] In order to solve the technical problem that the accuracy of labeling detection is affected by the shape of the product labeling surface and the transmission deviation, the purpose of the present invention is to provide a labeling machine mark quality detection method and system using AI vision technology. The technical solution adopted is as follows:

[0005] A method for detecting the marking quality of a labeling machine using AI vision technology, the method comprising:

[0006] Obtain continuous frame images of each object to be tested on the conveyor belt and extract the identification area in the image;

[0007] Filter out valid edge points based on edge similarity between the identified area and a preset standard identified area; transform different areas into the same coordinate system based on matching features between the valid edge points and the edge points of the standard identified area; analyze the grayscale difference between the identified area and the standard identified area in the same coordinate system to obtain a suspected defect area; obtain a matching index based on the similarity of the morphology and position of the suspected defect area in adjacent frame images; and divide the area to be analyzed and the defect area based on the distribution of all the matching indices corresponding to the same object to be tested and the grayscale change within the suspected defect area;

[0008] Based on the similarity of the areas to be analyzed of different items in the same batch, combined with the fluctuation of the matching index and the grayscale change of the same item, the defective areas are screened out; based on the number of all the valid edge points of each item to be tested and the area of ​​the defective area, the labeling effect evaluation value is obtained.

[0009] Furthermore, the method for obtaining effective edge points includes:

[0010] Extract edge points based on edge detection; select edge points of the marked area and edge points of the standard marked area one by one to form edge binary groups; in each edge binary group, obtain matching validity based on the similarity of vectors pointing from the center point of the area to the edge point, combined with the gradient amplitude similarity and curvature similarity between the edge points;

[0011] In the identified area, edge points whose matching validity is greater than a preset validity threshold are marked as valid edge points.

[0012] Furthermore, the method of transforming different regions into the same coordinate system includes:

[0013] For each of the identified areas, the valid edge points are matched with the edge points corresponding to the maximum matching validity, and a transformation matrix is ​​obtained using RANSAC based on the matching results. The identified area is transformed to the coordinate system of the standard identified area using the transformation matrix.

[0014] Furthermore, the method for obtaining the suspected defect area includes:

[0015] In the same coordinate system, suspected defect area points are screened out based on the prominent features of the grayscale difference between the effective edge points in the identification area and the alignment points in the standard identification area; a morphological closing operation is performed on the connected domain where all the suspected defect area points in the identification area are located to obtain the suspected defect area.

[0016] Furthermore, the method for obtaining the matching index includes:

[0017] In two adjacent frames of images of the same object to be inspected, the matching index of the suspected defect area in the corresponding two adjacent frames is obtained based on the similarity between the vectors between the center points of the two suspected defect areas in the coordinate system and the direction of movement of the conveyor belt, combined with the difference between the distance between the centers and the aspect ratio of the minimum circumscribed rectangle.

[0018] Furthermore, the method of dividing the area to be analyzed and the defect area includes:

[0019] Obtaining the inherent damage possibility of each object to be tested based on the average of all the matching indexes corresponding to each object to be tested and the range between the grayscale means in each suspected defect area;

[0020] The suspected defect area of ​​the test object whose inherent damage possibility is greater than the preset damage possibility is divided into a defect area; the suspected defect area of ​​the test object whose inherent damage possibility is less than or equal to the preset damage possibility is divided into an area to be analyzed.

[0021] Furthermore, the method for screening out defective areas includes:

[0022] For each article to be tested having the area to be analyzed, obtaining the range of the first-order difference values ​​of the time series of the matching index as the matching fluctuation factor; obtaining the absolute value of the average value of the first-order difference values ​​of the time series of the grayscale mean of the suspected defect area as the grayscale fluctuation factor;

[0023] In the coordinate system, the average distance between each area to be analyzed and the center point of all other areas to be analyzed is obtained as a search radius. Within the search radius of each area to be analyzed, the areas to be analyzed of other objects to be tested whose area difference ratio is less than a preset area difference threshold are screened out and matched;

[0024] According to the number of regions matched by each of the regions to be analyzed within the search radius, the defective regions are screened out in combination with the matching fluctuation factor and the grayscale fluctuation factor.

[0025] Furthermore, the method for obtaining the labeling effect evaluation value includes:

[0026] A labeling effect evaluation value is obtained according to the average value of the effective edge points in all the identification areas of each object to be tested, combined with the average value of the number of pixels in all the defective areas.

[0027] Furthermore, the method for obtaining the identification area includes:

[0028] Semantic recognition is used to extract the initial identification area in the image, and the initial identification area whose area is larger than the average area of ​​the identification areas in all images of the corresponding object to be tested is screened out as the identification area.

[0029] The present invention also proposes a labeling machine identification quality detection system using AI vision technology, the system including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements any one of the steps of the labeling machine identification quality detection method using AI vision technology.

[0030] The present invention has the following beneficial effects:

[0031] First, the identification area is extracted to avoid the influence of irrelevant and invalid areas, providing a basis for subsequent analysis; further, based on the similarity between the identification area and the standard identification area, the effective edge points are screened out to prepare for the subsequent transformation of the coordinate system and the evaluation of the labeling effect; the different areas are further transformed into the same coordinate system to reduce the displacement and rotation of the labeling position caused by the transmission of the items, which is convenient for comparison between areas; the grayscale anomalies are further compared to obtain the suspected defect area, preparing for the subsequent screening of the defect area; the matching index is further obtained to quantify the similarity of the suspected defect area in adjacent frame images, providing a basis for the preliminary distinction of the defect area; the area to be analyzed and the defect area are further divided from the distribution of all matching indexes of the same item to be tested and the grayscale change angle in the suspected defect area, and the defect area is preliminarily screened out; the similarity of the areas to be analyzed of different items in the same batch is further used to screen the defect area again from the area to be analyzed; finally, the labeling effect evaluation value is obtained according to the number of effective edge points and the area area of ​​the defect area, and the labeling quality of the labeling machine is quantified. To address the detection error problem caused by labeling surface deformation and transmission offset, the present invention extracts defect areas through multi-frame alignment, edge matching and grayscale analysis, combines matching fluctuations and batch statistics to eliminate pseudo defects, quantify labeling quality, and improve detection accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 A flowchart of a labeling machine marking quality detection method using AI vision technology provided by one embodiment of the present invention;

[0034] Figure 2 This is a simulation diagram of a labeling machine in an industrial scenario provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0035] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a labeling machine marking quality detection method and system based on AI vision technology proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable form.

[0036] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0037] The following describes in detail a method and system for detecting the quality of labeling marks on a labeler using AI vision technology provided by the present invention with reference to the accompanying drawings.

[0038] See also Figure 1 , which shows a flow chart of a labeling machine mark quality detection method using AI vision technology provided by one embodiment of the present invention, specifically including:

[0039] Step S1: Acquire continuous frame images of each object to be tested on the conveyor belt and extract the identification area in the image.

[0040] In actual industrial testing, it is usually necessary to test large quantities of products. The testing equipment is usually installed on both sides of the conveyor belt, and cooperates with the conveyor belt to obtain the marking quality of each product. Figure 2 , which shows a simulation diagram of a labeling machine in an industrial scenario provided by an embodiment of the present invention, and includes a model body of the labeling machine.

[0041] A camera is installed at a fixed position above the conveyor belt and the light source is stabilized. The objects to be tested are arranged in sequence and move with the movement of the conveyor belt. The objects to be tested gradually enter the camera shooting range and gradually leave the camera screen as their position changes. The conveyor belt is set to move at a constant speed, and the camera shooting frequency is set to once every 0.5 seconds. Continuous frame images of each object to be tested on the conveyor belt are obtained.

[0042] Considering that the captured image contains other content outside the identification area of ​​the label, and as the position of the object to be tested changes, the identification area in the captured image may be incomplete, it is necessary to extract the identification area in the image.

[0043] Preferably, in one embodiment of the present invention, semantic recognition (such as UNet) is used to extract the initial identification area in the image, and the holes in the obtained identification area are filled using morphological closing operations; at the same time, when the object to be tested gradually moves into or out of the camera shooting range, the incomplete acquisition of the identification area is avoided to affect the subsequent analysis, and the initial identification area whose area is larger than the average area of ​​the identification areas in all images corresponding to the object to be tested is screened out as the identification area, and the image is also screened.

[0044] It should be noted that, in one embodiment of the present invention, the area of ​​a region is represented by the number of pixels in the region. In other embodiments of the present invention, the implementer can adjust the image acquisition frequency on his own. Semantic recognition and morphological closing operations are already existing technologies and will not be described in detail. The analysis process for each object to be tested and the analysis process for each identified area therein are the same. Only one example will be described here and will not be repeated.

[0045] Step S2: Filter out valid edge points based on the edge similarity between the identification area and the preset standard identification area; transform different areas into the same coordinate system based on the matching features between the valid edge points and the edge points of the standard identification area; analyze the grayscale difference between the identification area and the standard identification area in the same coordinate system to obtain the suspected defect area; obtain the matching index based on the similarity of the shape and position of the suspected defect area in adjacent frame images; divide the area to be analyzed and the defect area according to the distribution of all matching indices corresponding to the same object to be tested and the grayscale change in the suspected defect area.

[0046] Because the texture (e.g., pattern outlines, text parts, etc.) and geometric features of the logo area in the image have strong edge properties, the edge points of the logo area are analyzed. Considering that labels are industrially produced products with high consistency, we can prepare standard products labeled by a labeling machine in advance, capture the image at a normal angle on the conveyor belt, and extract the logo area to serve as the preset standard logo area.

[0047] At this time, based on the edge similarity between the logo area and the preset standard logo area, the degree to which the edge points of the logo area effectively represent the logo texture can be analyzed, thereby screening out valid edge points to prepare for the subsequent transformation of the coordinate system and evaluation of the labeling effect.

[0048] Preferably, in one embodiment of the present invention, edge points are extracted based on edge detection; in order to analyze edge similarity between different regions, edge points of the marked region and edge points of the standard marked region are selected one by one to form edge tuples;

[0049] In each edge binary, considering that the labeling process may cause local perturbations of the edge (such as slight stretching or bending), the overall edge trend, gradient characteristics, and structural morphology of the label still maintain a certain consistency, the matching degree of the edge points can be evaluated by three dimensions: vector direction, gradient amplitude, and curvature. Therefore, the matching validity is obtained based on the similarity of the vector pointing from the center point of the region to the edge point, combined with the gradient amplitude similarity and curvature similarity between the edge points;

[0050] In the identified area, edge points whose matching validity is greater than a preset validity threshold are marked as valid edge points.

[0051] As an example, edge points are extracted using the Canny operator, with a preset effective threshold of 0.7. Based directly on the prominence of the marker region and the standard marker region, coordinate systems are constructed for each, with the centroid of the region as the region center. For each edge binary in any marker region, a vector is obtained pointing from the region center (starting point) to the edge point (end point). The sum of the cosine similarity of the vectors corresponding to the two edge points and a constant of 1 is used as the numerator, and the product of the absolute value of the difference in the gradient amplitude at the two edge points and the absolute value of the difference in the curvature, plus a preset positive division parameter of 0.1, is used as the denominator. The linear normalization result of the fractional ratio in the data dimension corresponding to the edge binary of all images is used as the matching validity.

[0052] Among them, the cosine similarity is used to express the similarity of the vector pointing from the center point of the region to the edge point, which represents the similarity of the edge trend; the absolute value of the difference in gradient amplitude is used to express the similarity of gradient features, and the absolute value of the difference in curvature represents the similarity of structural morphology. The three dimensions are combined to show the edge similarity between the marked area and the standard marked area.

[0053] It should be noted that the Canny operator, cosine similarity, the gradient amplitude of the pixel point, and the method for obtaining the curvature of each point on the edge line are all existing technologies and will not be described in detail.

[0054] In order to reduce the displacement and rotation of the labeling position caused by the transported items, different areas are transformed into the same coordinate system based on the standard identification area; considering that when screening valid edge points and analyzing edge similarity, the similarity between the valid edge points and the edge points of the standard identification area is analyzed, the similarity between different edge points is different and the matching degree is different, so based on the matching characteristics of the valid edge points and the edge points of the standard identification area, different areas are transformed into the same coordinate system.

[0055] Preferably, in one embodiment of the present invention, for each identification area, the valid edge points are matched with the edge points of the standard identification area corresponding to the maximum matching validity, and the transformation matrix is ​​obtained by using RANSAC based on the matching results of each identification area and the standard identification area, and the identification area is transformed to the coordinate system of the standard identification area using the transformation matrix, so that all the identification areas of the items to be tested are placed in the same coordinate system.

[0056] It should be noted that using RANSAC to obtain a transformation matrix and using the transformation matrix to perform transformation is already an existing technology and will not be described in detail.

[0057] In the same coordinate system, the identification area and the standard identification area are aligned pixel by pixel. Therefore, local grayscale anomalies can be accurately identified by comparing the grayscale values ​​pixel by pixel. The grayscale difference between the identification area and the standard identification area is analyzed to obtain the suspected defect area, preparing for the subsequent screening of defect areas.

[0058] Preferably, in one embodiment of the present invention, when the coordinate system of the marking area is transformed, pixel-by-pixel alignment of the marking area and the standard marking area is achieved. In the same coordinate system, the greater the grayscale difference between the effective edge point and the alignment point in the standard marking area, the more likely it is a defect point. Therefore, based on the prominent features of the grayscale difference between the effective edge point of the marking area and the alignment point in the standard marking area, suspected defect area points are screened out.

[0059] As an example, the grayscale difference is represented by the absolute value of the grayscale value difference. The average of the absolute values ​​of the grayscale value differences between each mark area and all the alignment points in the standard mark area is obtained as the difference reference value of each mark area. When the absolute value of the grayscale value difference between the valid edge point and the alignment point in the standard mark area is greater than the corresponding difference reference value, the valid edge point is determined to be a suspected defect area point.

[0060] Furthermore, a morphological closing operation is performed on the connected domains where all suspected defect area points are located in the marked area to obtain the suspected defect area.

[0061] During the labeling process, inherent defects such as stains or damage on the label itself may cause defective areas to appear in the acquired image. Variable defects may also exist, such as wrinkles caused by local squeezing or relaxation due to the curvature of the labeling surface or the label not being perpendicular to the labeling surface.

[0062] Among them, inherent damage is the physical damage of the mark, and its shape does not change with the change of the camera shooting angle. However, variable defects such as wrinkles have raised peaks, etc., and when the camera angle changes, the shape and light and shadow characteristics of the corresponding area also change accordingly.

[0063] Therefore, it is necessary to analyze the changes in the suspected defect area in the image over time. Since the object to be tested moves forward with the conveyor belt, it gradually enters the shooting range of the camera. That is, under normal circumstances, the shape and position of the same suspected defect area in the images at adjacent moments are similar. Therefore, based on the similarity of the shape and position of the suspected defect area in adjacent frame images, the matching index is obtained to quantify the degree of similarity of the suspected defect area in adjacent frame images; based on the distribution of all matching indices corresponding to the same object to be tested and the grayscale changes in the suspected defect area, the area to be analyzed and the defect area are divided.

[0064] Preferably, in one embodiment of the present invention, in two adjacent frames of images of the same object to be inspected, the more similar the vector between the centers of the two suspected defect areas is to the moving direction of the conveyor belt, the more consistent it is with the moving characteristics of the object to be inspected on the conveyor belt, and the higher the matching index; at the same time, the smaller the distance between the area centers, the higher the position similarity, and the higher the matching index; the smaller the difference between the aspect ratios of the minimum circumscribed rectangles, the higher the morphological similarity, and the higher the matching index;

[0065] Based on this, the matching index of the suspected defect area in the corresponding two adjacent frames is obtained according to the similarity between the vector between the centers of the two suspected defect areas in the coordinate system and the moving direction of the conveyor belt, combined with the difference between the distance between the area centers and the aspect ratio of the minimum circumscribed rectangle.

[0066] As an example, the similarity between vectors is expressed by cosine similarity. In two adjacent frames of the same object to be inspected, the vector between the centers of two suspected defective areas in the coordinate system, the cosine similarity of the conveyor belt moving direction, and the constant 1 are used as the numerator; the product of the Euclidean distance between the centers of the two areas and the absolute value of the difference between the aspect ratios of the minimum circumscribed rectangles of the two areas, plus a preset positive division parameter of 0.1, is used as the denominator, the fractional ratio is used as the matching factor, and the linear normalization result of the matching factor in the data dimensions corresponding to all images is used as the matching index.

[0067] Among them, two adjacent frame images refer to two frame images with suspected defect areas and adjacent in time sequence, the starting point of the vector between the centers corresponds to the center point with the smallest time sequence, and the constant 1 is used to correct the value range of the cosine similarity.

[0068] In another embodiment of the present invention, considering that the higher the similarity of the edge points of two suspected defect areas, the higher the edge similarity, the greater the matching index between the edge points at this time, so the matching validity of the edge binary groups in two adjacent frames of images is obtained, and the matching edge points are screened out. At this time, the more matching edge points screened out, the more similar the edges are, and the higher the matching index is, so the product of the matching factor and the number of matching edge points is linearly normalized in the data dimension of the same product corresponding to all images as the matching index.

[0069] The process of screening matching edge points is the same as that of screening valid edge points. To facilitate distinction, they are given different names.

[0070] In other embodiments of the present invention, considering that there are multiple defects in the identification area in the image of the object to be tested, and in order to more accurately compare the performance of the same defect in different images, the suspected defect area can also be matched using the matching index after calculating the matching index. Specifically, the matching threshold is set to 0.7, and two suspected defect areas with a matching index greater than the matching threshold are matched, and the matching relationship is transitive. For example, A matches B, and B matches C, then A, B, and C are finally matched, so that the suspected defect areas in different images are matched one by one in the chronological order starting from the first frame image of the time sequence.

[0071] Among them, when there are multiple suspected defect areas that meet the matching threshold in adjacent frame images for a suspected defect area, the suspected defect area corresponding to the maximum matching index is selected for matching; when the defect area is subsequently obtained, each matching result is analyzed separately.

[0072] Preferably, in one embodiment of the present invention, if a suspected defective area has a small position change and a small morphological difference while moving along the conveyor belt, and the fluctuation of the grayscale distribution of the pixel points is small, then the area is more likely to be inherent damage to the label mark;

[0073] Considering that the higher the mean of all matching indices corresponding to the object to be tested, the more stable the shape and position of the suspected defect area in multiple time-series images, that is, the area has strong temporal consistency, and the distribution of all matching indices is represented by the mean of the matching index; the smaller the range of the grayscale mean, the smaller the change in grayscale information such as illumination and material reflection in different frame images, and the more stable the defect characteristics, which may be caused by the inherent damage of the mark itself. The grayscale change in the suspected defect area is represented by the range of the grayscale mean;

[0074] Based on this, the inherent damage probability of each object under test is obtained based on the mean value of all matching indices corresponding to each object under test and the range between the grayscale means in each suspected defect area;

[0075] The suspected defect area of ​​the test object with an inherent damage possibility greater than the preset damage possibility is divided into the defect area; the suspected defect area of ​​the test object with an inherent damage possibility less than or equal to the preset damage possibility is divided into the area to be analyzed.

[0076] As an example, the mean of all matching indices corresponding to each test object is used as the numerator, and the range between the grayscale means in each suspected defect area is used as the denominator. The fractional ratio is linearly normalized across the data dimensions corresponding to all test objects in the same batch to obtain the inherent damage probability of each test object.

[0077] The preset damage probability is set to 0.7, and the defect area and the area to be analyzed are divided. At this time, the defect area divided corresponds to the inherent damage area, and the area to be analyzed corresponds to the suspected variable defect area.

[0078] It should be noted that, in one embodiment of the present invention, the article to be tested contains only one label, and a batch of articles to be tested and the label are the same. In other embodiments of the present invention, if there are multiple labels, they need to be classified during semantic recognition, and each type of identification area is analyzed separately.

[0079] Step S3: Based on the similarity of the areas to be analyzed of different items in the same batch, combined with the fluctuation of the matching index and the grayscale change of the same item, the defective areas are screened out; based on the number of all valid edge points of each item to be tested and the area of ​​the defective area, the labeling effect evaluation value is obtained.

[0080] Since the labeling surface may have curvature or be a special-shaped surface, the local marking area may be pseudo-stretched when the camera's shooting angle changes, causing it to be misjudged as a defective area. Therefore, it is necessary to exclude it to improve the accuracy of defect detection.

[0081] Pseudo-stretching deformation is caused by changes in the curvature of the labeling surface and the shooting angle. The larger the curvature of the labeling surface, the more severe the pseudo-stretching deformation may be. The pseudo-stretching deformation changes with the camera's viewing angle. Its position and shape change in a relatively regular manner, and similar pseudo-stretching areas are usually present in similar locations on multiple objects under test.

[0082] However, real variable defects (such as wrinkled areas) have obvious changes in light and shadow at different angles due to their own texture (grayscale distribution changes rapidly, and the morphology changes more obviously in images at different angles). They are usually defects in a single object to be tested and will not appear in a fixed form in similar positions on multiple objects to be tested.

[0083] Based on the similarity of the analyzed areas of different items in the same batch, combined with the fluctuation of the matching index and grayscale changes of the same item, defective areas are screened out to avoid pseudo-stretched areas being identified as defective areas, thereby improving the accuracy of defect detection.

[0084] Preferably, in one embodiment of the present invention, for each object to be tested having a region to be analyzed, the smaller the range of the first-order difference value of the matching index, the smaller the degree of fluctuation, the more stable the amplitude of the region's shape and position fluctuation with angle, and the less likely it is to be a defective region; the smaller the absolute value of the average value of the first-order difference value of the time series of the grayscale mean of the suspected defective region, the more regular the grayscale change with angle, and the less likely it is to be a defective region;

[0085] Based on this, the range of the first-order difference values ​​of the time series of the matching index is obtained as the matching fluctuation factor, which represents the fluctuation of the matching index of the same item. The absolute value of the average value of the first-order difference values ​​of the time series of the grayscale mean of the suspected defect area is obtained as the grayscale fluctuation factor, which represents the grayscale change of the suspected defect area within the same item.

[0086] Considering that in the coordinate system, the more areas of each area to be analyzed are within the local range of other objects to be tested, the greater the possibility that the area to be analyzed is a pseudo-stretching deformation area caused by the curvature of the labeling surface and the camera shooting angle;

[0087] Based on this, the average distance between each area to be analyzed and the center points of all other areas to be analyzed is obtained as the search radius. Within the search radius of each area to be analyzed, the areas to be analyzed of other objects to be tested whose area difference ratio is less than a preset area difference threshold are screened out and matched;

[0088] According to the number of areas matched by each area to be analyzed within the search radius, the defective areas are screened out by combining the matching fluctuation factor and the grayscale fluctuation factor.

[0089] As an example, the preset area difference threshold is 0.2, the absolute value of the difference between the area of ​​each area to be analyzed and the other areas to be analyzed is used as the numerator, the area of ​​each area to be analyzed is used as the denominator, and the fractional ratio is used as the ratio of the area difference between each area to be analyzed and the other areas to be analyzed;

[0090] The number of areas matched by each area to be analyzed within the search radius is taken as the numerator, the product of the matching fluctuation factor and the grayscale fluctuation factor is taken as the denominator, and the fractional ratio is taken as the pseudo-stretching deformation possibility of each area to be analyzed; the area with a pseudo-stretching deformation possibility less than or equal to the preset pseudo-stretching threshold (taken as 0.5) is taken as the defect area. The defect area screened out here corresponds to the real variable defect area.

[0091] The number of effective edge points reflects the clarity and completeness of the structural boundary in the labeling area, and the area of ​​the defective area represents the range of the abnormal area in the label. Therefore, the labeling effect evaluation value is obtained based on the number of all effective edge points and the area of ​​the defective area of ​​each item to be tested, which characterizes the labeling quality of the labeling machine and ultimately solves the technical problem of labeling detection accuracy affected by product labeling surface morphology and transmission offset.

[0092] Preferably, in one embodiment of the present invention, the larger the mean value of effective edge points in all labeling areas of the tested object and the more effective edge points there are, the more regular the label boundary is and the more fully it fits, reflecting the visual and geometric integrity of the label, and the higher the labeling effect evaluation value; the more pixels in the defective area, the more abnormal areas on the label that may affect recognition, aesthetics or function, and the lower the labeling effect evaluation value;

[0093] Therefore, the labeling effect evaluation value is obtained based on the mean value of the effective edge points in all identification areas of each object to be tested and the mean value of the number of pixels in all defective areas.

[0094] As an example, the mean of the effective edge points in all the identification areas of each object to be tested is used as the numerator, the mean of the number of pixels in all the defective areas of each object to be tested is used as the denominator, and the fractional ratio is used as the independent variable. After normalization by the sigmoid function, it is used as the labeling effect evaluation value of each object to be tested.

[0095] In another embodiment of the present invention, considering that the higher the degree of matching between the effective edge points and the edge points in the standard identification area, the more the labeling conforms to the standard and the better the labeling effect, the mean of the number of pixels in all defective areas of each item to be tested multiplied by the average value of the maximum matching effectiveness corresponding to all effective edge points can be used as the numerator, the mean of the number of pixels in all defective areas of each item to be tested can be used as the denominator, the fractional ratio can be used as the independent variable, and after normalization by the sigmoid function, the result can be used as the labeling effect evaluation value of each item to be tested.

[0096] Among them, the sigmoid function is a well-known function and will not be described in detail.

[0097] In one embodiment of the present invention, after obtaining the labeling effect evaluation value, it also includes: marking the items to be tested whose labeling effect evaluation value is lower than the preset quality threshold of 0.7 as defective products, removing them from normal products, and outputting the corresponding defect area image through a visualization device such as a screen for reference by relevant personnel.

[0098] An embodiment of the present invention also provides a labeling machine identification quality detection system using AI vision technology, which includes a memory, a processor, and a computer program, wherein the memory is used to store the corresponding computer program, and the processor is used to run the corresponding computer program. When the computer program runs in the processor, it can implement a labeling machine identification quality detection method using AI vision technology described in steps S1-S3.

[0099] In summary, in response to the technical problem that the accuracy of labeling detection is affected by the shape of the product labeling surface and the transmission offset, the present invention provides a labeling machine identification quality detection method and system using AI vision technology. First, the identification area is extracted; further, based on the similarity between the identification area and the standard identification area, the effective edge points are screened out, the different areas are transformed into the same coordinate system, and the suspected defect area is obtained; further, based on the similarity of the suspected defect area in the adjacent frame images, the matching index is obtained; further, based on the distribution of the matching index and the grayscale change in the suspected defect area, the area to be analyzed and the defect area are divided; further, based on the similarity of the areas to be analyzed of different items in the same batch, the defect area is screened out in combination with the fluctuation of the matching index and the grayscale change; finally, the labeling effect evaluation value is obtained based on the number of effective edge points and the area area of ​​the defect area. In response to the detection error problem caused by the deformation of the labeling surface and the transmission offset, the present invention extracts the defect area through multi-frame alignment, edge matching and grayscale analysis, eliminates pseudo defects by combining matching fluctuations and batch statistics, quantifies the labeling quality, and improves the detection accuracy and stability.

[0100] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A labeling machine marking quality detection method using AI vision technology, characterized in that: The method comprises: Obtain continuous frame images of each object to be tested on the conveyor belt and extract the identification area in the image; Filter out valid edge points based on edge similarity between the identified area and a preset standard identified area; transform different areas into the same coordinate system based on matching features between the valid edge points and the edge points of the standard identified area; analyze the grayscale difference between the identified area and the standard identified area in the same coordinate system to obtain a suspected defect area; obtain a matching index based on the similarity of the morphology and position of the suspected defect area in adjacent frame images; and divide the area to be analyzed and the defect area based on the distribution of all the matching indices corresponding to the same object to be tested and the grayscale change within the suspected defect area; Based on the similarity of the areas to be analyzed of different items in the same batch, combined with the fluctuation of the matching index and the grayscale change of the same item, the defective areas are screened out; based on the number of all the valid edge points of each item to be tested and the area of ​​the defective area, the labeling effect evaluation value is obtained.

2. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for obtaining the effective edge points includes: Extract edge points based on edge detection; select edge points of the marked area and edge points of the standard marked area one by one to form edge binary groups; in each edge binary group, obtain matching validity based on the similarity of vectors pointing from the center point of the area to the edge point, combined with the gradient amplitude similarity and curvature similarity between the edge points; In the identified area, edge points whose matching validity is greater than a preset validity threshold are marked as valid edge points.

3. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 2 is characterized in that: The method of transforming different regions into the same coordinate system includes: For each of the identified areas, the valid edge points are matched with the edge points corresponding to the maximum matching validity, and a transformation matrix is ​​obtained using RANSAC based on the matching results. The identified area is transformed to the coordinate system of the standard identified area using the transformation matrix.

4. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for obtaining the suspected defect area includes: In the same coordinate system, suspected defect area points are screened out based on the prominent features of the grayscale difference between the effective edge points in the identification area and the alignment points in the standard identification area; a morphological closing operation is performed on the connected domain where all the suspected defect area points in the identification area are located to obtain the suspected defect area.

5. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for obtaining the matching index includes: In two adjacent frames of images of the same object to be inspected, the matching index of the suspected defect area in the corresponding two adjacent frames is obtained based on the similarity between the vectors between the center points of the two suspected defect areas in the coordinate system and the direction of movement of the conveyor belt, combined with the difference between the distance between the centers and the aspect ratio of the minimum circumscribed rectangle.

6. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for dividing the area to be analyzed and the defect area includes: Obtaining the inherent damage possibility of each object to be tested based on the average of all the matching indexes corresponding to each object to be tested and the range between the grayscale means in each suspected defect area; The suspected defect area of ​​the test object whose inherent damage possibility is greater than the preset damage possibility is divided into a defect area; the suspected defect area of ​​the test object whose inherent damage possibility is less than or equal to the preset damage possibility is divided into an area to be analyzed.

7. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for screening out defective areas comprises: For each article to be tested having the area to be analyzed, obtaining the range of the first-order difference values ​​of the time series of the matching index as the matching fluctuation factor; obtaining the absolute value of the average value of the first-order difference values ​​of the time series of the grayscale mean of the suspected defect area as the grayscale fluctuation factor; In the coordinate system, the average distance between each area to be analyzed and the center point of all other areas to be analyzed is obtained as a search radius. Within the search radius of each area to be analyzed, the areas to be analyzed of other objects to be tested whose area difference ratio is less than a preset area difference threshold are screened out and matched; According to the number of regions matched by each of the regions to be analyzed within the search radius, the defective regions are screened out in combination with the matching fluctuation factor and the grayscale fluctuation factor.

8. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for obtaining the labeling effect evaluation value includes: A labeling effect evaluation value is obtained according to the average value of the effective edge points in all the identification areas of each object to be tested, combined with the average value of the number of pixels in all the defective areas.

9. The method for detecting the marking quality of a labeling machine using AI vision technology according to claim 1, characterized in that: The method for obtaining the identification area includes: Semantic recognition is used to extract the initial identification area in the image, and the initial identification area whose area is larger than the average area of ​​the identification areas in all images of the corresponding object to be tested is screened out as the identification area.

10. A labeling machine mark quality detection system using AI vision technology, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the labeling machine identification quality detection method using AI vision technology as described in any one of claims 1 to 9 are implemented.

Images