Defect detection method based on adaptive template comparison

Through the defect detection method of adaptive template comparison and grid division, the error detection problems caused by uneven lighting and camera dummy focus are solved, and higher defect detection accuracy and reliability are achieved.

CN119006360AActive Publication Date: 2024-11-22YOUWEI IMAGE TECH (SUZHOU) CO LTD

Patent Information

Application Number
CN202410867577.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-30
Publication Date
2024-11-22
Estimated Expiration
2044-06-30

AI Technical Summary

Technical Problem

The prior art has problems such as uneven light and camera blind focus in defect detection, resulting in uneven imaging grayscale and unclear edges, resulting in false detection and geometric dimension deviations in detection results.

Method used

Using defect detection method based on adaptive template comparison, templates are generated through image training, grayscale alignment and meshing are performed, abnormal areas are identified, and the main body and edge are divided through top hat operation, and the misaligned pixel areas are eliminated to obtain the accuracy of defects. Location.

Benefits of technology

It improves the defect detection rate and detection reliability, effectively deals with the influence of uneven light and ambient light, and improves the accuracy and robustness of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006360A_ABST
    Figure CN119006360A_ABST
Patent Text Reader

Abstract

The invention discloses a defect detection method based on self-adaptive template comparison, which comprises the steps of image training set preprocessing, gray level alignment, abnormity identification, abnormity screening, defect identification and the like, and comprises the following steps: firstly, generating a mean value template graph and a standard deviation template graph through an image training set; then, a defect pixel is highlighted by adopting a method of adaptively aligning the gray level of the template image to the gray level of the image to be detected, then, an abnormal area is divided according to a gray level threshold value formula, and finally, defects are cleaned out through splitting operation on the abnormal area. According to the defect detection method based on adaptive template comparison, the template is generated through image training, grid domain division is performed on the template based on the gray level of the to-be-detected image, and further abnormality cleaning is performed on the basis of abnormal defect identification, especially for the influence of ambient light, so that the defect detection accuracy is improved. Therefore, the defect detection rate and reliability are effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and automated detection, and in particular to a defect detection method based on adaptive template comparison. Background Art

[0002] In industrial automated production, defect detection is a key link to ensure product quality. Traditional manual detection has problems such as low detection efficiency and high false detection rate. In recent years, with the development of computer vision and deep learning technology, defect detection methods based on deep learning have gradually become a research hotspot. However, these methods often require a large amount of labeled data for training and are difficult to adapt to changing scenarios. Therefore, the present invention proposes a defect detection algorithm based on adaptive template comparison, aiming to solve the above problems.

[0003] 1. When the existing technology performs template comparison algorithms, due to the weak light source of the product line or controller problems, uneven lighting occurs, causing uneven grayscale imaging of the product under the camera. When comparing the templates, the difference with the standard image is greater than the set threshold, causing product misdetection.

[0004] 2. When the prior art performs the template comparison algorithm, due to inconsistent product height or camera out-of-focus, the edges of the captured images are unclear, and more pixels in the excessive area appear at the edge of the image, causing errors in the positioning algorithm, resulting in changes in the corresponding coordinates, causing the detected defects to have pixels in the excessive area, causing deviations in the geometric dimensions of the detected defects, thereby causing product misdetection. Summary of the invention

[0005] The main technical problem solved by the present invention is to provide a defect detection method based on adaptive template comparison, generate a template through image training, and divide the template into grid domains based on the grayscale of the image to be tested, and then further perform abnormal cleaning on the basis of abnormal defect identification, especially for the influence of ambient light, thereby effectively improving the defect detection rate and reliability.

[0006] In order to solve the above technical problems, a technical solution adopted by the present invention is: to provide a defect detection method based on adaptive template comparison, comprising the following steps:

[0007] The first step is to load the image training set, generate the mean template image according to the grayscale mean training of the training set images, and obtain the standard deviation template image according to the grayscale standard deviation training of the training set images;

[0008] The second step is image preprocessing, which involves position correction and cropping of the grayscale image of the product to be tested.

[0009] The third step is grayscale alignment, dividing the grids, forming a grayscale set based on the grids, stretching the grayscale of each grid of the mean template image to reach the grayscale level of each grid of the grayscale image of the product to be tested, and highlighting the defective pixels;

[0010] The fourth step is to identify anomalies and use formula t u (x,y)=i(x,y)±max{a u ,b u v(x,y)} calculates the grayscale upper and lower thresholds t of normal pixels u , where i represents the grayscale value of the pixel in the mean template image; (x, y) represents the pixel coordinates; max{…} represents the maximum grayscale value; a u represents the first preset parameter; b u represents the second preset parameter; v represents the gray value of the pixel point in the standard deviation template image; by comparing the threshold value, the abnormal area where the defective pixel is located is obtained;

[0011] The fifth step is to screen out the anomalies, top-hat the abnormal areas, segment the main body and the edge, and identify the misaligned pixel areas caused by the position correction error in the second step;

[0012] The sixth step is to identify defects, remove the misaligned pixel area from the abnormal area, and obtain the exact location of the defect.

[0013] In a preferred embodiment of the present invention, in the first step, a batch of grayscale images of qualified products without defects are used to form the image training set; the training method of the mean template image and the standard deviation template image is: for all pixel points of each image in the image training set, the pixel grayscale mean and standard deviation at the same pixel coordinates of each image are calculated, thereby generating the mean template image and the standard deviation template image.

[0014] In a preferred embodiment of the present invention, the grayscale mean of each image in the image training set is controlled at the same grayscale level by a variable magnification stretching method, so that the grayscale values ​​of each image are aligned; the variable magnification stretching method is: multiplying the grayscale of all pixels on a single image by a variable magnification at the same time, so that the grayscale mean of the single image is uniformly increased to a preset grayscale.

[0015] In a preferred embodiment of the present invention, in the second step, the grayscale image of the product to be tested is obtained in batches by a visual camera at a fixed position, and the position correction method is: using the NCC cross-correlation calculation method and performing normalization processing based on the correlation coefficient to align related pixels to the same coordinates; the cropping method is: cropping to the corresponding size based on the size of the mean template image.

[0016] In a preferred embodiment of the present invention, in the third step, the grids are divided on the grayscale image of the product to be tested and the mean template image, respectively, to form grid domains of the same size; the grid grayscale stretching method is: for each grayscale image of the product to be tested, the domain grayscale mean of the image pixel points in each grid domain of the mean template image is multiplied by a variable magnification one by one, and stretched one by one to the same level of the domain grayscale mean of the image pixel points in each grid domain of the grayscale image of the product to be tested.

[0017] In a preferred embodiment of the present invention, in the fourth step, the defective pixel has two types: bright part abnormality and dark part abnormality; the formula t u (x,y)=i(x,y)±max{a u ,b u v(x, y)} defines anomalies according to the grayscale range, so that pixels whose grayscale values ​​of the grayscale image of the product to be tested exceed the threshold are marked as abnormal pixels.

[0018] In a preferred embodiment of the present invention, in the fifth step, the top hat method is: respectively process the grayscale image of the product to be tested and the mean template image through the Sobel operator to obtain two grayscale gradient direction images; respectively for the main part and the edge part, by comparing the grayscale means of the two grayscale gradient direction images, the misaligned pixel area of ​​the main part and the misaligned pixel area of ​​the edge part are screened out.

[0019] The beneficial effect of the present invention is that the present invention provides a defect detection method based on adaptive template comparison, which generates a template through image training, and divides the template into grid domains based on the grayscale of the image to be tested, and then further performs abnormal cleaning based on the abnormal defect identification, especially for the influence of ambient light, thereby effectively improving the defect detection rate and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0021] Figure 1 It is a flow chart of a defect detection method based on adaptive template comparison of the present invention;

[0022] Figure 2 It is a schematic diagram of template generation of a defect detection method based on adaptive template comparison according to the present invention;

[0023] Figure 3 It is a schematic diagram of grid domain grayscale alignment of a defect detection method based on adaptive template comparison according to the present invention;

[0024] Figure 4 It is a schematic diagram of abnormal splitting of a defect detection method based on adaptive template comparison according to the present invention;

[0025] Figure 5 It is a grayscale gradient direction map of a defect detection method based on adaptive template comparison in the present invention. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] like Figure 1-5 As shown, the embodiment of the present invention includes:

[0028] A defect detection method based on adaptive template comparison comprises the following steps:

[0029] The first step is to load the image training set, generate the mean template image according to the grayscale mean training of the training set images, and obtain the standard deviation template image according to the grayscale standard deviation training of the training set images;

[0030] The second step is image preprocessing, which involves position correction and cropping of the grayscale image of the product to be tested.

[0031] The third step is grayscale alignment, dividing the grids, forming a grayscale set based on the grids, stretching the grayscale of each grid of the mean template image to reach the grayscale level of each grid of the grayscale image of the product to be tested, and highlighting the defective pixels;

[0032] The fourth step is to identify anomalies and use formula t u (x,y)=i(x,y)±max{a u ,b u v(x,y)} calculates the grayscale upper and lower thresholds t of normal pixels u , where i represents the grayscale value of the pixel in the mean template image; (x, y) represents the pixel coordinates; max{…} represents the maximum grayscale value; a u represents the first preset parameter; b u represents the second preset parameter; v represents the gray value of the pixel point in the standard deviation template image; by comparing the threshold value, the abnormal area where the defective pixel is located is obtained;

[0033] The fifth step is to screen out the anomalies, top-hat the abnormal areas, segment the main body and the edge, and identify the misaligned pixel areas caused by the position correction error in the second step;

[0034] The sixth step is to identify defects, remove the misaligned pixel area from the abnormal area, and obtain the exact location of the defect.

[0035] like Figure 2 As shown, in the first step, a batch of qualified product grayscale images without defects are used to form the image training set; the training method of the mean template image and the standard deviation template image is: for all the pixel points of each image in the image training set, the pixel grayscale mean at the same pixel coordinates of each image is calculated. and standard deviation Thus, the mean template image and the standard deviation template image are generated. Among them, x1 represents the grayscale value of the pixel at the same pixel coordinate in the first image, x2 represents the grayscale value of the pixel at the same pixel coordinate in the second image, and so on.

[0036] Furthermore, the grayscale mean of each image in the image training set is controlled at the same grayscale level by a variable magnification stretching method, so that the grayscale values ​​of each image are aligned; the variable magnification stretching method is: multiplying the grayscale of all pixels on a single image by a variable magnification at the same time, so that the grayscale mean of the single image is uniformly increased to a preset grayscale.

[0037] Furthermore, in the second step, the grayscale image of the product to be tested is obtained in batches by a visual camera at a fixed position, and the position correction method is: using the NCC cross-correlation calculation method and performing normalization processing based on the correlation coefficient to align related pixels to the same coordinates; the cropping method is: cropping to the corresponding size based on the size of the mean template image, and cropping the size of the grayscale image of the product to be tested to be consistent with the size of the mean template image.

[0038] Furthermore, in the third step, the grids are divided on the grayscale image of the product to be tested and the mean template image respectively to form grid domains of the same size; the grid grayscale stretching method is: for each grayscale image of the product to be tested, the domain grayscale mean of the image pixel points in each grid domain of the mean template image is multiplied by a variable magnification one by one, and stretched one by one to the same level of the domain grayscale mean of the image pixel points in each grid domain of the grayscale image of the product to be tested.

[0039] like Figure 3 As shown in the figure, the mean template image and the grayscale image of the product to be tested are divided into multiple grid regions (Regions) according to the scale of 32x32 pixels. If the sizes of the mean template image and the grayscale image of the product to be tested are both 64x64, 2x2 grid regions are divided, and the grayscale means of the mean template image and the grayscale image of the product to be tested in each grid region are calculated (the grayscale set of the mean template image [10, 20; 30, 40] and the grayscale set of the grayscale image of the product to be tested [20, 40; 30, 20] are obtained), and the grayscale means of each grid region of the mean template image are stretched to the same level as the grayscale image of the product to be tested (stretched to [20, 40; 30, 20]).

[0040] In actual situations, uneven lighting usually causes uneven grayscale of the grayscale image of the product to be tested. At this time, the grayscale of the grayscale image of the product to be tested is [20, 40; 10, 20], and grayscale alignment is required: for the first grid domain, the stretching ratio 20 / 10=2 should be calculated, and the grayscale of each pixel in the first grid domain of the mean template image should be multiplied by 2. Similarly, the grayscale values ​​of all other grid domains after stretching are obtained, and the mean template image is stretched from its original grayscale set [10, 20; 30, 40] to [20, 40; 10, 20] (consistent with the grayscale set of the grayscale image of the product to be tested). After grayscale alignment, when the grayscale mean is stretched according to the ratio, the grayscale of the defective pixel will also be stretched synchronously, thereby highlighting it.

[0041] For example, for the first grid domain, the grayscale mean of the mean template image is 10. Assuming the grayscale of the defect is 15, under the condition of uneven lighting, the grayscale mean of the first grid domain of the grayscale image of the product to be tested is 20, and the defect grayscale is 25. If grayscale alignment is not performed, the entire grid domain will be judged as a defect, resulting in a large number of false detections. In order to avoid the above situation, it is necessary to multiply the grayscale through affine transformation. After the grayscale alignment operation, only grayscales greater than 25 will be judged as defects.

[0042] Furthermore, in the fourth step, the defective pixels have two types: bright part abnormality and dark part abnormality; the formula t u (x,y)=i(x,y)±max{a u ,b u v(x, y)} defines anomalies according to the grayscale range, so that pixels whose grayscale values ​​of the grayscale image of the product to be tested exceed the threshold are marked as abnormal pixels.

[0043] For example, by the formula t u (x,y)=i(x,y)+max{a u ,b u v(x,y)} calculates the upper threshold:

[0044] Calculate t u (0,0) = gray value 5.5 of the mean template pixel (e.g. (0,0)) + the maximum of the two numbers in the brackets {parameter (e.g. set to 10), parameter (e.g. set to 5) x gray value (1.368) of the standard deviation template pixel (e.g. (0,0))};

[0045] Then t u (0,0) = 5.5 + the maximum of the two numbers in the brackets {10, 5x1.368 = 6.84};

[0046] Then t u(0,0)=5.5+10=15.5;

[0047] This value is the upper threshold of the grayscale of the image to be tested at the (0, 0) coordinate.

[0048] Therefore, if the grayscale value of the image to be tested at the coordinate (0, 0) is greater than 15.5, this pixel is judged as a defective pixel.

[0049] Similarly, through the formula t u (x,y)=i(x,y)-max{a u ,b u v(x,y)} calculates the lower threshold.

[0050] In the fourth step, if Figure 4 As shown in (a), the above method can obtain all the abnormal coordinate sets on the grayscale image of the product to be tested. These abnormalities are usually connected together, including bright abnormalities and / or dark abnormalities, and also include a group of other abnormalities (here the out-of-focus outline W2 of the edge of the product is taken as an example). This group of other abnormalities is mainly caused by the position correction in the second step. This piece of abnormal pixels forms a misaligned pixel area, that is, when the pixel coordinates are not completely aligned, misaligned coordinates will be generated, and the misaligned coordinates will be mistakenly identified as abnormalities.

[0051] In the fifth step, in order to solve the above-mentioned misalignment problem, it is necessary to separate these misaligned coordinates from the real bright and dark defects.

[0052] First, perform the split action:

[0053] Create a rectangle R1 with a width of 2 and a height of 1 (the width can be adjusted according to the positioning accuracy, and the height is fixed at 1) and a rectangle R2 with a width of 1 and a height of 2 (the width is fixed at 1 and the height can be adjusted). Use R1 and R2 as the region structure to perform region opening operations on the abnormal coordinate region to obtain the main part W1 (the real light and dark defect) of the abnormal coordinate region. Subtract the abnormal coordinate region from the main part to obtain the edge part W2 (misaligned coordinates), as shown in Figure 4 (b) as shown.

[0054] Then, the top-hat operation in the fifth step is used for comparison. The grayscale gradient directions of all split abnormal coordinate areas are compared with the grayscale gradient directions of the mean template image. The grayscale image of the product to be tested and the mean template image are processed separately by the Sobel operator to obtain two grayscale gradient direction images. For the main part and the edge part respectively, the grayscale mean of the two grayscale gradient direction images are compared to screen out the misplaced pixel areas of the main part and the misplaced pixel areas of the edge part.

[0055] like Figure 5As shown, the Sobel operator is used to make pixel value differences in the horizontal and vertical directions. For any pixel point, the grayscale gradient value in the X direction is Gx, and the grayscale gradient value in the Y direction is Gy. The gradient angle value of the pixel point is arctan (Gx / Gy). In this example, the gradient angle value is set in the range of 0 to 179°.

[0056] The image U to be tested after the second step of preprocessing is processed using the Sobel operator to obtain the grayscale gradient direction map A of the image to be tested, and the grayscale mean A1 of the main part and the grayscale mean A2 of the edge part are calculated.

[0057] The mean template image is processed using the Sobel filtering method to obtain the grayscale gradient direction map B of the mean value template image, and the grayscale mean B1 of the main part is calculated, and the grayscale mean B2 of the edge part is calculated.

[0058] If A1-B1 is greater than a certain set coefficient (for example, 5), it is judged as an abnormal coordinate area (defect); if A1-B1 is less than a certain set coefficient, it is judged as a normal coordinate area (false detection).

[0059] Finally, the coordinates judged as normal coordinate areas are removed from the abnormal coordinate set obtained in step 4 to obtain the cleaned abnormal coordinate set, which is the final output defect location result.

[0060] The beneficial effects of the present invention are:

[0061] (1) Grayscale adaptation of template: The template image is divided into grids and adapted to the image to be tested. This can effectively handle uneven lighting conditions and improve detection accuracy and robustness.

[0062] (2) The main part and the edge part are segmented through the top hat operation to improve the accuracy of the detection results.

[0063] In summary, the present invention provides a defect detection method based on adaptive template comparison, which generates a template through image training, and divides the template into a grid domain based on the grayscale of the image to be tested. Then, on the basis of abnormal defect identification, the abnormality cleaning is further performed, especially for the influence of ambient light, thereby effectively improving the defect detection rate and reliability.

[0064] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A defect detection method based on adaptive template comparison, characterized in that: The following steps are involved: The first step is to load the image training set, generate the mean template image according to the grayscale mean training of the training set images, and obtain the standard deviation template image according to the grayscale standard deviation training of the training set images; The second step is image preprocessing, which involves position correction and cropping of the grayscale image of the product to be tested. The third step is grayscale alignment, dividing the grids, forming a grayscale set based on the grids, stretching the grayscale of each grid of the mean template image to reach the grayscale level of each grid of the grayscale image of the product to be tested, and highlighting the defective pixels; The fourth step is to identify anomalies and use formula t u (x, y) = i (x, y) ± max {a u , b u v(x, y)} calculates the grayscale upper and lower thresholds t of normal pixels u , where i represents the grayscale value of the mean template pixel; (x, y) represents the pixel coordinates; max{…} represents the maximum grayscale value; a u represents the first preset parameter; b u represents the second preset parameter; v represents the gray value of the pixel point in the standard deviation template image; by comparing the threshold value, the abnormal area where the defective pixel is located is obtained; The fifth step is to screen out the anomalies, top-hat the abnormal areas, segment the main body and the edge, and identify the misaligned pixel areas caused by the position correction error in the second step; The sixth step is to identify defects, remove the misaligned pixel area from the abnormal area, and obtain the exact location of the defect.

2. The defect detection method based on adaptive template comparison according to claim 1, characterized in that: In the first step, a batch of grayscale images of qualified products without defects are used to form the image training set; The training method of the mean template map and the standard deviation template map is: for all pixel points of each image in the image training set, the pixel grayscale mean and standard deviation at the same pixel coordinates of each image are calculated, thereby generating the mean template map and the standard deviation template map.

3. The defect detection method based on adaptive template comparison according to claim 2, characterized in that: The grayscale mean of each image in the image training set is controlled at the same grayscale level by a variable magnification stretching method, so that the grayscale values ​​of each image are aligned; the variable magnification stretching method is: multiplying the grayscale of all pixels on a single image by a variable magnification at the same time, so that the grayscale mean of the single image is uniformly increased to a preset grayscale.

4. The defect detection method based on adaptive template comparison according to claim 1, characterized in that: In the second step, the grayscale image of the product to be tested is obtained in batches by a visual camera at a fixed position. The position correction method is: using the NCC cross-correlation calculation method and performing normalization processing based on the correlation coefficient to align related pixels to the same coordinates; the cropping method is: cropping to the corresponding size based on the size of the mean template image.

5. The defect detection method based on adaptive template comparison according to claim 1, characterized in that: In the third step, the grids are divided on the grayscale image of the product to be tested and the mean template image respectively to form grid domains of the same size; the grid grayscale stretching method is: for each grayscale image of the product to be tested, the domain grayscale mean of the image pixel points in each grid domain of the mean template image is multiplied by a variable magnification one by one, and stretched one by one to the same level of the domain grayscale mean of the image pixel points in each grid domain of the grayscale image of the product to be tested.

6. The defect detection method based on adaptive template comparison according to claim 1, characterized in that: In the fourth step, the defective pixels include two types: bright part abnormality and dark part abnormality; the formula t u (x,y)=i(x,y)±max{a u ,b u v(x, y)} defines anomalies according to the grayscale range, so that pixels whose grayscale values ​​of the grayscale image of the product to be tested exceed the threshold are marked as abnormal pixels.

7. The defect detection method based on adaptive template comparison according to claim 1, characterized in that: In the fifth step, the top hat method is: respectively process the grayscale image of the product to be tested and the mean template image through the Sobel operator to obtain two grayscale gradient direction images; respectively for the main part and the edge part, by comparing the grayscale means of the two grayscale gradient direction images, the misaligned pixel area of ​​the main part and the misaligned pixel area of ​​the edge part are screened out.

Citation Information

Patent Citations

  • Product edge defect detection method

    CN112330678A

  • Explosive development abnormal defect detection method and system

    CN115564727A

  • Visual detection algorithm and detection system for wafer surface defects

    CN115791822A

  • Railway fastener defect online detection and identification method based on machine vision

    CN116757990A

  • Element positioning method and system based on color and gradient

    WO2017177717A1

Cited By

  • Defect detection method and system based on hybrid expert model, and storage medium

    CN119785054A

  • Defect detection method, system and storage medium based on hybrid expert model

    CN119785054B

  • AOI detection method based on semiconductor chip consistency defect

    CN121961999A

  • Character printing defect detection method and system, medium and program product

    CN122473798A