An image defect positioning method based on dichotomy

Abstract

The application discloses a kind of based on dichotomy's image defect positioning method, comprising: the image to be measured and template image are divided into two parts;Respectively calculate the image similarity between the image to be measured sub-image and template sub-image with corresponding relationship;According to image similarity, the image to be measured sub-image and template sub-image are stored to abnormal image set, normal image set;If the image size of the image to be measured sub-image in abnormal image set is not less than equal to threshold value, then the image to be measured sub-image and template sub-image with corresponding relationship in abnormal image set are again divided into two parts, and image comparison is carried out;Otherwise, the image to be measured sub-image is recorded as the region where the defect is located, and the defect positioning is completed.The method uses dichotomy to improve the defect positioning speed, and also solves the problem of poor defect positioning accuracy caused by poor image stability and uneven brightness by fusing edge similarity, with the characteristics of high positioning accuracy and fast processing speed, especially suitable for online defect detection process.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to an image defect localization method based on the binary search method. Background Technology

[0002] In visual inspection, determining the presence and location of defects in acquired images is a perennial topic. Defects include imperfections arising from the manufacturing process of the object under test, such as cracks, dents, bulges, and missing parts, as well as foreign objects that have fallen onto the surface of the object, such as leaves, plastic bags, gloves, and tools. Common defect identification methods, such as edge extraction and template matching, achieve the expected detection accuracy when the image is clear, the background is simple, and there is little external environmental interference. However, when the image background is complex and easily affected by environmental interference, the detection accuracy will be significantly reduced. Especially in outdoor environments, the image acquisition process is easily affected by factors such as lighting and weather, resulting in poor image quality and increasing the difficulty of defect identification.

[0003] In summary, existing image defect detection schemes have the following shortcomings:

[0004] First, the coverage of defect detection is insufficient. Most defect detection solutions only detect the edges of the object under test and cannot comprehensively detect defects in key areas of the image or missing key components.

[0005] Second, the detection scheme is not stable enough, and it is easy to miss or misdetect. Furthermore, the detection process is easily affected by factors such as light and weather, resulting in inaccurate detection. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an image defect localization method based on the bisection method. This method aims to quickly and efficiently analyze the location of defects in an image. While improving the defect localization speed by utilizing the bisection method, it also solves the problem of poor defect localization accuracy caused by poor image stability and uneven brightness by fusing edge similarity. It features high localization accuracy and fast processing speed, and is particularly suitable for online defect detection processes.

[0007] Therefore, the technical solution of the present invention is as follows:

[0008] A method for image defect localization based on binary search includes the following steps:

[0009] Step 1: Record the acquired image of the object to be tested as the image to be tested, and record the pre-stored normal image of the object to be tested as the template image. The visual sensor pose of the image to be tested and the template image are the same when they are acquired.

[0010] The image to be tested and the template image are each divided into two equal parts along the same segmentation direction to form two sub-images to be tested and two template sub-images; the segmentation direction is either horizontal or vertical.

[0011] Step 2: Calculate the image similarity between the corresponding sub-images to be tested and the template sub-images respectively;

[0012] If the calculated image similarity is greater than the preset value, the sub-image to be tested and the template sub-image will be stored in the normal image set.

[0013] Otherwise, store the sub-image to be tested and the template sub-image in the abnormal image set;

[0014] Step 3: Determine if any images exist in the abnormal image set:

[0015] If it does not exist, the image to be tested will be marked as a normal image.

[0016] If it exists, proceed to step four;

[0017] Step 4: Determine whether the image size of the sub-image to be tested in the abnormal image set is less than or equal to the threshold:

[0018] If not, then divide the corresponding sub-images to be tested and template sub-images in the abnormal image set into two equal parts according to the same segmentation direction, and use the two newly obtained sub-images to be tested and two template sub-images to perform step two, while clearing the abnormal image set.

[0019] If so, the sub-image to be tested is recorded as the area where the defect is located, thus completing the defect localization.

[0020] Furthermore, in step three, if there is more than one pair of test sub-images and template sub-images with corresponding relationships in the abnormal image set, then step four is executed synchronously for each test sub-image and template sub-image.

[0021] Preferably, in step two, the preset value is set to 0.9-0.95;

[0022] In order to accurately locate the position of the defect in the image, in step four, the coordinates of the upper left corner of the defect area in the image to be tested are marked as the coordinates of the defect location.

[0023] Preferably, the segmentation direction is the direction of the long side centerline of the image to be tested / sub-image to be tested.

[0024] Furthermore, in step two, the correspondence between the sub-image to be tested and the template sub-image is determined according to the position of the sub-image in the original image, as follows:

[0025] Record the coordinates of the top left corner of the sub-image to be tested in the sub-image to be tested, and the coordinates of the top left corner of the template sub-image in the template image, respectively;

[0026] Establish a correspondence between the test sub-image and the template sub-image whose coordinate values ​​are closest.

[0027] Preferably, in step two, the method for calculating the image similarity s between the corresponding sub-image to be tested and the template sub-image is as follows:

[0028] s = a × S1 + b × S 边缘

[0029] Where a and b are both proportionality coefficients, and a+b=1, S1 represents the similarity between the sub-image to be tested and the template sub-image, S 边缘 This represents the similarity between the edge images of the sub-image to be tested and the edge images of the template sub-image.

[0030] Preferably, the proportionality coefficient 'a' is set as follows:

[0031] Calculate the average gray levels of the image to be tested and the template image, and denote them as aver. g aver m ;

[0032] Next, calculate the absolute value of the difference between the average gray levels. If the absolute value of the difference is greater than the value A, then a is taken as... The smaller of the values, when the difference is less than or equal to the value A, then a takes the value of . The larger value in the range.

[0033] Furthermore, similarity calculation methods include cosine similarity, Pearson correlation coefficient similarity, SSIM structural similarity, and normalized mutual information similarity.

[0034] Preferably, the value of A is between 30 and 150.

[0035] Furthermore, in step four, it is determined whether the image size of the sub-image to be tested in the abnormal image set is less than or equal to the threshold, as follows:

[0036] Calculate the area or diagonal length of the sub-image to be tested, and determine whether the value is less than the threshold.

[0037] The present invention has the following beneficial effects:

[0038] (1) Image-based contactless computing methods can replace manual inspection. In defect detection schemes, they can not only accurately and quickly locate defects, but also avoid safety accidents that occur during manual inspection.

[0039] (2) The image similarity calculation method that combines grayscale information with edge information can solve the problem of uneven image brightness caused by external environment, bad weather and other factors, and improve the robustness of the defect detection method.

[0040] (3) The image binary search method is used to find defect areas. Compared with sliding window and image partition detection methods, the algorithm search efficiency is improved. When there are no defects in the image, only one judgment is needed. When there are multiple defects in the image, multiple test sub-images and template sub-images can be detected simultaneously, and the regions where multiple defects are located can be found in the fastest speed. The average time to process a 4096×3000 resolution image is 50-200ms. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the detection method flow in a specific implementation embodiment;

[0042] Figure 2 This is a schematic diagram of the bisection image processing procedure in a specific implementation. Detailed Implementation

[0043] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0044] A method for image defect localization based on the bisection method, such as Figure 1 As shown, it includes the following steps:

[0045] Step 1: Record the acquired image of the object to be tested as the image to be tested, and record the pre-stored normal image of the object to be tested as the template image. The visual sensor pose is the same when the image to be tested and the template image are acquired.

[0046] Among them, the pre-stored normal test object image can be manually confirmed to be free of defects.

[0047] like Figure 2 As shown, the image to be tested and the template image are each divided into two equal parts according to the same segmentation direction, forming two sub-images to be tested and two template sub-images; the segmentation direction is either horizontal or vertical.

[0048] Step 2: Calculate the image similarity between the corresponding sub-images to be tested and the template sub-images respectively;

[0049] If the calculated image similarity is greater than the preset value (0.9-0.95), the sub-image to be tested and the template sub-image are stored in the normal image set.

[0050] Otherwise, store the sub-image to be tested and the template sub-image in the abnormal image set;

[0051] Step 3: Determine if any images exist in the abnormal image set:

[0052] If it does not exist, the image to be tested will be marked as a normal image.

[0053] If it exists, proceed to step four;

[0054] Step 4: Determine whether the image size of the sub-image to be tested in the abnormal image set is less than or equal to the threshold:

[0055] If not, then divide the corresponding sub-images to be tested and template sub-images in the abnormal image set into two equal parts according to the same segmentation direction, and use the two newly obtained sub-images to be tested and two template sub-images to perform step two, while clearing the abnormal image set.

[0056] If so, the sub-image to be tested is recorded as the area where the defect is located, thus completing the defect localization.

[0057] In this step, it is ensured that the sub-images executed in step two each time are the latest sub-images obtained by equal division, and the abnormal image set is cleared in a timely manner, so that in step three, the abnormal image set only stores the latest sub-images obtained by judgment, and there are no historically stored sub-images.

[0058] Specifically, in step three, if there is more than one pair of test sub-images and template sub-images with corresponding relationships in the abnormal image set, then step four is executed synchronously for each test sub-image and template sub-image.

[0059] For example, defects in the image under test, multiple defects, or large defects can cause the defect area to be divided into two parts.

[0060] In order to accurately locate the position of the defect in the image, in practice, the coordinates of the upper left corner of the defect area in the image to be tested are used as the coordinates of the defect location.

[0061] In specific implementation, step four, determining whether the image size of the sub-image to be tested in the abnormal image set is less than or equal to the threshold, is done as follows:

[0062] Calculate the area or diagonal length of the sub-image to be tested, and determine whether the value is less than the threshold.

[0063] The threshold can be set based on empirical values ​​of defects, or it can be set as the product of the minimum size of the defect to be detected and the pixel equivalent of the template image.

[0064] The segmentation direction is set based on the empirical size of the defect. For example, if the defect is long and horizontally distributed in the image, the segmentation direction is horizontal.

[0065] If the defect is long and vertically distributed in the image, then the segmentation direction is vertical.

[0066] If the defect shape is irregular or there is no empirical data on its location and distribution, the preferred segmentation direction is the central axis of the longer side of the image to be tested / sub-image to be tested. Since the width (horizontal direction) of the image / sub-image is greater than its height (vertical direction), it is divided in two along the central axis of the width (vertical direction), i.e., the width is divided in two while the height remains unchanged. This makes the final segmented area approximate a square.

[0067] Specifically, in step two, the correspondence between the sub-image to be tested and the template sub-image is determined according to the position of the sub-image in the original image, as follows:

[0068] Record the coordinates of the top left corner of the sub-image to be tested in the sub-image to be tested, and the coordinates of the top left corner of the template sub-image in the template image, respectively;

[0069] Establish a correspondence between the test sub-image and the template sub-image whose coordinate values ​​are closest.

[0070] The method for calculating the image similarity s between the test sub-image and the template sub-image that have a corresponding relationship is as follows:

[0071] s = a × S1 + b × S 边缘

[0072] Where a and b are both proportionality coefficients, and a+b=1, S1 represents the similarity between the sub-image to be tested and the template sub-image, S 边缘 This represents the similarity between the edge images of the sub-image to be tested and the edge images of the template sub-image.

[0073] More specifically, the proportionality coefficients a and b can be reasonably set based on empirical values. The values ​​of a and b are both greater than or equal to 0 and less than or equal to 1.

[0074] To make the proportional coefficient setting more reasonable and increase the accuracy of defect identification, the preferred implementation is as follows:

[0075] The method for setting the scaling factor 'a' is as follows:

[0076] Calculate the average gray levels of the image to be tested and the template image, and denote them as aver. g aver m ;

[0077] Next, calculate the absolute value of the difference between the average gray levels. If the absolute value of the difference is greater than the value A, then a is taken as... The smaller of the values, when the difference is less than or equal to the value A, then a takes the value of . The larger value in the range.

[0078] The value of A is generally between 30 and 150;

[0079] For image acquisition scenarios that are outdoors for extended periods, where there are significant differences in brightness between images, the value of A should range from 80 to 150.

[0080] For image acquisition scenarios that are in indoor environments for extended periods, where the difference in brightness between images is small, the value of A ranges from 30 to 80.

[0081] The similarity calculation methods include cosine similarity, Pearson correlation coefficient similarity, SSIM structural similarity, and normalized mutual information similarity.

[0082] Taking cosine similarity as an example, the calculation process is as follows:

[0083] The test sub-image and the template sub-image have the same size, with a width of c and a height of r. The grayscale value of any point in the test sub-image is g(i,j), and the grayscale value of any point in the template sub-image is m(i,j), where i is the index in the width direction (0≤i≤c) and j is the index in the height direction (0≤j≤r). The cosine similarity S1 is calculated as follows:

[0084]

[0085] Edge extraction is performed on both the test sub-image and the template sub-image to obtain edge images of the test sub-image and the template sub-image. The gray level of any point on the edge image of the detected sub-image is g. c (i,j), where the gray level of any point on the edge image of the template sub-image is m. c If (i,j), then the cosine similarity S between edge images 边缘 The calculation method is as follows:

[0086]

[0087] Calculate the image similarity s between the corresponding sub-image to be tested and the template sub-image:

[0088] s = a × S1 + b × S 边缘

[0089] Other methods for calculating the similarity between images, such as Pearson correlation coefficient similarity, SSIM structural similarity, and normalized mutual information similarity, all use the calculation formulas provided in the existing technology, and will not be elaborated here.

[0090] When image acquisition takes place outdoors, conditions such as rain and snow are common. The brightness of the measured defect image under rain or snow can differ significantly from the template image. Furthermore, due to limitations in the detection system's hardware, inconsistent lighting during image acquisition can also cause discrepancies between the measured image and the template image. This brightness difference between the detected image and the template image can lead to false positives in defect detection. To address these issues, this embodiment introduces edge image similarity into the image similarity calculation, improving the accuracy of the calculation.

[0091] For ease of interpretation and precise definition of the appended claims, the terms “upper,” “lower,” “inner,” and “outer” are used to describe features of exemplary embodiments with reference to the positions of these features shown in the accompanying drawings.

[0092] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A dichotomous-based image defect positioning method, characterized in that, The method comprises the following steps: Step one, record the collected image of the object to be measured as the image to be measured, and record the pre-stored normal image of the object to be measured as the template image, the visual sensor poses of the image to be measured and the template image are the same when they are collected; Divide the image to be measured and the template image into two parts according to the same segmentation direction, forming two image to be measured and two template images; the segmentation direction is the horizontal direction or the vertical direction; Step two, calculate the image similarity between the image to be measured and the template image with corresponding relationship respectively; If the calculated image similarity is greater than the preset value, store the image to be measured and the template image into the normal image set; Otherwise, store the image to be measured and the template image into the abnormal image set; Step three, judge whether there is an image in the abnormal image set: If not, directly mark the image to be measured as a normal image; If yes, execute step four; Step four, judge whether the image size of the image to be measured in the abnormal image set is less than or equal to the threshold value: If not, divide the image to be measured and the template image with corresponding relationship in the abnormal image set into two parts according to the same segmentation direction respectively, and execute step two using the newly obtained two image to be measured and two template images, and clear the abnormal image set at the same time; If yes, record the image to be measured as the defect area, and complete the defect positioning.

2. The method of claim 1, wherein: In step three, if there is more than one pair of image to be measured and template image with corresponding relationship in the abnormal image set, each pair of image to be measured and template image is executed synchronously in step four.

3. The method of claim 1, wherein: In step two, the preset value is 0.9-0.95; in step four, the coordinates of the upper left corner of the defect area in the image to be measured are recorded as the coordinates of the defect position.

4. The method of claim 1, wherein: The segmentation direction is the long edge axis direction of the image to be measured / image to be measured.

5. The method of claim 1, wherein: In step two, the corresponding relationship between the image to be measured and the template image is determined according to the position of the image to be measured in the original image, in the following way: Record the coordinates of the upper left corner of the image to be measured in the image to be measured, and the coordinates of the upper left corner of the template image in the template image respectively; Establish the corresponding relationship between the pair of image to be measured and template image with the closest coordinates.

6. The method of claim 1, wherein: In step two, the method for calculating the image similarity s between the image to be measured and the template image with corresponding relationship is as follows: s = a x S1 + b x S 边缘 wherein a and b are both proportional coefficients, and a+b=1, S1 represents the similarity between the to-be-tested sub-image and the template sub-image, S 边缘 represents the similarity between the edge image of the to-be-tested sub-image and the edge image of the template sub-image.

7. The method of claim 6, wherein: The setting method of the proportion coefficient a is as follows: Calculate the average gray scale of the test image and the template image, respectively denoted as aver g , aver m ; The absolute value of the difference between the average gray scales is recalculated, and when the absolute value of the difference is greater than a numerical value A, a takes the smaller value in When the difference is less than or equal to the numerical value A, a takes the larger value in ​ 8. The method of claim 1 or 6, wherein: The calculation method of the similarity includes cosine similarity, Pearson correlation coefficient similarity, SSIM structural similarity, and normalized mutual information similarity.

9. The method of claim 8, wherein: The value of A is 30-150.

10. The method of claim 1, wherein: In step four, the method for judging whether the image size of the image to be measured in the abnormal image set is less than or equal to the threshold value is as follows: Calculate the area or diagonal length of the image to be measured, and judge whether the value is less than the threshold value.

Images