Image defect detection method based on non-rigid transformation

Abstract

The invention relates to an image defect detection method based on non-rigid transformation, and the method comprises the following steps: 1, selecting a zero-defect image as a reference image for a current working point; 2, for the to-be-detected image of the current working point position, performing non-rigid transformation on the zero-defect image, so that the zero-defect image and the to-be-detected image are strictly aligned; and 3, comparing the aligned zero-defect image with the current detection image to determine the position and size of the defect. The method does not depend on collection of defective samples, and the problem of overfitting in the prior art due to the fact that defects cannot be exhaustive is effectively avoided; the problem that mass data required in the prior art cannot be effectively collected due to low defect occurrence probability is effectively avoided.

Description

Technical field:

[0001] The present invention belongs to the technical field of visual inspection, and in particular relates to an image defect detection method based on non-rigid transformation. Background technology:

[0002] In modern manufacturing, product quality control is an important manifestation of corporate competitiveness. With the increasing complexity of product design and the diversification of material types, traditional optical inspection methods have been unable to meet the needs of high precision and high efficiency. These methods usually rely on visible light or other simple light sources, which limits their ability in deep defect detection and material analysis. In addition, the manual inspection process is not only inefficient, but also prone to human errors, resulting in potential quality problems not being discovered in time.

[0003] X-ray technology, with its excellent penetration and sensitivity to different materials, has gradually been applied to industrial inspection fields such as aerospace, electronic products, and automotive parts. It can penetrate deep into the material and reveal defects such as pores, cracks, and inclusions that are difficult to detect through surface observation. This makes X-ray an indispensable quality assurance tool in high-end manufacturing. With the rise of Industry 4.0 and smart manufacturing, how to combine X-ray technology with automated and intelligent inspection systems has become a new trend in the development of the industry.

[0004] In the existing technology, X-ray defect inspection often relies on machine learning of a large number of defective samples, which brings two disadvantages: First, in many industrial scenarios, the occurrence of defects is a low-probability event. Faced with the demand for massive data by deep neural networks, collecting a large number of defect samples is almost an impossible task. Second, the inherent defects of deep neural networks, such as overfitting, often lose the ability to detect defects that deviate from the data distribution. On the production line, the forms of defects cannot be exhausted. Using the known defect forms reflected by limited data to train an intelligent algorithm cannot effectively deal with unknown defect forms, and there is a risk of missed judgments in actual use. Missed judgments are an unacceptable problem in quality control management, so an effective technical means is urgently needed to solve the above problems. Summary of the invention:

[0005] The technical problem to be solved by the present invention is to provide an image defect detection method based on non-rigid transformation. The method does not rely on the collection of defect samples, effectively avoiding the overfitting problem of the prior art caused by the inability to exhaust defects; and effectively avoiding the problem that the massive data required by the prior art cannot be effectively collected due to the low probability of defect occurrence.

[0006] The technical solution of the present invention is to provide an image defect detection method based on non-rigid transformation, comprising the following steps:

[0007] Step 1: For the current working point, select a zero-defect image as a reference image;

[0008] Step 2: For the image to be inspected at the current working point, perform non-rigid transformation on the zero-defect image so that the zero-defect image and the image to be inspected are strictly aligned;

[0009] Step 3: Compare the difference between the aligned zero-defect image and the current inspection image to determine the location and size of the defect.

[0010] As a preferred embodiment, the specific operation of the second step is as follows:

[0011] Step 1: The zero-defect image of the current working point is recorded as I0, and the image to be detected at the current working point is recorded as I;

[0012] Step 2: Extract the feature points of the zero-defect image I0 and the feature points of the image to be detected I; and record the corresponding feature point sets as C0 and C respectively;

[0013] Step 3: Perform rigid matching on the feature point sets C0 and C obtained in step 2, and output the transformation matrix T from image I0 to image I;

[0014] Step 4: Use the matrix T obtained in step 3 to perform a rigid transformation on image I0 to obtain a new image I0_1;

[0015] Step 5: Perform non-rigid matching on image I0_1 and image I; the operation is as follows: divide image I into N*N blocks, record the size of the image block as Δ, and estimate the translation range of the image block on the target image as β=ρΔ;

[0016] Step 6: Define the search range of non-rigid matching for each image block on image I0_1. The search range is represented by the set S:

[0017] S={(θ,δx,δy)|θ min <θ<θ max -β<δ x <β,-β<δ y <β}; where θ refers to the matching angle between the corresponding image block and the matching image block on the target image; δ x Refers to the offset in the x direction between the corresponding image block and the matching image block on the target image; δ y Refers to the offset in the y direction between the corresponding image block and the matching image block on the target image;

[0018] Step 7: For each image block (i, j), according to the following matrix M ij Perform affine transformation on the image to obtain a new block image Iij , take the size of the transformed image as Δ×Δ;

[0019]

[0020] Step 8: Let s = (θ, δ x , δ y ), for s∈S, according to the following transformation matrix T(s), the image I0_1 is transformed radially to obtain the new image I0 ij , take the transformed image size as Δ×Δ;

[0021]

[0022] Step 9: Calculate image block I ij and image block I0 ij The similarity of ij，s ;

[0023] Step 10: Repeat step 8 and step 9 until the entire search range S is traversed, and the maximum similarity in the search range S is taken as the image block I ij The similarity is:

[0024] Step 11: Repeat step 7 to step 10 until all image blocks are traversed. After traversal, output P = {(Δi, Δj, s ij )|i∈N,j∈N};

[0025] Step 12: Take pt src =(Δi, Δj, 1.0) T , where the superscript T represents transposition; according to the transformation matrix T(s ij ), we can get: pt dst =T(s ij )*pt src ;

[0026] Step 13: Point set pt src and pt dst As control points, the image pixel grid is deformed to obtain the final non-rigid transformation parameters of each pixel;

[0027] Step 14: Perform trilinear interpolation on the deformed image pixel grid to obtain the final image I_final after non-rigid transformation;

[0028] Step 15: Set I0_1 to I_final and repeat step 5 to step 14 until the images are fully aligned.

[0029] Preferably, in Step 2, the method for extracting feature points is a traditional operator in the field of computer vision, such as SIFT, SURF, OBB; or a deep neural network.

[0030] Preferably, in Step 3, the rigid matching algorithm is the ICP algorithm.

[0031] Preferably, ρ is determined according to the variation range between the parts to be inspected and the zero-defect parts. In the casting industry, ρ is 0.2, and in the PCB circuit board industry, ρ is 0.4.

[0032] Preferably, in Step 9, the similarity is calculated by any one of the sum of square differences (SSD), mean square difference (MSD), and structural similarity (SSIM), Structural Similarity Index Model.

[0033] Furthermore, in Step 13, the image pixel grid deformation method adopts the Laplace editing method.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] The present invention provides an image defect detection algorithm based on non-rigid transformation. The method does not rely on the collection of defect samples, effectively avoiding the overfitting problem of the prior art caused by the inability to exhaust defects; and effectively avoiding the problem that the massive data required by the prior art cannot be effectively collected due to the low probability of defect occurrence. Description of the drawings:

[0036] Figure 1 This is a schematic diagram showing the difference between an aligned zero-defect image and a test image using the SSIM algorithm in an embodiment of the present invention.

[0037] Figure 2 Schematic diagram of Step 13 in an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram showing the difference between an aligned zero-defect image and a test image using an SSD method according to another embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of an image block of 9×9 in an embodiment of the present invention.

[0040] Figure 5 It is a schematic diagram of taking an image block of 15×15 in another embodiment of the present invention. Specific implementation method:

[0041] The present invention will be further described below with reference to the accompanying drawings:

[0042] A method for detecting image defects based on non-rigid transformation comprises the following steps:

[0043] Step 1: For the current working point, select a zero-defect image as a reference image;

[0044] Step 2: For the image to be inspected at the current working point, perform non-rigid transformation on the zero-defect image so that the zero-defect image and the image to be inspected are strictly aligned;

[0045] Step 3: Compare the difference between the aligned zero-defect image and the current inspection image to determine the location and size of the defect.

[0046] Among the above three steps, the third step has relatively mature technology that can be used for reference; the invention content given in this invention is mainly for the second step. In this embodiment, the specific operation of the second step is as follows:

[0047] Step 1: The zero-defect image of the current working point is recorded as I0, and the image to be detected at the current working point is recorded as I;

[0048] Step 2: Extract the feature points of the zero-defect image I0 and the feature points of the image to be detected I; and record the corresponding feature point sets as C0 and C respectively; the method of extracting feature points is a traditional operator in the field of computer vision, such as SIFT, SURF, OBB, etc.; of course, it can also be a deep neural network;

[0049] Step 3: Perform rigid matching on the feature point sets C0 and C obtained in step 2, and output the transformation matrix T from image I0 to image I. In this embodiment, the rigid matching algorithm is the ICP algorithm, and the full name of the ICP algorithm is Iterative Closest Points, which is a relatively mature method. After the ICP algorithm, the transformation matrix T from image I0 to image I can be output.

[0050] Step 4: Use the matrix T obtained in step 3 to perform a rigid transformation on image I0 to obtain a new image I0_1;

[0051] Step 5: Perform non-rigid matching on image I0_1 and image I; the operation is as follows: divide image I into N*N blocks, let the size of the image block be Δ, and estimate the translation range of the image block on the target image as β=ρΔ; where ρ is determined according to the variation range between the part to be inspected and the zero-defect part. In the casting industry, ρ is generally taken as 0.2, and in the PCB circuit board industry, ρ is generally taken as 0.4.

[0052] Step 6: Define the search range of non-rigid matching for each image block on image I0_1. The search range is represented by the set S:

[0053] S={(θ,δx,δy)|θ min <θ<θ max , -β<δ x <β,-β<δ y <β}; where θ refers to the matching angle between the corresponding image block and the matching image block on the target image; δ x Refers to the offset in the x direction between the corresponding image block and the matching image block on the target image; δ y Refers to the offset in the y direction between the corresponding image block and the matching image block on the target image;

[0054] Step 7: For each image block (i, j), according to the following matrix M ij Perform affine transformation on the image to obtain a new block image I ij , take the size of the transformed image as Δ×Δ;

[0055]

[0056] Step 8: Let s = (θ, δ x , δ y ), for s∈S, according to the following transformation matrix T(s), the image I0_1 is transformed radially to obtain the new image I0 ij , take the transformed image size as Δ×Δ;

[0057]

[0058] Step 9: Calculate image block I ij and image block I0 ij The similarity of ij，s In this embodiment, the method for calculating the similarity between two images is Structural Similarity Index Model (SSIM). It should be noted that any method involving similarity calculation between two image blocks can achieve the purpose of this step.

[0059] Step 10: Repeat step 8 and step 9 until the entire search range S is traversed, and the maximum similarity in the search range S is taken as the image block I ij The similarity is:

[0060] Step 11: Repeat step 7 to step 10 until all image blocks are traversed. After traversal, output P = {(Δi, Δj, s ij)|i∈N,j∈N};

[0061] Step 12: Take pt src =(Δi, Δj, 1.0) T , where the superscript T represents transposition; according to the transformation matrix T(s ij ), we can get: pt dst =T(s ij )*pt src ;

[0062] Step 13: Point set pt src and pt dst As control points, the image pixel grid is deformed to obtain the final non-rigid transformation parameters of each pixel; the image pixel grid deformation method adopts the Laplace editing method; Laplace grid deformation is a relatively conventional technology and will not be described in detail; for the purpose of vividly illustrating this step, please refer to Figure 2 , Figure 2 The image on the left is divided into 3×3 blocks. The white points in the figure are the vertices of each image block, and their coordinates are pt src , Figure 2 The white dots in the right image represent the points after being moved, which are the pts calculated in step 12 based on the non-rigid transformation parameters. dst ;

[0063] Step 14: Perform trilinear interpolation on the deformed image pixel grid to obtain the final image I_final after non-rigid transformation;

[0064] Step 15: Set I0_1 to I_final and repeat step 5 to step 14 until the images are fully aligned.

[0065] By using the above method, image defect detection based on non-rigid transformation can be realized. In this embodiment, the SSIM algorithm is used to compare the difference between the aligned zero-defect image and the test image. Figure 1 .

[0066] The image defect detection based on non-rigid transformation involved in the present invention is based on the following premise or basis: the component to be detected, which can be an industrial component or a circuit board, is highly standardized and has little morphological variation. In the implementation case of the present invention, the steering knuckle has this feature.

[0067] In another embodiment of the present invention, compared with the above embodiment, the similarity calculation method in Step 9 is replaced by the sum of square differences (SSD), and the SSD method is used as the image block similarity comparison method to search for non-rigid transformation parameters. This method is fast and can be processed in parallel. The effect can be referred to Figure 3 .

[0068] Figure 4 This is a schematic diagram of an image block of 9×9 in an embodiment of the present invention. Figure 5 It is a schematic diagram of taking an image block of 15×15 in another embodiment of the present invention.

Claims

1. An image defect detection method based on non-rigid transformation, characterized in that: The following steps are included: Step 1: For the current working point, select a zero-defect image as a reference image; Step 2: For the image to be inspected at the current working point, perform non-rigid transformation on the zero-defect image so that the zero-defect image and the image to be inspected are strictly aligned; Step 3: Compare the difference between the aligned zero-defect image and the current inspection image to determine the location and size of the defect.

2. The image defect detection method based on non-rigid transformation according to claim 1, characterized in that: The specific operations of the second step are as follows: Step 1: The zero-defect image of the current working point is recorded as I0, and the image to be detected at the current working point is recorded as I; Step 2: Extract the feature points of the zero-defect image I0 and the feature points of the image to be detected I; and record the corresponding feature point sets as C0 and C respectively; Step 3: Perform rigid matching on the feature point sets C0 and C obtained in step 2, and output the transformation matrix T from image I0 to image I; Step 4: Use the matrix T obtained in step 3 to perform a rigid transformation on image I0 to obtain a new image I0_1; Step 5: Perform non-rigid matching on image I0_1 and image I; the operation is as follows: divide image I into N*N blocks, record the size of the image block as Δ, and estimate the translation range of the image block on the target image as β=ρΔ; Step 6: Define the search range of non-rigid matching for each image block on image I0_1. The search range is represented by the set S: S={(θ,δx,δy)|θ min <θ<θ max , -β<δ x <β,-β<δ y <β}; where θ refers to the matching angle between the corresponding image block and the matching image block on the target image; δ x Refers to the offset in the x direction between the corresponding image block and the matching image block on the target image; δ y Refers to the offset in the y direction between the corresponding image block and the matching image block on the target image; Step 7: For each image block (i, j), according to the following matrix M ij Perform affine transformation on the image to obtain a new block image I ij , take the size of the transformed image as Δ×Δ; Step 8: Let s = (θ, δ x , δ y ), for s∈S, according to the following transformation matrix T(s), the image I0_1 is transformed radially to obtain the new image I0 ij , take the transformed image size as Δ×Δ; Step 9: Calculate image block I ij and image block I0 ij The similarity of ij，s ; Step 10: Repeat step 8 and step 9 until the entire search range S is traversed, and the maximum similarity in the search range S is taken as the image block I ij The similarity is: Step 11: Repeat step 7 to step 10 until all image blocks are traversed. After traversal, output P = {(Δi, Δj, s ij )|i∈N,j∈N}; Step 12: Take pt src =(Δi, Δj, 1.0) T , where the superscript T represents transposition; according to the transformation matrix T(s ij ), we can get: pt dst =T(s ij )*pt src ; Step 13: Point set pt src and pt dst As control points, the image pixel grid is deformed to obtain the final non-rigid transformation parameters of each pixel; Step 14: Perform trilinear interpolation on the deformed image pixel grid to obtain the final image I_final after non-rigid transformation; Step 15: Set I0_1 to I_final and repeat step 5 to step 14 until the images are fully aligned.

3. The image defect detection method based on non-rigid transformation according to claim 2, characterized in that: In Step 2, the method for extracting feature points is traditional operators in the field of computer vision or deep neural networks.

4. The image defect detection method based on non-rigid transformation according to claim 2, characterized in that: In Step 3, the rigid matching algorithm is the ICP algorithm.

5. The image defect detection method based on non-rigid transformation according to claim 2, characterized in that: ρ is determined according to the variation range between the parts to be inspected and the zero-defect parts. In the casting industry, ρ is 0.2, and in the PCB circuit board industry, ρ is 0.

4.

6. The image defect detection method based on non-rigid transformation according to claim 2, characterized in that: In Step 9, the similarity is calculated by any one of the sum of squared differences, mean squared error, or structural similarity.

7. The image defect detection method based on non-rigid transformation according to claim 2, characterized in that: In Step 13, the image pixel grid deformation method adopts the Laplace editing method.

Images