Surface Defect Detection System

Abstract

An inspection system for detecting defects on a surface is provided. The system uses a pattern having different colors or shades facing the surface. Light illuminates the pattern onto the surface, causing the pattern and any defects on the surface to be reflected and captured for image analysis. A processor then isolates the pattern from the image to identify the location of any defects on the surface.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001]

[0001] This patent application claims the benefit of U.S. Patent Application No. 16 / 999,361, filed August 21, 2020, which is incorporated by reference in its entirety. [Technical Field]

[0002] This invention relates generally to inspection systems, and more particularly to identifying defects on surfaces. [Background technology]

[0003] Detecting defects on a surface can be difficult and time-consuming. One example where defect detection is important and challenging relates to painted surfaces. When manufacturing products having painted surfaces, it is important to ensure that the painted surface is evenly covered with paint and that there are no anomalies in the paint coverage. However, using manual inspection can be unreliable and costly. Therefore, an improved surface inspection system is desirable. Summary of the Invention

[0004] An inspection system for detecting defects on a surface, such as anomalies in a paint coating, is described. The system reflects a pattern from the surface and separates the pattern from the reflected image to reveal any defects. Low-rank and sparse matrices may be used to separate the pattern from the defects. The locations of the defects on the surface may then be displayed for an operator to further evaluate the surface. The invention may also include any other aspect described below in the specification or accompanying drawings, and any combination thereof.

[0005] The present invention can be more fully understood from the following description taken in conjunction with the drawings in which: [Brief explanation of the drawings]

[0006] [Figure 1]FIG. 1 is a schematic diagram of a surface defect detection system. [Figure 2] Figure 2 is a reflected image from a surface defect detection system. [Figure 3] Figure 3 is the image of Figure 2 after applying a noise filter to the image. [Figure 4] FIG. 4 is a low-rank matrix of the image in FIG. [Figure 5] Figure 5 is a sparse matrix of the image in Figure 3. [Figure 6] FIG. 6 is a statistical analysis of the image of FIG. 5 showing the location of defects in the image. [Figure 7] FIG. 7 is a display output of a reflected image showing the location of the defect in the reflected image. DETAILED DESCRIPTION OF THE INVENTION

[0007]

[0013] One problem solved by the improved system herein is identifying and locating defects 12 on a surface 10 reflected 24 from the painted surface 10 of a vehicle body. Thus, uneven paint coverage or other paint coverage anomalies can be automatically identified and located. The defect detection system can improve inspection speed and lower inspection costs. Lower costs can be made possible by the reduced amount of data needed to analyze the images and reduced image acquisition requirements. It is also desirable for such a system to minimize false positives and provide acceptable accuracy in identifying actual defects.

[0008]

[0014] In a typical manufacturing facility, the surfaces of manufactured products, such as the quality of automotive paint, are evaluated by experts. However, this can be a costly, labor-intensive process that may require multiple inspection lines to meet mass production. Furthermore, the judgments made by experts regarding surface quality are subjective, and therefore, classification results can be inconsistent. Advances in machine learning algorithms for vision-based systems may enable the development of automated defect detection solutions for automotive paint quality inspection or other surface quality inspection. Challenges for such systems include the difficulty of acquiring images of adequate quality, which typically cannot be achieved with standard cameras. For example, providing consistent and acceptable lighting conditions is difficult due to the high reflectivity of painted surfaces. Therefore, image acquisition is typically based on different principles than current solutions. One such technique is phase measuring deflectometry (PMD), in which topographic information of a specular surface is acquired by analyzing the reflection of a structured light source. This is done by first displaying a stripe pattern on an LCD screen located a fixed distance from the test surface and then capturing the reflected pattern with a camera. To detect defects, existing algorithms require multiple images (e.g., up to 20 images) generated with a particular phase-shift pattern, which makes such systems costly and time-consuming to use.

[0009]

[0015] In contrast to automated defect detection systems such as PMDs, the improved automated system herein can utilize as little as one image to identify defects on a surface. The improved system herein can also be used without the need to explicitly train the system, since each sample can be analyzed independently. The basic structure of an automated system for detecting surface defects is shown in FIG. 1. As shown, a pattern 14 having regions 16A, B of different colors or darkness faces a surface 10 having a defect 12 thereon. A light source 18 illuminates the surface 10, causing the pattern 14 to be reflected 24 by the surface 10. While the pattern 14 and the light source 18 can be separate from one another (e.g., a static placard and a lamp), it may be preferable for the pattern 14 and the light source 18 to be emitted together from a single device, such as a display screen 20 (e.g., an LCD). The reflection 24 of the pattern 14 is then received by an imaging device 22, such as a camera. The received reflection 24 may then be automatically analyzed by a processor 26, as further described below, to isolate regions 16A, B of different color or darkness from the reflection 24 and then identify defects 12 in the reflection 24. After identifying the location of any defects 12, the system may display the defect locations on an output device 28, such as a separate display screen.

[0010]

[0016] Preferably, the pattern 14 has either vertical or horizontal linear stripes 16A, B of different colors or shades that alternate with one another and extend across the pattern 14. The reflected image 24 therefore includes both the pattern 14 and any defects 12 in the surface 10, which are typically small in size. The present improved defect detection system may use low-rank sparse matrix decomposition, where the low-rank component represents the sinusoidal fringe pattern 14 and the sparse matrix includes the defects 12 and ambient noise. Statistical anomaly detection methods may then be used to identify the locations of the defects 12 on the scanned surface 10.

[0011]

[0017] An example of a reflectance image 24 having a pattern 14 and multiple defects 12 (marked by boxes 30 for illustrative purposes) is shown in Figure 2. In this case, the pattern 14 includes horizontal regions 16A,B of alternating light and dark regions 16A, 16B. The image 24 is preferably analyzed as a matrix of pixels, with each pixel having a value defined by its color or darkness. In the case of Figure 2, M∈R n 1 ×n The relationship between M and S may represent pixel values ​​of image 24. Image 24 is grayscale, so in this example there is only one channel. A particular feature of matrix M, or any other reflectance image generated using a stripe pattern, is that the matrix has low-rank components, i.e., patterns 14, and sparse components representing anomalies 12. Thus, original reflectance image 24 may be represented by the relationship M=L0+S0, where L0 and S0 represent low-rank and sparse matrices, respectively. In other words, the values ​​of the low-rank and sparse matrices may be added together to arrive at the matrix of original reflectance image 24. Preferably, the matrix values ​​are pixel values ​​(i.e., darkness or color values ​​of each pixel), so that when corresponding pixel values ​​in the two matrices are added together, the result is the pixel value of the corresponding pixel in original reflectance image 24.

[0012]

[0018] The matrix relationship can be reduced to finding the matrices L and S that solve the following optimization problem:

[0013]

number

[0014]

[0019] In relation (1),

[0015]

number

[0016] denotes the nuclear norm, which is defined as the sum of the singular values ​​of a matrix. For a non-square matrix A, A T may represent the transpose, so that the square matrix K:=A TA. In this case, the singular values ​​of A can be defined as the square roots of the eigenvalues ​​of matrix K. Because the karyotype norm is a good convex lower approximation of the rank function, it can be used in optimization problems to find low-rank matrices. In the system herein, the karyotype norm helps to find horizontal (or vertical) patterns 16A, B in an image 24. Meanwhile,

[0017]

number

[0018] in the case of,

[0019]

number

[0020] teeth,

[0021]

number

[0022] of a matrix viewed as a vector in

[0023]

number

[0024] which in turn helps to identify the sparse components and therefore the anomalies 12 in the reflectance image 24. Thus, when optimizing the low-rank matrix and the sparse matrix, the processor 26 may minimize the karyon norm of the low-rank matrix and the karyon norm of the sparse matrix.

[0025]

number

[0026] In particular, when optimizing the matrices, different properties of the first and second matrices may be taken into account.

[0027]

number

[0028] It would be desirable to minimize

[0029]

[0020] The above formulation is called Robust Principal Component Analysis (RPCA), which can be solved via a tractable convex optimization problem. Under certain conditions, the matrices L0 and S0 can be exactly recovered. These conditions roughly state that the low-rank matrix L0 should not be sparse, and the sparse matrix S0 should not be low-rank. These conditions are explained in more detail below.

[0030]

[0021] Several generalizations and algorithms for efficiently computing sparse and low-rank matrices using RPCA can be used. One possible approach is Principal Component Pursuit by Alternating Directions, where the following augmented Lagrangian can be defined:

[0031]

number

[0032]

[0022] Where:

[0033]

number

[0034] denotes the Frobenius norm. For low-rank and sparse decomposition problems, the alternating steps are

[0035]

number

[0036] which have closed form solutions. More explicitly,

[0037]

number

[0038] is S τ (x) = sign(x)max(|x|-τ,0), which can be extended to matrices by applying it to each element. In this case, we get:

[0039]

number

[0040] where D μ (X) is D τ (X)=US τ (Σ)V * represents the singular value thresholding operator given by, where X=UΣV * is an arbitrary singular value decomposition. Thus, the algorithm may first minimize l with respect to L (fixing S), then minimize l with respect to S (fixing L), and finally update the Lagrange multiplier matrix Y based on the residual MLS. This strategy can be summarized as follows:

[0041] [Table 1]

[0042] Here, the convergence criterion is δ=10 -7 in

[0043]

number

[0044] It can be set by:

[0045] Images acquired using a reflected pattern are typically imperfect and may contain undesirable environmental artifacts, such as lightning, noise, and the like. Therefore, it may be preferable to apply a denoising filter to the reflected image 24 before using a low-rank matrix and a sparse matrix to separate the pattern 14 from the reflected image 24. A simple denoising method may be used based on simple averaging, in which a pixel's value may be replaced with the average of similar pixels, where similar pixels are not necessarily pixels close to the particular pixel. This method, called non-local means denoising, may help suppress noise while keeping defect information more visible. Applying such a filter to FIG. 2 may result in the image of FIG. 3. Therefore, the first two steps of the method performed by the processor 26 may include applying a non-local means denoising filter and applying a low-rank sparse decomposition to the filtered image. FIG. 4 illustrates a low-rank matrix with the pattern 14 and without the defect 12, and FIG. 5 illustrates a sparse matrix with the defect 12 and without the pattern 14.

[0046] For a low-rank sparse decomposition to be completely valid, the sparse components should not contain any low-rank components. However, for a surface 10 having defects 12, this condition does not always hold, and therefore, certain noise components and some information from the background, i.e., pattern 12, may leak into the sparse matrix. Therefore, it is not desirable to apply a simple thresholding decision rule to identify defects. Instead, statistical properties of the sparse matrix may be used by the processor 26 to identify the location of defects 12. In this regard, each column of the sparse matrix may be treated as a sample, and thus, for the sample under consideration, the column may be defined as n2 = 976. The sample mean is

[0047]

number

[0048] Then, the distance of each sample, for i=1,...,n2, can be defined as

[0049]

number

[0050] That is, the values ​​(e.g., pixel values) of the portions of each column are compared to each other to identify portions in the column that have values ​​that are distinctly different from the other portions in that column. In a preferred embodiment, the regions used for defect analysis may be lines of pixels that extend across the reflected image 24, with the pixels in each line being compared to the other pixels in that line. Preferably, the regions used for defect analysis are oriented transversely to alternating regions of the pattern. For example, in this example, the pattern regions 16A, B are oriented horizontally, while the defect analysis regions are oriented vertically (i.e., columnar). In this way, if a fraction of the pattern leaks into the sparse matrix, the remainder of the pattern 14 can be screened out during defect analysis due to the transverse orientation of the defect analysis regions. FIG. 6 shows the distances of each column vector, and as shown, the error is significant at the location of the defect 12. If Chebyshev's inequality for 3a (3 standard deviations) is applied and errors are identified that have deviations from the remaining samples, then column 32, as illustrated in FIG. 7, may be displayed on output display screen 28 to indicate the location of defect 12. Other anomaly detection techniques, such as Local Outlier Factor (LOF), which compares the deviation of each sample to its neighbors, may also be used to detect defects 12. Note that the display output of column 32 only indicates the location of defect 12 along the axis corresponding to the defect analysis area (column, in this case). If it is desired to identify the location of defect 12 along both the x-axis and the y-axis, further analysis is required, although this may not be necessary or desirable in all cases. However, if desired, further location identification may be achieved by rotating pattern 14 and data analysis and repeating the above process.

[0051] While preferred embodiments of the present invention have been described, it should be understood that the invention is not so limited and that modifications may be made without departing from the invention herein. While each embodiment described herein may refer only to certain features and may not specifically refer to all features described with respect to other embodiments, it should be recognized that features described herein are interchangeable, even if reference to a particular feature is not made, unless otherwise stated. It should also be understood that the advantages described above are not necessarily the only advantages of the present invention, and that not all of the described advantages are necessarily expected to be achieved in all embodiments of the present invention. The scope of the present invention is defined by the appended claims, and all apparatus and methods that fall within the meaning of the claims, either literally or by the doctrine of equivalents, are intended to be embraced therein. The following is a summary of the claims as originally filed: [1] A system for detecting surface defects, comprising: a surface having a defect thereon; a pattern facing the surface having areas of different color or darkness; a light source for illuminating the surface; an imaging device that receives a reflection of the pattern from the surface; a processor that identifies the defects in the reflection of the pattern after separating the regions of different color or darkness from the reflection of the pattern; an output device for displaying the location of the defect on the surface; A system comprising: [2] The system of [1], wherein the pattern comprises alternating regions of the different colors or darknesses extending across the pattern. [3] The system described in [2], wherein the processor divides the reflection of the pattern into a plurality of regions, each region extending across the reflection of the pattern, and the processor separates the alternating regions of different color or darkness from the reflection of the pattern, and then compares portions within each region to each other to identify defects in one of the plurality of regions. [4] The system of [3], wherein the plurality of regions are oriented transversely to the alternating regions, whereby the plurality of regions screen out any remaining portions of the alternating regions through separation of the alternating regions from the reflection of the pattern. [5] The system of [1], wherein the processor divides the reflection of the pattern into a plurality of regions, each region extending across the reflection of the pattern, and after separating the regions of different color or darkness from the reflection of the pattern, the processor compares portions within each region to each other to identify defects in one of the plurality of regions. [6] The system described in [5], wherein the position of the defect is displayed by the output device along an axis corresponding to the plurality of regions. [7] The system of [5], wherein each of the regions comprises a line of pixels, and the portion comprises pixels within each line. [8] The system of [1], wherein the processor separates the reflections of the pattern into a first matrix containing the areas of different color or darkness and a second matrix containing the defects. [9] The system of [8], wherein the first matrix and the second matrix each define values ​​that, when added together, provide a representation of the reflectivity of the pattern.

[10] The system of [9], wherein the values ​​of the first matrix and the second matrix are values ​​of corresponding pixels in the first matrix and the second matrix.

[11] The system described in [8], wherein the processor optimizes the first matrix and the second matrix.

[12] The system of

[11] , wherein the first matrix comprises a low-rank matrix and the second matrix comprises a sparse matrix.

[13] The system of

[12] , wherein the processor minimizes a karyon norm of the low-rank matrix.

[14] The processor

number

[12] minimizes

[15] The system of

[12] , wherein the processor minimizes a first property of the low-rank matrix and a second property of the sparse matrix, the first property and the second property being different from each other.

[16] The first property comprises a karyotype norm, and the second property comprises:

number

[15] , comprising:

[17] The system described in

[11] , wherein the processor minimizes a first property of the first matrix and a second property of the second matrix, and the first property and the second property are different from each other.

[18] The system of [1], wherein the processor filters noise from the reflection of the pattern before separating the regions of different color or darkness from the reflection of the pattern.

[19] The system of [1], wherein the pattern and the light source are emitted from a display screen.

[20] The system of [4], wherein the surface is painted and the defect is an anomaly in the paint coverage.

Claims

1. 1. A system for detecting surface defects, comprising: a surface having a defect thereon; a pattern facing said surface having areas of different color or darkness, said pattern comprising alternating areas of said different color or darkness extending across said pattern; a light source for illuminating the surface; an imaging device that receives a reflection of the pattern from the surface; separating a reflected image of the pattern obtained from the imaging device into a first matrix showing the alternating regions of different color or darkness and a second matrix containing defects; dividing the separated reflected image into a plurality of regions, each region extending transversely to alternating regions of the pattern; comparing portions within each of the plurality of regions to one another to identify the defect in one of the plurality of regions; a processor configured to: an output device for displaying the location of the defect on the surface; Equipped with the first matrix and the second matrix each define values ​​that, when summed, comprise a representation of a reflected image of the pattern; the processor optimizes the first matrix and the second matrix; The system, wherein the first matrix comprises a low-rank matrix and the second matrix comprises a sparse matrix.

2. The system of claim 1 , wherein the location of the defect is displayed by the output device along axes corresponding to the plurality of regions.

3. The system of claim 1 , wherein each of the plurality of regions comprises a line of pixels, and the portion comprises pixels within each line.

4. The system of claim 1 , wherein the values ​​in the first matrix and the second matrix are values ​​of corresponding pixels in the first matrix and the second matrix.

5. The system of claim 1 , wherein the processor minimizes a karyon norm of the low-rank matrix.

6. The processor [Equation 1] The system of claim 1 , wherein:

7. The system of claim 1 , wherein the processor minimizes a first property of the low-rank matrix and a second property of the sparse matrix, the first property and the second property being different from one another.

8. The first property comprises a karyotype norm and the second property comprises [Equation 2] The system of claim 7, comprising:

9. The system of claim 1 , wherein the processor minimizes a first property of the first matrix and a second property of the second matrix, the first property and the second property being different from one another.

10. The system of claim 1 , wherein the processor filters noise from the reflected image of the pattern before isolating the regions of different color or darkness from the reflected image of the pattern.

11. The system of claim 1 , wherein the pattern and the light source are emitted from a display screen.

Images