Surface defect detection system

Abstract

To detect a defect in a surface such as an abnormality in an applied coating.SOLUTION: An inspection system for detecting a defect in a surface uses a pattern having different colors or darkness in the surface. Light radiates the pattern on the surface, and the pattern and any defect on the surface are thus reflected and are captured for the analysis of an image. After that, the processor separates the pattern from the image to identify the position of any defect on the surface.SELECTED DRAWING: Figure 1

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001]

[0001] This patent application claims the benefit of U.S. patent application Ser. No. 16 / 999,361, filed Aug. 21, 2020, which is incorporated by reference in its entirety herein. [Technical field]

[0002] This invention relates generally to inspection systems, and more particularly to identifying defects on surfaces. [Background technology]

[0003]

[0003] Detecting defects on a surface can be difficult and time consuming. One example where defect detection is important and difficult involves painted surfaces. When manufacturing products having painted surfaces, it is important to ensure that the painted surface is evenly covered with paint and that there are no anomalies in the paint coverage. However, using manual inspection can be unreliable and costly. Therefore, an improved surface inspection system is desired. Summary of the Invention

[0004]

[0004] An inspection system for detecting defects on a surface, such as anomalies in a paint coating, is described. The system reflects a pattern from the surface and separates the pattern from the reflected image to reveal any defects. Low rank and sparse matrices may be used to separate the pattern from the defects. The locations of the defects on the surface may then be displayed for an operator to further evaluate the surface. The invention may also include any other aspect described below in the specification or the accompanying drawings, and any combination thereof.

[0005]

[0005] The present invention can be more fully understood from the following description taken in conjunction with the drawings, in which: [Brief description of the drawings]

[0006] [Figure 1]FIG. 1 is a schematic diagram of a surface defect detection system. [Diagram 2] Figure 2 is a reflected image from a surface defect detection system. [Diagram 3] FIG. 3 is the image of FIG. 2 after applying a noise filter to the image. [Figure 4] FIG. 4 is a low-rank matrix of the image in FIG. [Diagram 5] FIG. 5 is a sparse matrix of the image in FIG. [Figure 6] FIG. 6 is a statistical analysis of the image of FIG. 5 showing the location of defects in the image. [Figure 7] FIG. 7 is a display output of a reflected image showing the location of the defect in the reflected image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007]

[0013] One problem solved by the improved system herein is identifying and locating defects 12 on a surface 10 reflected 24 from a painted surface 10 of a vehicle body. Thus, uneven paint coverage or other paint coverage anomalies may be automatically identified and located. The defect detection system may improve inspection speed and lower inspection costs. Lower costs may be possible due to reduced amounts of data required to analyze the images and reduced image acquisition requirements. It is also desirable for such a system to minimize false positives and provide acceptable accuracy in identifying actual defects.

[0008]

[0014] In a typical manufacturing facility, the surfaces of manufactured products, such as the quality of car paint, are evaluated by experts. However, this can be a costly and labor-intensive process that may require multiple inspection lines to meet mass production. Also, the judgements made by experts regarding the quality of the surface are subjective and therefore the classification results may be inconsistent. With developments in the field of machine learning algorithms for vision-based systems, it may be possible to develop automated defect detection solutions for car paint quality inspection or other surface quality inspection. Challenges for such systems include the difficulty of acquiring images of the right quality, which typically cannot be achieved by standard cameras. For example, providing consistent and acceptable lighting conditions is difficult due to the high reflection coefficient of painted surfaces. Thus, image acquisition is typically based on different principles in current solutions. One such technique is Phase Measuring Deflectometry (PMD), where topographical information of a specular surface is acquired by analyzing the reflection of a structured light source. This is done by first displaying a stripe pattern on an LCD screen located a fixed distance from the test surface and then capturing the reflected pattern with a camera. To detect defects, existing algorithms require multiple images (e.g., up to 20 images) generated with a particular phase-shifting pattern, which makes such systems costly and time-consuming to use.

[0009]

[0015] In contrast to automated defect detection systems such as PMD, the improved automated system herein may utilize as little as one image to identify defects on a surface. The improved system herein may also be used without the need to explicitly train the system, since each sample may be analyzed independently. The basic structure of an automated system for detecting surface defects is shown in FIG. 1. As shown, a pattern 14 having areas 16A, B of different colors or darkness faces a surface 10 having a defect 12 thereon. A light source 18 illuminates the surface 10, causing the pattern 14 to be reflected 24 by the surface 10. While the pattern 14 and the light source 18 can be separate from each other (e.g., a static placard and a lamp), it may be preferable for the pattern 14 and the light source 18 to be emitted together from a single device, such as a display screen 20 (e.g., an LCD). The reflection 24 of the pattern 14 is then received by an imaging device 22, such as a camera. The received reflection 24 may then be automatically analyzed by a processor 26, as further described below, to isolate regions of different color or darkness 16A,B from the reflection 24 and then identify defects 12 in the reflection 24. After identifying the location of any defects 12, the system may display the defect locations on an output device 28, such as a separate display screen.

[0010]

[0016] Preferably, the pattern 14 has either vertical or horizontal linear stripes 16A,B of different colors or shades that alternate with each other and extend across the pattern 14. Thus, the reflected image 24 includes both the pattern 14 and any defects 12 in the surface 10, which are typically small in size. The improved defect detection system may use a low-rank sparse matrix decomposition, where the low-rank component represents the sinusoidal stripe pattern 14 and the sparse matrix includes the defects 12 and the surrounding noise. Statistical anomaly detection methods may then be used to identify the location of the defects 12 on the scanned surface 10.

[0011]

[0017] An example of a reflectance image 24 having a pattern 14 and a number of defects 12 (which are marked with boxes 30 for illustrative purposes) is shown in Figure 2. In this case, the pattern 14 includes horizontal regions 16A,B of alternating light and dark regions 16A,16B. The image 24 is preferably analyzed as a matrix of pixels, where each pixel has a value defined by its color or darkness. In the case of Figure 2, M∈R n 1 ×n The relationship M = L0 + S0 may represent the pixel values ​​of the image 24. Since the image 24 is grayscale, in this example there is only one channel. A particular feature of the matrix M, or any other reflectance image generated using a stripe pattern, is that it has low-rank components, i.e., patterns 14, and sparse components representing anomalies 12. The original reflectance image 24 may therefore be represented by the relationship M = L0 + S0, where L0 and S0 represent the low-rank and sparse matrices, respectively. In other words, the values ​​of the low-rank matrix and the sparse matrix may be added together to arrive at the matrix of the original reflectance image 24. Preferably, the values ​​of the matrix are pixel values ​​(i.e., the darkness or color value of each pixel), so that when corresponding pixel values ​​in the two matrices are added together, the result is the pixel value of the corresponding pixel in the original reflectance image 24.

[0012]

[0018] The matrix relationship can be reduced to finding the matrices L and S that solve the following optimization problem:

[0013]

number

[0014]

[0019] In relation (1),

[0015]

number

[0016] denotes the nuclear norm, which is defined as the sum of the singular values ​​of a matrix. For a non-square matrix A, A T can represent the transpose, so that the square matrix K:=A TA. In this case, the singular values ​​of A can be defined as the square roots of the eigenvalues ​​of matrix K. The karyotype norm is a good convex lower approximation of the rank function, so it can be used in optimization problems to find low-rank matrices. In the system herein, the karyotype norm helps to find horizontal (or vertical) patterns 16A, B in an image 24. Meanwhile,

[0017]

number

[0018] in the case of,

[0019]

number

[0020] teeth,

[0021]

number

[0022] of a matrix viewed as a vector in

[0023]

number

[0024] which in turn helps to identify the sparse components and therefore the anomalies 12 in the reflectance image 24. Thus, when optimizing the low-rank matrix and the sparse matrix, the processor 26 may minimize the kernel type norm of the low-rank matrix and the kernel type norm of the sparse matrix.

[0025]

number

[0026] In particular, when optimizing the matrices, different properties of the first and second matrices may be taken into account.

[0027]

number

[0028] It would be desirable to minimize

[0029]

[0020] The above formulation is called Robust Principal Component Analysis (RPCA), which can be solved via a tractable convex optimization problem. Under certain conditions, the matrices L0 and S0 can be exactly recovered. These conditions, roughly speaking, are that the low-rank matrix L0 should not be sparse, and the sparse matrix S0 should not be low-rank. These conditions are explained in more detail below.

[0030]

[0021] Several generalizations and algorithms for efficiently computing sparse and low-rank matrices using RPCA can be used. One possible approach is Principal Component Pursuit by Alternating Directions, where the following augmented Lagrangian can be defined:

[0031]

number

[0032]

[0022] Wherein,

[0033]

number

[0034] denotes the Frobenius norm. For low rank and sparse decomposition problems, the alternating steps are

[0035]

number

[0036] These have closed form solutions. More explicitly,

[0037]

number

[0038] is S τ (x) = sign(x)max(|x|-τ,0), which can be extended to matrices by applying it to each element. In this case, we get:

[0039]

number

[0040] Here, D μ (X) is D τ (X)=US τ (Σ)V * Denotes the singular value thresholding operator given by, where X=UΣV * is an arbitrary singular value decomposition. Thus, the algorithm may first minimize l over L (with S fixed), then minimize l over S (with L fixed), and finally update the Lagrangian multiplier matrix Y based on the residual MLS. This strategy may be summarized as follows:

[0041] [Table 1]

[0042] Here, the convergence criterion is δ=10 -7 in

[0043]

number

[0044] It can be set by:

[0045]

[0023] Images acquired using reflected patterns are typically incomplete and may contain undesirable effects from the environment, such as lightning, noise, and the like. Therefore, it may be preferable to apply a denoising filter to the reflected image 24 before using the low-rank and sparse matrices to separate the pattern 14 from the reflected image 24. A simple denoising method may be used based on simple averaging, where the value of a pixel may be replaced with the average of similar pixels, where similar pixels are not necessarily pixels close to the particular pixel. This method, called non-local means denoising, may help to suppress noise while keeping the defect information more visible. Applying such a filter to FIG. 2 may result in the image of FIG. 3. Thus, the first two steps of the method performed by the processor 26 may include applying a non-local means denoising filter and applying a low-rank sparse decomposition to the filtered image. FIG. 4 illustrates a low-rank matrix with the pattern 14 and without the defect 12, and FIG. 5 illustrates a sparse matrix with the defect 12 and without the pattern 14.

[0046]

[0024] For the low-rank sparse decomposition to be completely valid, the sparse components should not contain any low-rank components. However, for a surface 10 having defects 12, this condition does not always hold, and therefore some information from certain noise components and the background, i.e., the pattern 12, may leak into the sparse matrix. Therefore, it is not desirable to apply a simple thresholding decision rule to identify the defects. Instead, statistical properties of the sparse matrix may be used by the processor 26 to identify the location of the defects 12. In this regard, each column of the sparse matrix may be treated as a sample, and thus, for the sample under consideration, the column may be defined as n2=976. The sample mean is

[0047]

number

[0048] Then, the distance of each sample, for i=1,...,n2, can be defined as

[0049]

number

[0050] That is, the values ​​(e.g., pixel values) of the portions of each column are compared to each other to identify portions in the column that have values ​​that are distinctly different from the other portions in that column. In a preferred embodiment, the regions used for defect analysis may be lines of pixels that extend across the reflected image 24, with the pixels in each line being compared to the other pixels in that line. Preferably, the regions used for defect analysis are oriented transversely to the alternating regions of the pattern. For example, in this example, the pattern regions 16A,B are oriented horizontally, while the defect analysis regions are oriented vertically (i.e., columnar). In this way, if a fraction of the pattern leaks into the sparse matrix, the remaining portions of the pattern 14 can be screened out during defect analysis due to the transverse orientation of the defect analysis regions. FIG. 6 shows the distance of each column vector, and as shown, the error is noticeable at the location of the defect 12. If Chebyshev's inequality for 3a (3 standard deviations) is applied and errors are identified that have deviations from the remaining samples, then a column 32, illustrated in FIG. 7, may be displayed on the output display screen 28 to indicate the location of the defect 12. Other anomaly detection techniques, such as Local Outlier Factor (LOF), which compares the deviation of each sample to its neighbors, may also be used to detect the defect 12. Note that the display output of column 32 only indicates the location of the defect 12 along the axis corresponding to the defect analysis area (column in this case). If it is desired to identify the location of the defect 12 along both the x-axis and the y-axis, further analysis is required, which may not be necessary or desirable in all cases. However, if desired, further location identification may be achieved by rotating the pattern 14 and data analysis and repeating the above process.

[0051]

[0025] Although preferred embodiments of the present invention have been described, it should be understood that the present invention is not so limited and modifications may be made without departing from the invention herein. Although each embodiment described herein may refer only to certain features and may not specifically refer to all features described with respect to other embodiments, it should be recognized that features described herein may be interchangeable, even if reference to a particular feature is not made, unless otherwise described. It should also be understood that the advantages described above are not necessarily the only advantages of the present invention, and that it is not necessarily expected that all of the described advantages will be achieved in all embodiments of the present invention. The scope of the present invention is defined by the appended claims, and all apparatus and methods that fall within the meaning of the claims, either literally or by the doctrine of equivalents, are intended to be embraced therein.

Claims

1. 1. A system for detecting surface defects, comprising: a surface having a defect thereon; a pattern facing said surface having areas of different color or darkness; a light source for illuminating said surface; an imaging device for receiving a reflection of the pattern from the surface; a processor for identifying the defects in the reflection of the pattern after isolating the regions of different color or darkness from the reflection of the pattern; an output device for displaying the location of the defect on the surface; A system comprising:

2. The system of claim 1 , wherein the pattern comprises alternating areas of the different colors or shades extending across the pattern.

3. 3. The system of claim 2, wherein the processor divides the reflection of the pattern into a plurality of regions, each region extending across the reflection of the pattern, and the processor, after isolating the alternating regions of different color or darkness from the reflection of the pattern, compares portions within each region to one another to identify defects in one of the plurality of regions.

4. 4. The system of claim 3, wherein the plurality of regions are oriented transversely to the alternating regions, whereby the plurality of regions screen out any remaining portions of the alternating regions that slip through separations of the alternating regions from the reflection of the pattern.

5. 2. The system of claim 1, wherein the processor divides the reflection of the pattern into a plurality of regions, each region extending across the reflection of the pattern, and the processor, after isolating the regions of different color or darkness from the reflection of the pattern, compares portions within each region to one another to identify defects in one of the plurality of regions.

6. The system of claim 5 , wherein the location of the defect is displayed by the output device along an axis corresponding to the plurality of regions.

7. The system of claim 5 , wherein each of the regions comprises a line of pixels, and the portions comprise the pixels within each line.

8. The system of claim 1 , wherein the processor separates the reflections of the pattern into a first matrix containing the regions of different color or darkness and a second matrix containing the defects.

9. The system of claim 8 , wherein the first matrix and the second matrix each define values ​​that when summed comprise a representation of the reflectance of the pattern.

10. 10. The system of claim 9, wherein the values ​​in the first matrix and the second matrix are values ​​of corresponding pixels in the first matrix and the second matrix.

11. The system of claim 8 , wherein the processor optimizes the first matrix and the second matrix.

12. The system of claim 11 , wherein the first matrix comprises a low rank matrix and the second matrix comprises a sparse matrix.

13. The system of claim 12 , wherein the processor minimizes a karyon norm of the low-rank matrix.

14. The processor [0010] The system of claim 12 , wherein:

15. 13. The system of claim 12, wherein the processor minimizes a first property of the low-rank matrix and a second property of the sparse matrix, the first property and the second property being distinct from one another.

16. The first characteristic comprises a karyotype norm and the second characteristic comprises [0025] The system of claim 15 , comprising:

17. 12. The system of claim 11, wherein the processor minimizes a first property of the first matrix and a second property of the second matrix, the first property and the second property being distinct from one another.

18. The system of claim 1 , wherein the processor filters noise from the reflection of the pattern prior to isolating the regions of different color or darkness from the reflection of the pattern.

19. The system of claim 1 , wherein the pattern and the light source are emitted from a display screen.

20. The system of claim 4 , wherein the surface is painted and the defect is an anomaly in the paint coverage.

Images