Mirror-like defect detection method and system

Abstract

The present invention discloses a method and system for detecting mirror-like defects, including: collecting a stripe image projected onto a mirror-like surface to be detected; using a defect detection model to detect the initial position and size of the defect according to the stripe image; intercepting the stripe image of the initial position area where the defect is located, calculating the folding phase, and obtaining a folding phase image; unfolding the folding phase image, calculating the gradients in the X and Y directions after unfolding, integrating the gradients in the X and Y directions, and obtaining a depth image; morphologically processing the depth image to obtain the accurate position of the defect. This application uses a single camera plus an LED lamp bead screen to project a stripe image to ensure the acquisition rate, obtains the initial position of the defect in the stripe image through an AI algorithm to ensure the accuracy of defect detection, and uses the preliminary defect position combined with phase unfolding to obtain a local depth image, thereby obtaining a more accurate defect position and size, which not only ensures the detection efficiency but also improves the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a mirror-like defect detection method and system. Background Art

[0002] With the rapid development of the economy, smooth planes or painted surfaces often need to be inspected in many industries. Currently, the commonly used inspection method is mainly visual inspection. The minimum size that the human eye can distinguish at a normal viewing distance (25cm) is about 0.2mm, and the detection rate can reach about 70%~80%. Compared with traditional manual visual inspection, inspection equipment with visual capabilities can complete defect detection tasks faster, more accurately and more stably, and is therefore widely used in the production process.

[0003] For example, in the prior art, the application number is CN 111257338 A and the name is: Mirror and quasi-mirror surface defect detection method, which provides a binocular defect detection method, the core idea of ​​which is to use the image captured by the first camera to obtain a pre-selected point set, and convert any marked point into the world coordinate system, and obtain the three-dimensional point on the projection device according to the absolute phase equality, and combine the world coordinates of the measured object point to obtain the normal vector of the measured object point to obtain the normal vector of the measured object point of the first camera, and use the second camera to perform the same operation. Take the points with a small angle between the two normal vectors as spatial candidate points, traverse each spatial candidate point, and calculate the average value of all spatial candidate points to realize the spatial positioning of the defect area. However, this method needs to perform detection based on the overlapping field of view of the two cameras, the detection area becomes smaller, and the measurement speed is slower.

[0004] For large equipment manufacturing companies, after the product is formed, the outer surface of the product needs to be painted. The spraying effect of the paint surface directly affects the appearance or performance of the product, which is a problem that companies are more concerned about. At present, the detection method of body paint defects usually cannot take into account the speed, detection accuracy, and defect precision at the same time. To this end, this application proposes a mirror-like defect detection method and system. Summary of the invention

[0005] Therefore, the purpose of the present invention is to provide a mirror-like defect detection method and system, which combines AI algorithm with monocular phase deflectometry to detect automobile body defects, uses a single camera plus an LED lamp screen to project a stripe image to ensure the acquisition rate, and uses AI algorithm to ensure the detection accuracy of defects to obtain the initial position of the defect in the stripe image, uses the preliminary defect position combined with phase deflectometry to obtain a local depth image, and performs morphological processing on the local depth image to obtain a more accurate and clear defect.

[0006] In order to achieve the above object, an embodiment of the present invention provides a method for detecting mirror-like defects, comprising the following steps:

[0007] S1, collecting the fringe image projected onto the mirror surface to be detected;

[0008] S2. Detecting the initial position and size of the defect using a defect detection model according to the fringe image;

[0009] S3, intercepting the fringe image of the initial position area where the defect is located, calculating the folding phase, and obtaining a folding phase image;

[0010] S4, unfolding the folded phase image, calculating the X and Y direction gradients after unfolding, and integrating the X and Y direction gradients to obtain a depth image;

[0011] S5. Perform morphological processing on the depth image to obtain the accurate position and size of the defect.

[0012] Further preferably, S1 also includes: placing the mirror to be detected in a detection environment, wherein a specific light source is provided in the detection environment, and the specific light source is irradiated on the mirror to be detected to form a stripe image.

[0013] Further preferably, in S2, the detecting the initial position and size of the defect using a defect detection model according to the fringe image also includes:

[0014] S201, constructing a defect detection model, wherein the defect detection model includes a defect detection sub-model and an anomaly detection sub-model;

[0015] S202, inputting the labeled sample image into a supervised network model for training to generate a defect detection sub-model;

[0016] S203, inputting the unlabeled sample image into the unsupervised network model for training to generate an anomaly detection sub-model;

[0017] S204: Combine the trained defect detection sub-model and anomaly detection sub-model to form a complete defect detection model.

[0018] Further preferably, in S3, the step of intercepting the fringe image of the initial position area where the defect is located further comprises proportionally expanding the initial position area where the defect is located and intercepting the expanded fringe image.

[0019] Further preferably, in S3, when calculating the folding phase, the folding phase of the intercepted area is calculated according to the following formula (2) to obtain a folding phase image;

[0020] (Formula 2)

[0021] in, Represents the pixel point in the nth sinusoidal fringe image The light intensity value at represents the phase shift of the fringe image, and N represents the total number of projected sinusoidal fringe images.

[0022] Further preferably, in S4, the folded phase image is unfolded using a spatial phase unfolding method, and the unfolded X and Y direction gradients are calculated using the following formula (3) to obtain a depth image;

[0023] (Formula 3)

[0024] in, are all world coordinates, express The measured gradient value in the direction, express The measured gradient value in the direction, subscript and are matrix indices of the gradient map.

[0025] Further preferably, the performing morphological processing on the depth image includes:

[0026] Extracting the edges of defects in the depth image;

[0027] After the detection area where the defect is located is binarized, modulation filtering is performed to filter out the area with the largest area in the binarized area;

[0028] The region with the largest area is first closed and then opened to obtain the exact position of the defect.

[0029] The present invention also provides a mirror-like defect detection system, comprising: a specific light source, an image acquisition module, and an image processing module;

[0030] The specific light source is used to project specific light onto the mirror surface to be detected;

[0031] The image acquisition module is used to acquire the fringe image projected onto the mirror surface to be detected;

[0032] The image processing module is used to detect the initial position and size of the defect according to the fringe image;

[0033] intercepting a fringe image of an initial position area where the defect is located, calculating a folded phase, and obtaining a folded phase image;

[0034] Unfolding the folded phase image, calculating the gradients in the X and Y directions after the unfolding, and integrating the gradients in the X and Y directions to obtain a depth image;

[0035] The depth image is morphologically processed to obtain the accurate position and size of the defect.

[0036] Further preferably, the specific light source adopts an LED lamp bead screen, and the LED lamp bead screen includes a control circuit, a PCD board, multiple rows of LED lamp beads (the light source row and column distribution rows: 136, columns: 240, and the LED lamp bead spacing is 3.5mm), an anti-reflection film, a diffusion film and an acrylic board; the control circuit is used to control the LED lamp beads to light up according to the front and rear 1 / 4 stripe sine cycle difference;

[0037] The anti-reflection film, the diffusion film and the acrylic plate are sequentially arranged to scatter and refract the light emitted by the LED lamp beads and then project it onto the mirror surface to be detected to form a sinusoidal fringe image.

[0038] Further preferably, the image processing module further comprises constructing a defect detection model, wherein the defect detection model comprises a defect detection sub-model and an anomaly detection sub-model;

[0039] Among them, the labeled sample images are input into the supervised network model for training to generate a defect detection sub-model;

[0040] Input the unlabeled sample images into the unsupervised network model for training to generate anomaly detection sub-model;

[0041] The trained defect detection sub-model and anomaly detection sub-model are combined to form a complete defect detection model.

[0042] The mirror-like defect detection method and system disclosed in the present application have at least the following advantages over the prior art:

[0043] This application adopts a mirror-like defect detection method and system. Under the premise of ensuring the acquisition rate, a single camera combined with a defect detection model is used to obtain the precise position and size of the defect area through preliminary detection and phase processing of the formed stripe image, which not only improves the detection accuracy but also improves the detection speed. The minimum detectable size is as high as 0.2mm, the detection rate is as high as more than 95%, the missed detection rate is <1%, and the defect classification accuracy is greater than 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic flow chart of the mirror-like defect detection method provided by the present invention.

[0045] Figure 2 A schematic diagram of the partial structure of the mirror-like defect detection system provided by the present invention.

[0046] Figure 3 The defect detection model of the present invention identifies the initial position and size of the defect map.

[0047] Figure 4This is a vertical stripe folded phase diagram in the folded phase diagram mentioned in an embodiment of the present invention.

[0048] Figure 5 This is a horizontal stripe folded phase diagram in the folded phase diagram mentioned in an embodiment of the present invention.

[0049] Figure 6 This is the gradient map in the X direction mentioned in the embodiment of the present invention.

[0050] Figure 7 It is the gradient map in the Y direction mentioned in the embodiment of the present invention.

[0051] Figure 8 It is a depth image after gradient integration in an embodiment of the present invention.

[0052] Fig. 9 The accurate position of the final defect selected by the horizontal stripe image obtained in the embodiment of the present invention.

[0053] Fig.10 The accurate position of the final defect selected by the vertical stripe image obtained in the embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below through the accompanying drawings and specific embodiments.

[0055] like Figure 1 As shown, an embodiment of the present invention provides a mirror-like defect detection method, comprising the following steps:

[0056] S1, collecting the fringe image projected onto the mirror surface to be detected;

[0057] During the inspection, the mirror to be inspected needs to be placed in an inspection environment, in which a specific light source is provided, and the specific light source is irradiated on the mirror to be inspected to form a fringe image. The specific light source generates 8 sinusoidal fringe images in a time-sharing manner, and the phase difference between these images is one quarter of a fringe period, that is, a sinusoidal fringe phase difference of 1 / 4 period corresponds to a phase difference of 90 degrees.

[0058] That is, assuming that one cycle of the sinusoidal stripes contains 8 rows of lamp beads, the stripes with a difference of 1 / 4 of the cycle are lit as follows Figure 2As shown (white means on, black means off). In fact, the light source is distributed in rows and columns as follows: rows: 136; columns: 240, and the spacing between LED beads is 3.5mm. When irradiating, the irradiation is switched according to horizontal stripes and vertical stripes. When the horizontal beads of the LED bead matrix form a stripe pattern according to the sine wave ripples, it is projected onto the mirror to be detected, and the camera performs an image acquisition. When the vertical beads of the LED bead matrix form a stripe pattern according to the sine wave ripples, it is projected onto the mirror to be detected, and the camera performs another image acquisition. The refresh rate of the LED bead screen in this application (trigger response time 10us (MAX) = 10 5 The refresh rate of the LCD display screen (120HZ) is several orders of magnitude higher than that of the LCD display screen (120HZ). Compared with the situation of using the LCD display screen to control the display of the stripe pattern as a whole, controlling the LED lamp beads to independently control a row or a column of lamp beads is more in line with the situation of simulating sinusoidal wave fluctuations, so the phase deflection relationship calculated can be more accurate; on this basis, only a high-speed camera is needed to achieve a faster detection rate.

[0059] Taking the control of the vertical lamp beads of the LED lamp bead matrix as an example, the LED lamp bead matrix has a total of 240 columns. If 8 sinusoidal fringe images are generated, 30 lamp beads are required to change on and off for each image. That is to say, in each image, 15 lamp beads are lit and 15 lamp beads are off; these 15 lit lamp beads and 15 off lamp beads together form a sinusoidal fringe image; the next adjacent fringe image is displayed with a phase difference of 1 / 4 cycle. Thus, 8 fringe images at different phase moments are formed.

[0060] S2. According to the fringe image, a defect detection model is used to detect the initial position and size of the defect; the defect detection model includes a defect detection sub-model and an abnormality detection sub-model;

[0061] Input the labeled sample images into the supervised network model for training to generate a defect detection sub-model;

[0062] Input the unlabeled sample images into the unsupervised network model for training to generate anomaly detection sub-model;

[0063] The trained defect detection sub-model and anomaly detection sub-model are combined to form a complete defect detection model.

[0064] S3, intercept the stripe image of the initial position area where the defect is located, calculate the folded phase, and obtain the folded phase image; when intercepting, the initial position area where the defect is located needs to be expanded proportionally, and the expanded stripe image is intercepted; after a small-scale expansion of the defect position, a small part of the surrounding plane area can be intercepted, and the defect is reconstructed using the intercepted and expanded stripe image to improve the contrast between the defect area and the surrounding plane area (the defect area is brighter and the plane area is darker). It should be noted that when expanding proportionally, the area is evenly expanded to the surrounding area at 1.5 times the initial position area; after expansion, the defect depth is reconstructed. Figure 8 is the reconstructed depth map, that is, Figure 6 and Figure 7 The depth image after gradient integration of the gradient map in ; the black outer circle represents the surrounding plane area, and the white inner circle represents the defect.

[0065] The fringe image projected by the LED lamp screen is reflected by the object to be measured as shown in formula (1):

[0066] I n (x,y)=a(x,y)+b(x,y)*cos(Ф(x,y)+ ) n=1,2,....N (Formula 1)

[0067] in, Indicates n Pixels in a sinusoidal fringe image The light intensity value at represents the background light intensity of the fringe image, represents the modulation degree of the fringe image, represents the folding phase of the fringe image. N represents the total number of projected sinusoidal fringe images.

[0068] The intercepted fringe image is used to obtain the folding phase of the area through formula (2). The obtained folding phase image is as follows: Figure 4 and Figure 5 shown.

[0069] (Formula 2)

[0070] in, represents the phase shift of the fringe image, and N represents the total number of projected sinusoidal fringe images.

[0071] S4. Unfold the folded phase image, calculate the gradients in the X and Y directions after unfolding, and integrate the gradients in the X and Y directions to obtain a depth image. It should be noted that the matrix obtained after integrating the gradients in the X and Y directions can be represented as an image after being converted into a mat format.

[0072] When the folded phase image is unfolded, the spatial phase unfolding method is used to obtain the unfolded phase, and the gradient map of the unfolded phase in the XY direction is calculated as follows: Figure 6 and Figure 7 shown.

[0073] The gradient data is integrated by the Southwell method using formula (3) to obtain the depth image of the defect location, such as Figure 8 shown.

[0074] (Formula 3)

[0075] in, is the world coordinate, and Respectively and The measured gradient value in the direction, subscript and is the matrix index into the phase unwrapped plot.

[0076] S5. Perform morphological processing on the depth image to obtain the accurate position and size of the defect. It should be noted that since the position of the defect in the depth image changes relative to the surrounding plane, the defect is brighter relative to the surrounding plane in the depth image, and the higher the brightness, the deeper the depth. The area with higher brightness reflects the position and size of the defect. Therefore, the accurate position and size of the defect can be obtained using the depth image at this time.

[0077] Further preferably, in S5, the performing morphological processing on the depth image includes:

[0078] Extracting the edges of defects in the depth image;

[0079] After the detection area where the defect is located is binarized, modulation filtering is performed to filter out the area with the largest area in the binarized area;

[0080] The region with the largest area is first closed and then opened. The noise points in the region with the largest area are eliminated by the closing operation, and the noise points outside the region are eliminated by the opening operation, thereby obtaining a clearer defect image. The edge of the defect image is then extracted to obtain the accurate position and size of the defect. Fig. 9 and Fig.10 shown.

[0081] The present invention also provides a mirror-like defect detection system, comprising a specific light source, an image acquisition module, and an image processing module;

[0082] The specific light source is used to project specific light onto the mirror surface to be detected; further, Figure 2As shown, the specific light source adopts an LED lamp bead screen, and the LED lamp bead screen includes a control circuit, a PCB board, multiple rows of LED lamp beads (the light source row and column distribution row: 136, column: 240, LED lamp bead spacing is 3.5mm), an anti-reflection film, a diffusion film and an acrylic plate. The control circuit controls the on and off of the lamp beads to produce a sinusoidal fringe pattern with a difference of 1 / 4 fringe period. The four monocular cameras in the figure are used to obtain the four horizontal and vertical sinusoidal fringe patterns generated by the lamp bead screen.

[0083] The anti-reflection film, diffusion film and acrylic plate are sequentially arranged to scatter the light emitted by the LED lamp beads and then project it onto the mirror surface to be detected to form a sinusoidal fringe image. It should be noted that the anti-reflection film first reduces reflection to ensure that the transmittance of the LED light reaches the highest, providing a more uniform initial light source for the subsequent diffusion film. The diffusion film scatters the light in the middle layer to make it evenly distributed and eliminate uneven light. The acrylic plate finally further optimizes the light through refraction to enhance the clarity of the sinusoidal fringe pattern. This application is not limited to the above three materials, and other materials that achieve the above functions are within the scope of this application.

[0084] The image acquisition module is used to acquire the fringe image projected onto the mirror surface to be detected;

[0085] The image processing module is used to detect the initial position and size of the defect according to the fringe image;

[0086] intercepting a fringe image of an initial position area where the defect is located, calculating a folded phase, and obtaining a folded phase image;

[0087] Unfolding the folded phase image, calculating the gradients in the X and Y directions after the unfolding, and integrating the gradients in the X and Y directions to obtain a depth image;

[0088] The depth image is morphologically processed to obtain the accurate position of the defect. The specific process of the image processing module performing image processing is referred to the above method embodiment, which will not be repeated here.

[0089] Further preferably, the image processing module further comprises constructing a defect detection model, wherein the defect detection model comprises a defect detection sub-model and an anomaly detection sub-model;

[0090] Among them, the labeled sample images are input into the supervised network model for training to generate a defect detection sub-model;

[0091] Input the unlabeled sample images into the unsupervised network model for training to generate anomaly detection sub-model;

[0092] The trained defect detection sub-model and anomaly detection sub-model are combined to form a complete defect detection model.

[0093] Obviously, the above embodiments are only examples for clear explanation, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A method for detecting mirror-like defects, characterized in that: The following steps are involved: S1. Place the mirror to be inspected in a detection environment, and use a monocular camera to collect the fringe image projected onto the mirror to be inspected; a specific light source is provided in the detection environment, and the specific light source uses an LED lamp bead screen, and the LED lamp bead screen includes a control circuit, multiple rows of LED lamp beads, an anti-reflection film, a diffusion film, and an acrylic plate; the control circuit is used to control the LED lamp beads to light up according to a 1 / 4 fringe cycle difference between the front and back; the anti-reflection film, the diffusion film, and the acrylic plate are arranged in sequence to scatter and refract the light emitted by the LED lamp beads and then project it onto the mirror to be inspected to form a sinusoidal fringe image; The specific light source generates 8 sinusoidal fringe images in a time-sharing manner, and the phase difference between adjacent images is one quarter of a fringe period. When irradiating, the irradiation is switched according to the horizontal and vertical stripes. When the horizontal beads of the LED lamp bead matrix form a stripe pattern according to the sinusoidal ripples, it is projected onto the mirror to be detected, and the camera performs an image acquisition. When the vertical beads of the LED lamp bead matrix form a stripe pattern according to the sinusoidal ripples, it is projected onto the mirror to be detected, and the camera performs another image acquisition. By controlling the LED lamp beads, a row or a column of lamp beads are independently controlled to simulate the situation of sinusoidal wave fluctuations. S2. Detecting the initial position and size of the defect using a defect detection model according to the fringe image; S3, intercepting the fringe image of the initial position area where the defect is located, calculating the folding phase, and obtaining a folding phase image; S4, unfolding the folded phase image, calculating the gradients in the X and Y directions after unfolding, integrating the gradients in the X and Y directions, and obtaining a depth image; in S4, the folded phase image is unfolded using a spatial phase unfolding method, and the gradients in the X and Y directions after unfolding are calculated using the following formula (3) to obtain a depth image; (Formula 3) in, are all world coordinates, express The measured gradient value in the direction, express The measured gradient value in the direction, subscript and are all matrix indices of gradient maps; S5. Perform morphological processing on the depth image to obtain the accurate position and size of the defect.

2. The mirror-like defect detection method according to claim 1, characterized in that: In S2, the step of detecting the initial position and size of the defect using a defect detection model according to the fringe image also includes: Constructing a defect detection model, wherein the defect detection model includes a defect detection sub-model and an anomaly detection sub-model; Input the labeled sample images into the supervised network model for training to generate a defect detection sub-model; Perform image segmentation on the sample image to generate a preset number of unlabeled sample images, input the unlabeled sample images into the unsupervised network model for training, and generate anomaly detection sub-model; The trained defect detection sub-model and anomaly detection sub-model are combined to form a complete defect detection model.

3. The mirror-like defect detection method according to claim 1, characterized in that: In S3, the step of intercepting the fringe image of the initial position region where the defect is located further includes proportionally expanding the initial position region where the defect is located and intercepting the expanded fringe image.

4. The mirror-like defect detection method according to claim 1, characterized in that: In S3, when calculating the folding phase, the folding phase of the intercepted area is calculated according to the following formula (2) to obtain a folding phase image; (Formula 2) in, Represents the pixel point in the nth sinusoidal fringe image The light intensity value at represents the phase shift of the fringe image, and N represents the total number of projected sinusoidal fringe images.

5. The mirror-like defect detection method according to claim 1, characterized in that: In S5, performing morphological processing on the depth image includes: Extracting the edges of defects in the depth image; After the detection area where the defect is located is binarized, modulation filtering is performed to filter out the area with the largest area in the binarized area; The region with the largest area is first closed and then opened to obtain the exact position of the defect.

6. A mirror-like defect detection system, used to implement the mirror-like defect detection method according to any one of claims 1 to 5, characterized in that: Including specific light source, image acquisition module, image processing module; The specific light source is used to project specific light onto the mirror surface to be detected; The image acquisition module is used to acquire the fringe image projected onto the mirror surface to be detected; The image processing module is used to detect the initial position and size of the defect based on the fringe image collected by the image acquisition module; intercept the fringe image of the initial position area of ​​the defect, calculate the folding phase, and obtain the folding phase image; Unfolding the folded phase image, calculating the gradients in the X and Y directions after the unfolding, and integrating the gradients in the X and Y directions to obtain a depth image; The depth image is morphologically processed to obtain the accurate position of the defect.

Images