Defect detection method, device, system, computer equipment and readable storage medium

Abstract

The present application relates to a defect detection method, device, system, computer equipment and readable storage medium. The method includes: obtaining a structured light reflection image corresponding to the object to be detected, and performing target detection on the structured light reflection image to obtain at least one target image area; performing mask image conversion on the target image area to obtain a mask image of the target area; performing three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model; based on the target absolute phase data extracted from the mask image of the target area, correcting the first three-dimensional reconstruction model to obtain a second three-dimensional reconstruction model; and measuring the defect size of the object to be detected according to the second three-dimensional reconstruction model. The present method can accurately measure the defect size.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing technology, and in particular to a defect detection method, apparatus, system, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of intelligent manufacturing technology, defect detection of objects has become a popular technology. By detecting the defects of objects, defect detection results are obtained, and then based on the defect detection results, it is accurately determined whether the objects need maintenance, which effectively improves the service life of the objects.

[0003] In traditional technology, the method of detecting object defects usually adopts algorithms such as real-time target detection to detect defects in real time, which can accurately and effectively detect the number of defective pixels. Specifically, the defect measurement model is trained through the historical defect data of the same type of object, and then the defect measurement model is used to detect the object to be detected to obtain the initial defect measurement result of the object to be detected, wherein the historical defect data includes historical defects and their corresponding number of defective pixels, and the defect measurement result includes the number of defective pixels. The initial defect measurement result is then converted by the resolution of the optical system to obtain the defect size.

[0004] However, the current defect detection method has the problem of being unable to accurately measure the defect size. Summary of the invention

[0005] Based on this, it is necessary to provide a defect detection method, device, system, computer equipment, computer-readable storage medium and computer program product for accurately measuring defect size in response to the above technical problems.

[0006] In a first aspect, the present application provides a defect detection method, comprising:

[0007] Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0008] Performing mask image conversion on the target image area to obtain a mask image of the target area;

[0009] Performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0010] Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model;

[0011] The defect size of the object to be measured is measured according to the second 3D reconstructed model.

[0012] In one embodiment, obtaining a structured light reflection image corresponding to the object to be measured includes:

[0013] Acquire a structured light reflection image obtained by irradiating the structured light image in each direction onto the object to be measured and reflecting it, wherein there is more than one structured light image in each direction;

[0014] Perform mask image conversion on the target image area to obtain the target area mask image, including:

[0015] For any direction, the target image area corresponding to the multiple structured light reflection images is fused into a modulation image of the target area, and the modulation image of the target area is converted into a mask image to generate an initial mask image of the target area;

[0016] The overlapping area images between the target area initial mask images in all directions are determined as the target area mask image.

[0017] In one embodiment, performing mask image conversion on the modulation image of the target area to generate an initial mask image of the target area includes:

[0018] Performing grayscale normalization processing on the modulation image of the target area to obtain a normalized modulation image of the target area;

[0019] Obtaining a grayscale binarization threshold, and based on the grayscale binarization threshold, converting the target area normalized modulation image into a target area binary image;

[0020] The connected domain analysis is performed on the binary image of the target area, and the connected domain with the largest area in the binary image of the target area is determined as the initial mask image of the target area.

[0021] In one embodiment, based on a grayscale binarization threshold, converting the target area normalized modulation image into a target area binary image includes:

[0022] Perform gray value histogram processing on the normalized modulation image of the target area to obtain a gray histogram image of the target area;

[0023] The average value between the grayscale values ​​of the two peak areas in the grayscale histogram of the target area is obtained, and the average value is determined as the grayscale binarization threshold.

[0024] In one embodiment, performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model includes:

[0025] Based on the initial absolute phase data corresponding to the structured light reflection image and the preset phase calibration data, generating first incident light position information and first outgoing light position information for each pixel in the structured light reflection image;

[0026] Generate first three-dimensional position information of each pixel in the structured light reflection image based on the first incident light position information and the first outgoing light position information of each pixel;

[0027] The first three-dimensional position information of each pixel is fitted and reconstructed to obtain a first three-dimensional reconstructed model.

[0028] In one embodiment, based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain the second three-dimensional reconstruction model, including:

[0029] triangulate the first three-dimensional reconstruction model to obtain a triangulated three-dimensional reconstruction model;

[0030] Generate second reflected light position information of each pixel in the first three-dimensional reconstruction model based on the preset phase calibration data, and generate an initial second three-dimensional reconstruction model based on the second reflected light position information of each pixel and the triangulated three-dimensional reconstruction model;

[0031] Based on the target absolute phase data extracted from the target area mask image and the preset phase calibration data, the second incident light position information and the third reflected light position information of each pixel in the target area mask image are generated, and based on the second incident light position information and the third reflected light position information, a gradient normal vector of each pixel is generated;

[0032] A second 3D reconstruction model is generated based on the initial second 3D reconstruction model and the gradient normal vector of each pixel.

[0033] In a second aspect, the present application also provides a defect detection device, comprising:

[0034] A positioning module, used to obtain a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0035] The mask processing module is used to perform mask image conversion on the target image area to obtain the mask image of the target area;

[0036] A reconstruction module, used for performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0037] A correction module, used for correcting the first three-dimensional reconstruction model based on the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstruction model;

[0038] The defect size measurement module is used to measure the defect size of the object to be measured according to the second three-dimensional reconstruction model.

[0039] In a third aspect, the present application further provides a system, including a controller, wherein the controller is configured to implement the following steps:

[0040] Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0041] Performing mask image conversion on the target image area to obtain a mask image of the target area;

[0042] Performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0043] Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model;

[0044] The defect size of the object to be measured is measured according to the second 3D reconstructed model.

[0045] In a fourth aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0046] Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0047] Performing mask image conversion on the target image area to obtain a mask image of the target area;

[0048] Performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0049] Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model;

[0050] The defect size of the object to be measured is measured according to the second 3D reconstructed model.

[0051] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0052] Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0053] Performing mask image conversion on the target image area to obtain a mask image of the target area;

[0054] Performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0055] Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model;

[0056] The defect size of the object to be measured is measured according to the second 3D reconstructed model.

[0057] In a sixth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0058] Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0059] Performing mask image conversion on the target image area to obtain a mask image of the target area;

[0060] Performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0061] Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model;

[0062] The defect size of the object to be measured is measured according to the second 3D reconstructed model.

[0063] The above-mentioned defect detection method, device, system, computer equipment, computer-readable storage medium and computer program product detect at least one target image area of ​​the structured light reflection image corresponding to the object to be measured to accurately obtain a mask image of the target area, and then perform three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model, and then correct the first three-dimensional reconstruction model based on the target absolute phase data extracted from the mask image of the target area to obtain a second three-dimensional reconstruction model. Finally, according to the second three-dimensional reconstruction model, accurate defect size measurement is performed on the object to be measured. In the present application, firstly, by accurately performing mask image conversion on the target image area in the structured light reflection image, a mask image of the target area is obtained, thereby laying a foundation for subsequent three-dimensional reconstruction. Furthermore, combined with topographic optical measurement technology, compared with the traditional optical system resolution conversion method, the reconstructed and corrected second three-dimensional reconstruction model is more accurate for defect size measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 is an application environment diagram of a defect detection method in an embodiment;

[0066] Figure 2 is a schematic flow chart of a defect detection method in one embodiment;

[0067] Figure 3 The figure is a schematic diagram of a process of performing mask image conversion on a target image area in one embodiment, taking the horizontal direction and the vertical direction as examples;

[0068] Figure 4 is a flow chart of a defect detection method in another embodiment;

[0069] Figure 5 is a flow chart of a defect detection method in yet another embodiment;

[0070] Figure 6 is a schematic diagram of triangulation of a point cloud of a first three-dimensional reconstructed model in one embodiment;

[0071] Figure 7 A schematic diagram of image processing during the measurement of the actual size of a defect in one embodiment;

[0072] Figure 8 is a structural block diagram of a defect detection device in one embodiment;

[0073] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present application and are not used to limit the present application.

[0075] Existing automated defect detection equipment is generally based on the principle of machine vision, using cameras to collect image data after enhancing defect features through lighting. Due to the excellent performance of deep learning technology in this field with high detection rates in recent years, deep learning technology has gradually become mainstream in the industrial field. Real-time defect detection is usually performed using algorithms such as real-time target detection, which can accurately and effectively detect the number of defective pixels. However, this method cannot accurately locate the pixel area position of the defect and is heavily dependent on the accuracy of artificial data standards. Usually, this method obtains the pixel size of the defect in the two-dimensional image, and then obtains the size of the defect through theoretical conversion calculation of the optical system. In this calculation process, many deviations are introduced, and the accurate true size of the defect cannot be obtained.

[0076] Therefore, the present application provides a more accurate defect detection method, which combines conventional two-dimensional defect size measurement and three-dimensional morphology optical measurement, that is, the three-dimensional data of the defective pixels is calculated through a series of processes such as decoding and reconstruction, and the background pixels are excluded by combining the defect size mask calculation to accurately calculate the true size of the defect and obtain an accurate defect size measurement result. Compared with the traditional method, the accuracy is higher and closer to the true size of the defect, which is of great significance for the subsequent accurate classification according to the defect size, and is conducive to the subsequent timely repair to reduce the outflow of defects.

[0077] The defect detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the controller 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the controller 104, or placed on the cloud or other network servers.

[0078] First, the controller 104 obtains the structured light reflection image corresponding to the object to be measured. Specifically, the staff triggers a defect size measurement request at the terminal 102, and the terminal 102 sends the defect size measurement request to the controller 104. The controller 104 responds to the defect size measurement request and obtains the structured light reflection image corresponding to the object to be measured that is pre-collected by the acquisition device 106 from the database. Alternatively, the staff triggers a defect size measurement request at the terminal 102, and the terminal 102 sends the defect size measurement request to the controller 104. The controller 104 responds to the defect size measurement request and controls the acquisition device 106 to collect the structured light reflection image corresponding to the object to be measured in real time.

[0079] Further, the controller 104 performs target detection on the structured light reflection image to obtain at least one target image area, performs mask image conversion on the target image area to obtain a mask image of the target area, performs three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model, corrects the first three-dimensional reconstruction model based on the target absolute phase data extracted from the mask image of the target area to obtain a second three-dimensional reconstruction model, and measures the defect size of the object to be measured based on the second three-dimensional reconstruction model. Further, the controller 104 can feed back the defect size measurement result to the terminal 102, and the terminal 102 displays the defect size measurement result of the object to be measured to the user.

[0080] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0081] In an exemplary embodiment, Figure 2 As shown, a defect detection method is provided, which is applied to Figure 1 The controller 104 in FIG. 1 is used as an example for explanation.

[0082] S100, obtaining a structured light reflection image corresponding to the object to be detected, and performing target detection on the structured light reflection image to obtain at least one target image area.

[0083] Among them, structured light technology uses a pre-designed pattern with a special structure (such as discrete light spots, stripe light, coded structured light, etc.), and then projects the pattern onto the surface of a three-dimensional object, and uses another camera to observe the distortion of the image on the three-dimensional physical surface. The structured light reflection image is the structured light reflection image obtained after the structured light image is irradiated onto the object to be tested and reflected. The target image area refers to the area with defects in the structured light reflection image.

[0084] Specifically, the projector or display generates a series of structured light images with phase differences according to their respective resolutions, and then projects the structured light images onto the surface of the object to be measured to generate a structured light reflection image after reflection. Generally, the structured light image can be a sine fringe image or a cosine fringe image.

[0085] The controller controls the acquisition device to acquire the generated structured light reflection image, performs target detection on the structured light reflection image, and obtains the area with defects in the structured light reflection image.

[0086] In one of the embodiments, target detection is performed on the structured light reflection image to obtain areas in the structured light reflection image that may contain defects, including: locating areas in the structured light reflection image that contain defects through a deep learning algorithm, such as a pre-trained YOLO (You only lookonce, a target detection algorithm) model.

[0087] In one of the embodiments, before performing target detection on the structured light reflection image to obtain at least one target image area, it also includes: preprocessing the structured light reflection image and updating the structured light reflection image; preprocessing the structured light reflection image and updating the structured light reflection image, including: preprocessing the structured light reflection image and updating the structured light reflection image through image processing algorithms such as image enhancement and filtering to enhance defect features that may be contained in the structured light reflection image and reduce noise in the structured light reflection image.

[0088] S200, performing mask image conversion on the target image area to obtain a mask image of the target area.

[0089] Among them, in image processing, a mask is a special image used to control the image processing area or processing process. It is usually a two-dimensional matrix of the same size as the original image, which is used to selectively cover or display specific areas of the image. Therefore, the target area mask image is an image used to characterize the actual defect area size of the target image area.

[0090] Specifically, the deep learning model is highly dependent on pre-annotated manual samples. Due to annotation errors and algorithm characteristics, the target image area obtained by detection and positioning is usually larger than the actual defect area. Therefore, it is necessary to perform mask image conversion on the target image area to obtain the actual size mask of the target image area, that is, the mask image of the target area.

[0091] S300, performing three-dimensional reconstruction based on initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model.

[0092] Among them, the absolute phase is the specifically measured phase, usually referred to as the phase. To obtain the absolute phase is to determine the position in the entire image.

[0093] Specifically, the absolute phase of the structured light reflection image is solved to obtain the initial absolute phase data corresponding to the structured light reflection image, so as to obtain a unique coding value for each pixel in the image, and the unique coding value is a unique coding value composed of the absolute phases in the horizontal and vertical directions. Among them, the method of solving the absolute phase of the structured light reflection image includes but is not limited to: solving the absolute phase based on the spatial domain phase solution method, such as the least squares unwrapping method, or solving the absolute phase based on the time domain phase solution method, such as the multi-frequency heterodyne method.

[0094] Further, based on the initial absolute phase data corresponding to the structured light reflection image, the object to be measured is reconstructed in three dimensions to obtain a first three-dimensional reconstruction model, that is, a theoretical surface shape of the object to be measured. The three-dimensional reconstruction of the object to be measured includes but is not limited to Zernike polynomial fitting.

[0095] In one of the embodiments, three-dimensional reconstruction of the object to be measured is performed based on the initial absolute phase data corresponding to the structured light reflection image, which also includes: down-sampling the initial absolute phase data to a certain extent, and then reconstructing the object to be measured in three dimensions based on the down-sampled initial absolute phase data to improve the reconstruction processing speed.

[0096] S400, based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstructed model is corrected to obtain a second three-dimensional reconstructed model.

[0097] Specifically, when three-dimensional reconstruction is performed based on the initial absolute phase data corresponding to the structured light reflection image, the first three-dimensional reconstruction model obtained, that is, the theoretical surface shape of the object to be measured is a filtered and smoothed surface shape. Therefore, it is necessary to correct the first three-dimensional reconstruction model in combination with the target absolute phase data extracted from the target area mask image so that the first three-dimensional reconstruction model contains accurate phase data to obtain a corrected second three-dimensional reconstruction model.

[0098] In one of the embodiments, obtaining target absolute phase data extracted from a target area mask image includes: filtering out the initial absolute phase data corresponding to the target area mask image from the initial absolute phase data corresponding to the structured light reflection image, and determining the filtered initial absolute phase data corresponding to the target area mask image as the target absolute phase data.

[0099] S500: measuring defect size of the object to be measured according to the second three-dimensional reconstructed model.

[0100] Specifically, the corrected second three-dimensional reconstruction model contains accurate three-dimensional phase data. According to the corrected second three-dimensional reconstruction model, the defect size of the object to be measured is measured to obtain a defect size measurement result.

[0101] In the above-mentioned defect detection method, at least one target image area of ​​the structured light reflection image corresponding to the object to be measured is detected to accurately obtain a mask image of the target area, and then three-dimensional reconstruction is performed based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first 3D reconstruction model, and then the first 3D reconstruction model is corrected based on the target absolute phase data extracted from the mask image of the target area to obtain a second 3D reconstruction model. Finally, according to the second 3D reconstruction model, the defect size of the object to be measured is accurately measured. In the present application, firstly, by accurately performing mask image conversion on the target image area in the structured light reflection image, a mask image of the target area is obtained, thereby laying a foundation for subsequent 3D reconstruction. Furthermore, combined with topographic optical measurement technology, compared with the traditional optical system resolution conversion method, the reconstructed and corrected second 3D reconstruction model is more accurate for defect size measurement.

[0102] In an exemplary embodiment, a structured light reflection image corresponding to the object to be measured is obtained, including: obtaining a structured light reflection image obtained after irradiating the structured light image in each direction onto the object to be measured and reflecting it; performing a mask image conversion on the target image area to obtain a target area mask image, including: for any direction, merging the target image areas corresponding to multiple structured light reflection images into a target area modulation image, and performing a mask image conversion on the target area modulation image to generate an initial mask image of the target area; and determining the overlapping area images between the initial mask images of the target area in all directions as the mask image of the target area.

[0103] There is more than one structured light image in each direction.

[0104] Specifically, obtaining the structured light reflection image corresponding to the object to be measured is essentially to project multiple structured light images in each direction onto the object to be measured and then obtain multiple structured light reflection images by reflection. Taking the sinusoidal fringe image as the structured light image as an example, the generation process of the sinusoidal fringe image includes: the projector or display generates a series of sinusoidal fringe images with phase difference in each direction according to their respective resolutions. Generally, the series of sinusoidal fringe images with phase difference in each direction include but are not limited to N sinusoidal fringe images with phase difference in the horizontal direction and N sinusoidal fringe images with phase difference in the vertical direction. Generally, N≥3.

[0105] Furthermore, the light intensity distribution function of the nth sinusoidal fringe image can be expressed as:

[0106]

[0107] Where (x, y) represents the pixel coordinates in the image, I n(x, y) represents the grayscale value of the pixel coordinate (x, y) in the nth image. A(x, y) represents the average light intensity, which is related to the pattern brightness and background illumination. B(x, y) represents the intensity modulation, which is related to the pattern contrast and the reflectivity of the object surface. Indicates the phase value of the current point, Represents the phase difference or phase shift value of the nth image. .

[0108] It should be explained that when generating sinusoidal fringe images based on resolution, the resolution mainly affects the intensity distribution function. , for example, to generate stripes with a frequency of n, the period of the stripes is the resolution divided by n, It is equal to the pixel coordinate divided by the period and then multiplied by 2π, that is, the phase value of one period of the stripe is 0-2π, and then substituted into the formula of the light intensity distribution function to calculate the gray value of the corresponding pixel position, thereby generating a sinusoidal stripe image. The average light intensity A is independent for each pixel. For example, in the four-step phase shift method, the average light intensity is obtained by averaging the corresponding pixels of the four images collected.

[0109] Generally, A = B = maximum grayscale value / 2 is taken during generation. The specific grayscale value depends on the maximum grayscale value of the required image. For example, if the maximum grayscale value is 255, A and B are 122.5.

[0110] Furthermore, the projector generates a series of sinusoidal fringe images with phase differences in each direction according to the resolution, which is achieved by phase measurement profilometry based on the diffuse reflection properties of the optical surface. The display screen generates a series of sinusoidal fringe images with phase differences in each direction according to the resolution, which is achieved by phase deflectometry based on the specular reflection properties of the optical surface.

[0111] After acquiring the structured light reflection images in each direction, for any direction, the target image areas corresponding to the multiple structured light reflection images are fused into a target area modulation image. The fusion expression is:

[0112]

[0113] Where N is the number of structured light reflection images in each direction.

[0114] Then, the target area modulation image is converted into a mask image to generate an initial mask image of the target area in each direction, and the overlapping area images between the initial mask images of the target area in all directions are obtained, and the overlapping area images between the initial mask images of the target area in all directions are determined as the target area mask image.

[0115] Obtaining overlapping area images between initial mask images of target areas in all directions includes: performing an AND operation on the initial mask images of target areas in all directions to obtain a final defect actual size mask, that is, a target area mask image.

[0116] Figure 3 The following is a flow chart of mask image conversion for a target image area, taking the horizontal and vertical directions as examples. First, for the horizontal and vertical directions, the target image areas corresponding to the multiple structured light reflection images in the horizontal direction are fused into a target area modulation image, and the target area modulation image is converted into a mask image to generate an initial mask image of the target area in the horizontal direction. The target image areas corresponding to the multiple structured light reflection images in the vertical direction are fused into a target area modulation image, and the target area modulation image is converted into a mask image to generate an initial mask image of the target area in the vertical direction. The overlapping area image between the initial mask images of the target area corresponding to the horizontal and vertical directions is determined as the mask image of the target area.

[0117] In an exemplary embodiment, the target image area corresponding to the multiple structured light reflection images is fused into a target area modulation image, which also includes: fusing the multiple structured light reflection images into an initial area modulation image, and then screening the target area modulation image corresponding to the target image area from the initial area modulation image.

[0118] In this embodiment, the target image areas corresponding to the multiple structured light reflection images are fused into a target area modulation image, and the target area modulation image is converted into a mask image to generate an initial mask image of the target area. Then, the overlapping area images between the initial mask images of the target area in all directions are used to remove the shadow influence caused by stripe lighting in different directions, so as to improve the accuracy of determining the mask image of the target area.

[0119] In an exemplary embodiment, performing mask image conversion on the target region modulation image to generate an initial mask image of the target region includes:

[0120] The target area modulation image is grayscale normalized to obtain a normalized modulation image of the target area, a grayscale binarization threshold is obtained, and based on the grayscale binarization threshold, the normalized modulation image of the target area is converted into a binary image of the target area, a connected domain analysis is performed on the binary image of the target area, and the connected domain with the largest area in the binary image of the target area is determined as the initial mask image of the target area.

[0121] The grayscale binarization threshold refers to the threshold for segmenting the foreground and background in the normalized modulation image of the target area.

[0122] Specifically, the target area modulation image is subjected to grayscale normalization, that is, the target area modulation image is subjected to grayscale mapping and normalized to a grayscale value range of 0-255 to obtain a target area normalized modulation image. The mapping expression is:

[0123]

[0124] Where (x, y) represents the pixel coordinates in the image, Represents the grayscale value of the (x, y) pixel coordinates in the normalized modulation image of the target area. It represents the modulation value corresponding to the pixel coordinate (x, y) in the initial structured light reflection image. min(g) represents the minimum value in the modulation image of the target area, and max(g) represents the maximum value in the modulation image of the target area.

[0125] In order to remove some noise in the background of the image, the present application also separates the background and foreground of the normalized modulation image of the target area. Specifically, a grayscale binarization threshold is obtained, and based on the grayscale binarization threshold, the background and foreground of the normalized modulation image of the target area are separated to obtain a binary image of the target area, wherein the foreground of the normalized modulation image of the target area is often a defect. Further, based on the grayscale binarization threshold, the background and foreground of the normalized modulation image of the target area are separated, including: obtaining the grayscale value of each pixel in the normalized modulation image of the target area, determining the pixel points with grayscale values ​​greater than the grayscale binarization threshold as the background, and determining the pixel points with grayscale values ​​less than or equal to the grayscale binarization threshold as the foreground (defect).

[0126] The connected domain analysis is performed on the binary image of the target area, and the connected domain with the largest area in the binary image of the target area is determined as the initial mask image of the target area.

[0127] In this embodiment, by normalizing and binarizing the modulation image of the target area, the noise of the background area in the modulation image of the target area is significantly reduced, the accuracy of determining the initial mask image of the target area is improved, and an accurate defect actual size mask is obtained.

[0128] In an exemplary embodiment, based on a grayscale binarization threshold, converting a target region normalized modulation image into a target region binary image includes:

[0129] The grayscale value histogram processing is performed on the normalized modulation image of the target area to obtain the grayscale histogram image of the target area, the average value between the grayscale values ​​of two peak areas in the grayscale histogram image of the target area is obtained, and the average value is determined as the grayscale binarization threshold.

[0130] Among them, a histogram is a statistical chart in which a set of statistical data is represented by rectangular bars, such that each bar in the histogram is proportional to its corresponding frequency.

[0131] Specifically, grayscale value histogram processing is performed on the normalized modulation image of the target area, and the grayscale value of the normalized modulation image of the target area is represented by a histogram to obtain a grayscale histogram image of the target area.

[0132] Traverse each area in the grayscale histogram of the target area, determine at least one peak area in the grayscale histogram of the target area, and calculate the peak average value of multiple peak areas as the grayscale binarization threshold, so as to distinguish the foreground (mainly defects) and background of the normalized modulation image of the defect area through the grayscale binarization threshold.

[0133] In this embodiment, the peak value average of multiple peak areas in the grayscale histogram of the target area is used as the grayscale binarization threshold instead of presetting a grayscale binarization threshold, so that the foreground and background are segmented more accurately.

[0134] In an exemplary embodiment, Figure 4 As shown, S300 includes:

[0135] S320 , generating first incident light position information and first outgoing light position information of each pixel in the structured light reflection image based on the initial absolute phase data corresponding to the structured light reflection image and the preset phase calibration data.

[0136] S340 , generating first three-dimensional position information of each pixel in the structured light reflection image based on the first incident light position information and the first outgoing light position information of each pixel.

[0137] S360: Perform fitting reconstruction on the first three-dimensional position information of each pixel to obtain a first three-dimensional reconstruction model.

[0138] Specifically, through the imaging process of phase profilometry or phase deflectometry, based on the principle of triangulated imaging, using the pre-performed system calibration data and the initial absolute phase data corresponding to the structured light reflection image, the first incident light position information and the first outgoing light position information of each pixel in the structured light reflection image can be restored, and then based on the first incident light position information and the first outgoing light position information of each pixel, the first three-dimensional position information of each pixel in the structured light reflection image is generated, wherein (the system calibration data mainly refers to the intrinsic parameter matrix of the camera and the light source (projector or LCD (Liquid Crystal Display) display) and the relative posture relationship between them, and the relative posture relationship is the relative rotation and translation matrix. This process can be called a sparse reconstruction process, and the first three-dimensional position information of each pixel is the sparse reconstruction result.

[0139] Furthermore, based on the sparse reconstruction result of the previous step, the surface fitting and reconstruction of the object to be detected can be performed, such as Zernike polynomial fitting, to obtain the theoretical surface shape of the detection area, that is, the first three-dimensional reconstruction model.

[0140] In one of the embodiments, based on the first incident light position information and the first outgoing light position information of each pixel, the first three-dimensional position information of each pixel in the structured light reflection image is generated, including: determining the intersection of the first incident light position information and the first outgoing light position information of each pixel as the first three-dimensional position information of each pixel in the structured light reflection image.

[0141] In this embodiment, compared with directly densely reconstructing the absolute phase data of the target area mask image, a three-dimensional fitting reconstruction is performed based on the initial absolute phase data corresponding to the structured light reflection image to obtain a theoretical surface shape, and then a three-dimensional reconstruction is performed based on the theoretical surface shape and the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstruction model, which is simpler and more efficient.

[0142] In an exemplary embodiment, Figure 5 As shown, S400 includes:

[0143] S420: triangulate the first three-dimensional reconstructed model to obtain a triangulated three-dimensional reconstructed model.

[0144] S440, generating second reflected light position information of each pixel in the first 3D reconstruction model based on the preset phase calibration data, and generating an initial second 3D reconstruction model based on the second reflected light position information of each pixel and the triangulated 3D reconstruction model.

[0145] S460, based on the target absolute phase data extracted from the target area mask image and the preset phase calibration data, generate the second incident light position information and the third reflected light position information of each pixel in the target area mask image, and based on the second incident light position information and the third reflected light position information, generate the gradient normal vector of each pixel.

[0146] S480: Generate a second 3D reconstructed model based on the initial second 3D reconstructed model and the gradient normal vector of each pixel.

[0147] Specifically, since the first three-dimensional reconstruction model obtained by sparse reconstruction is a theoretical surface shape, the internal data will be filtered and smoothed by the fitting process. Therefore, it is necessary to correct the first three-dimensional reconstruction model based on the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstruction model.

[0148] The specific correction process includes:

[0149] 1. If Figure 6 As shown, the point cloud of the first 3D reconstruction model is triangulated to obtain a triangulated 3D reconstruction model, and the second reflected light position information corresponding to each pixel is restored based on the preset phase calibration data, and the initial second 3D reconstruction model is generated based on the second reflected light position information of each pixel and the triangulated 3D reconstruction model. The triangulation means include but are not limited to triangulating the theoretical surface point cloud using Delaunay; generating the initial second 3D reconstruction model based on the second reflected light position information of each pixel and the triangulated 3D reconstruction model, including: finding the intersection of the second reflected light position information and the triangulated 3D reconstruction model, obtaining the original dense point cloud data of the defect area point cloud, and determining the original dense point cloud data of the defect area point cloud as the initial second 3D reconstruction model. This step is based on the dense points obtained by sampling the nominal surface (sparse points), and there is a certain error.

[0150] 2. Based on the target absolute phase data extracted from the target area mask image and the preset phase calibration data, the second incident light position information and the third reflected light position information corresponding to each pixel are restored, and then the corresponding gradient normal vector (bisector) is obtained according to the law of optical reflection. This step obtains the gradient result of the actual defect data, which is relatively accurate.

[0151] 3. Generate a second 3D reconstruction model based on the initial second 3D reconstruction model and the gradient normal vector of each pixel. Specifically, based on the dense points obtained by sampling in the first step and the gradient obtained in the second step, path integration is performed based on the local gradient increment to obtain an accurate dense point cloud result of the local defect area, i.e., the second 3D reconstruction model.

[0152] In this embodiment, by correcting the first three-dimensional reconstructed model based on the target absolute phase data extracted from the target area mask image, more accurate defect location information can be obtained, so as to accurately detect the actual size of the defect based on the defect location information.

[0153] The defect detection method will be described in detail below with a specific application example:

[0154] 1. Generate projection images: According to the resolution of the projector or display screen, a series of sinusoidal fringe projection images with phase difference are generated in the horizontal and vertical directions. Usually, the number of sinusoidal fringe projection images in each direction is greater than or equal to 3. Let the number of sinusoidal fringe projection images in each direction be N. The light intensity distribution function of the nth image can be expressed as:

[0155]

[0156] Where (x, y) represents the pixel coordinates in the image, A(x, y) represents the average light intensity, which is related to the pattern brightness and background illumination, and B(x, y) represents the intensity modulation, which is related to the pattern contrast and the reflectivity of the object surface. Indicates the phase value of the current point, Represents the phase difference or phase shift value of the nth image. , I n (x, y) represents the grayscale value of the pixel coordinate (x, y) in the nth image.

[0157] 2. Collect the original phase-shift fringe image: The projector projects the sinusoidal fringe projection image to the detection area of ​​the object to be tested, and it is synchronously collected by the camera. The reflected image data collected by the camera is determined as the original phase-shift fringe image, or the display screen displays the phase-encoded image data reflected by the mirror surface of the object to be tested, and the phase-encoded image data is determined as the original phase-shift fringe data.

[0158] 3. Positioning defect regions of interest: The original phase-shift fringe image is pre-processed through image enhancement, filtering and other image processing algorithms to enhance defect features and reduce image noise. Then, deep learning algorithms such as the YOLO model can be used to detect and position the defect regions of interest.

[0159] 4. Calculation of the actual size mask image of the defect: The deep learning model relies on pre-annotated manual samples. Due to annotation errors and algorithm characteristics, the defect area of ​​interest obtained by detection and positioning is usually larger than the actual defect position. Therefore, further processing is required to obtain the actual size mask image of the defect in this area.

[0160] (1) First, for each direction, N original phase-shifted fringe images are synthesized into an initial modulation image. The synthesis formula is:

[0161]

[0162] Among them, B (x, y) is the actual intensity modulation value of the original phase-shifted fringe image captured by the camera after the sinusoidal fringe image is attenuated during the entire imaging process, which can be calculated by synthesizing multiple original fringe images.

[0163] Furthermore, a modulation image of the target region corresponding to the defective region of interest is extracted from the initial modulation image, and the modulation of the pixels in the defective region of interest can also be calculated separately.

[0164] (2) The target area modulation image is normalized to a grayscale value range of 0-255 by grayscale mapping to generate a normalized modulation image of the defect area. The mapping formula is:

[0165]

[0166] Where (x, y) represents the pixel coordinates in the image, Represents the grayscale value of the (x, y) pixel coordinates in the normalized modulation image of the target area. It represents the modulation value corresponding to the pixel coordinate (x, y) in the initial structured light reflection image. min(g) represents the minimum value in the modulation image of the target area, and max(g) represents the maximum value in the modulation image of the target area.

[0167] (3) Global binarization based on histogram statistics: First, obtain the grayscale binarization threshold. Specifically, perform grayscale value histogram processing on the normalized modulation image of the target area to obtain the grayscale histogram image of the target area. Determine the average value between the grayscale values ​​of the two peak areas in the grayscale histogram image of the target area, and determine the average value as the grayscale binarization threshold. Based on the grayscale binarization threshold, perform foreground and background segmentation processing on the normalized modulation image of the target area, that is, determine the pixel points with grayscale values ​​less than or equal to the grayscale binarization threshold in the normalized modulation image of the target area as defect areas, and determine the pixel points with grayscale values ​​greater than the grayscale binarization threshold in the normalized modulation image of the target area as background. Therefore, retain the pixel points with grayscale values ​​less than or equal to the grayscale binarization threshold in the normalized modulation image of the target area to generate a binary image of the target area.

[0168] Furthermore, a connected domain analysis is performed on the binary image of the target area, and the connected domain with the largest area is determined as the initial mask image of the target area of ​​the defect area to remove noise in the background.

[0169] (4) Performing an AND operation on the initial mask images of the target area obtained in each direction, that is, determining the overlapping image area between the initial mask images of the target area in all directions as the mask image of the target area, that is, the mask image of the actual size of the defect, so as to remove the shadow effect caused by stripe lighting in different directions.

[0170] 5. Phase-shift fringe decoding in the detection area: The initial absolute phase data of the original phase-shift fringe image is solved to obtain a unique coding value for each pixel in the image (the absolute phases in the horizontal and vertical directions together constitute a unique coding value). Specifically, the implementation methods for solving the initial absolute phase data include but are not limited to spatial domain dephasing methods, such as the least squares unwrapping method, or time domain dephasing methods, such as the multi-frequency heterodyne method.

[0171] 6. Sparse reconstruction of the detection area: Through the imaging process of phase profilometry or phase deflectometry, based on the principle of triangular imaging, the preset system phase calibration data and initial absolute phase data are used to restore the position information of the incident light and the position information of the outgoing light, and then the three-dimensional coordinates of each pixel in the detection area of ​​the object to be tested are obtained, and the three-dimensional coordinates of each pixel in the detection area are used as the sparse reconstruction result. In addition, the absolute phase data is usually downsampled to a certain extent to increase the processing speed. Among them, the system phase calibration data mainly refers to the intrinsic parameter matrix of the camera and light source (projector or LCD display) and the relative posture relationship between them (that is, the relative rotation and translation matrix).

[0172] 7. Dense reconstruction of defect area: Based on the sparse reconstruction results, the surface fitting and reconstruction of the detection area is performed, such as Zernike polynomial fitting, to obtain the theoretical surface shape of the detection area. Then, based on the theoretical surface shape of the detection area and the target absolute phase data of the mask image of the actual size of the defect, the defect area is densely reconstructed to accurately obtain the actual three-dimensional data of the defect, and then the accurate real size result of the defect is calculated.

[0173] The present application solves the problem of over-calculation of defect size caused by the large defect area in the deep learning detection algorithm by calculating the actual defect size mask based on the modulation degree, and achieves pixel-level accuracy to obtain accurate actual defect size mask, that is, the mask image of the target area; in addition, by combining the three-dimensional morphology optical measurement technology, the three-dimensional data of the pixels in the defect area is obtained, which can more accurately calculate the actual size of the defect compared to the traditional optical system resolution conversion method.

[0174] In an exemplary embodiment, the image processing of the measurement process of the defect real size result is as follows: Figure 7 As shown, first, the structured light reflection images in the horizontal and vertical directions are obtained, and the target detection is performed on the structured light reflection images to obtain at least one target image area; the target image area is converted into a mask image to obtain a mask image of the target area. Secondly, the absolute phase data corresponding to the structured light reflection images in different directions are obtained, and two point cloud reconstructions are performed based on the absolute phase data corresponding to the structured light reflection images in different directions, that is, sparse point cloud reconstruction and surface fitting are first performed based on the initial absolute phase data corresponding to the structured light reflection images to obtain a first three-dimensional reconstruction model, and then the first three-dimensional reconstruction model is corrected based on the target absolute phase data extracted from the mask image of the target area to obtain a second three-dimensional reconstruction model.

[0175] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0176] Based on the same inventive concept, the embodiment of the present application also provides a defect detection device for implementing the defect detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more defect detection device embodiments provided below can refer to the limitations on the defect detection method above, and will not be repeated here.

[0177] In an exemplary embodiment, Figure 8 As shown, a defect detection device is provided, comprising: a positioning module 100, a mask processing module 200, a reconstruction module 300, a correction module 400 and a defect size measurement module 500, wherein:

[0178] The positioning module 100 is used to obtain a structured light reflection image corresponding to the object to be detected, and perform target detection on the structured light reflection image to obtain at least one target image area;

[0179] The mask processing module 200 is used to perform mask image conversion on the target image area to obtain a mask image of the target area;

[0180] A reconstruction module 300 is used to perform three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model;

[0181] A correction module 400, configured to correct the first three-dimensional reconstructed model based on the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstructed model;

[0182] The defect size measurement module 500 is used to measure the defect size of the object to be measured according to the second three-dimensional reconstructed model.

[0183] In one of the embodiments, the positioning module 100 is also used to obtain a structured light reflection image obtained after irradiating the structured light images in each direction onto the object to be measured and reflecting them, wherein there is more than one structured light image in each direction; the mask processing module 200 is also used to fuse the target image areas corresponding to the multiple structured light reflection images into a target area modulation image for any direction, and perform mask image conversion on the target area modulation image to generate an initial mask image of the target area; and determine the overlapping area images between the initial mask images of the target area in all directions as the mask image of the target area.

[0184] In one of the embodiments, the mask processing module 200 is also used to perform grayscale normalization processing on the target area modulation image to obtain a target area normalized modulation image; obtain a grayscale binarization threshold, and based on the grayscale binarization threshold, convert the target area normalized modulation image into a target area binary image; perform connected domain analysis on the target area binary image, and determine the connected domain with the largest area in the target area binary image as the target area initial mask image.

[0185] In one of the embodiments, the mask processing module 200 is also used to perform grayscale value histogram processing on the normalized modulation image of the target area to obtain a grayscale histogram image of the target area; obtain the average value between the grayscale values ​​of two peak areas in the grayscale histogram image of the target area, and determine the average value as the grayscale binarization threshold.

[0186] In one embodiment, the reconstruction module 300 is also used to generate first incident light position information and first outgoing light position information of each pixel in the structured light reflection image based on the initial absolute phase data and preset phase calibration data corresponding to the structured light reflection image; generate first three-dimensional position information of each pixel in the structured light reflection image based on the first incident light position information and the first outgoing light position information of each pixel; and perform fitting and reconstruction on the first three-dimensional position information of each pixel to obtain a first three-dimensional reconstruction model.

[0187] In one of the embodiments, the correction module 400 is also used to triangulate the first 3D reconstructed model to obtain a triangulated 3D reconstructed model; based on preset phase calibration data, generate second reflected light position information of each pixel in the first 3D reconstructed model, and based on the second reflected light position information of each pixel and the triangulated 3D reconstructed model, generate an initial second 3D reconstructed model; based on the target absolute phase data extracted from the target area mask image and the preset phase calibration data, generate second incident light position information and third reflected light position information of each pixel in the target area mask image, and based on the second incident light position information and the third reflected light position information, generate a gradient normal vector for each pixel; based on the initial second 3D reconstructed model and the gradient normal vector of each pixel, generate a second 3D reconstructed model.

[0188] Each module in the above defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0189] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as structured light reflection images corresponding to the object to be tested. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a defect detection method is implemented.

[0190] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0191] In one embodiment, a defect detection system is further provided, including a controller, wherein the controller is used to execute the steps in the above method embodiments.

[0192] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0193] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0194] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0195] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0196] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0197] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Acquire a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area; Performing mask image conversion on the target image area to obtain a target area mask image; Performing three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model, wherein the performing three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain the first three-dimensional reconstruction model includes: generating first incident light position information and first outgoing light position information of each pixel in the structured light reflection image based on the initial absolute phase data corresponding to the structured light reflection image and preset phase calibration data; generating first three-dimensional position information of each pixel in the structured light reflection image based on the first incident light position information and the first outgoing light position information of each pixel; performing fitting reconstruction on the first three-dimensional position information of each pixel to obtain the first three-dimensional reconstruction model; Based on the target absolute phase data extracted from the target area mask image, the first three-dimensional reconstruction model is corrected to obtain a second three-dimensional reconstruction model; The defect size of the object to be measured is measured according to the second three-dimensional reconstructed model.

2. The method according to claim 1, characterized in that The step of obtaining a structured light reflection image corresponding to the object to be measured includes: Acquire a structured light reflection image obtained by irradiating the structured light image in each direction onto the object to be measured and reflecting it, wherein there is more than one structured light image in each direction; The performing mask image conversion on the target image area to obtain the target area mask image includes: For any of the directions, the target image regions corresponding to the plurality of structured light reflection images are merged into a target region modulation image, and the target region modulation image is converted into a mask image to generate an initial mask image of the target region; The overlapping area images between the target area initial mask images in all directions are determined as the target area mask image.

3. The method according to claim 2, characterized in that The step of converting the modulation degree image of the target area into a mask image to generate an initial mask image of the target area includes: Performing grayscale normalization processing on the target area modulation image to obtain a target area normalized modulation image; Obtaining a grayscale binarization threshold, and based on the grayscale binarization threshold, converting the target area normalized modulation image into a target area binary image; A connected domain analysis is performed on the target region binary image, and a connected domain with the largest area in the target region binary image is determined as an initial mask image of the target region.

4. The method according to claim 3, characterized in that The step of converting the target area normalized modulation image into a target area binary image based on the grayscale binarization threshold comprises: Performing grayscale value histogram processing on the normalized modulation image of the target area to obtain a grayscale histogram image of the target area; The average value between the grayscale values ​​of two peak areas in the grayscale histogram of the target area is obtained, and the average value is determined as the grayscale binarization threshold.

5. The method according to claim 1, characterized in that The first three-dimensional reconstruction model is corrected based on the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstruction model, including: triangulate the first three-dimensional reconstructed model to obtain a triangulated three-dimensional reconstructed model; Generate second reflected light position information of each pixel in the first three-dimensional reconstruction model based on the preset phase calibration data, and generate an initial second three-dimensional reconstruction model based on the second reflected light position information of each pixel and the triangulated three-dimensional reconstruction model; Based on the target absolute phase data extracted from the target area mask image and the preset phase calibration data, second incident light position information and third reflected light position information of each pixel in the target area mask image are generated, and based on the second incident light position information and the third reflected light position information, a gradient normal vector of each pixel is generated; A second three-dimensional reconstruction model is generated based on the initial second three-dimensional reconstruction model and the gradient normal vector of each pixel.

6. A defect detection device, characterized in that: The device comprises: A positioning module, used to obtain a structured light reflection image corresponding to the object to be measured, and perform target detection on the structured light reflection image to obtain at least one target image area; A mask processing module, used for performing mask image conversion on the target image area to obtain a target area mask image; A reconstruction module, configured to perform three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain a first three-dimensional reconstruction model, wherein the three-dimensional reconstruction based on the initial absolute phase data corresponding to the structured light reflection image to obtain the first three-dimensional reconstruction model includes: generating first incident light position information and first outgoing light position information of each pixel in the structured light reflection image based on the initial absolute phase data corresponding to the structured light reflection image and preset phase calibration data; generating first three-dimensional position information of each pixel in the structured light reflection image based on the first incident light position information and the first outgoing light position information of each pixel; and performing fitting reconstruction on the first three-dimensional position information of each pixel to obtain the first three-dimensional reconstruction model; A correction module, configured to correct the first three-dimensional reconstruction model based on the target absolute phase data extracted from the target area mask image to obtain a second three-dimensional reconstruction model; The defect size measurement module is used to measure the defect size of the object to be measured according to the second three-dimensional reconstruction model.

7. The device according to claim 6, characterized in that The positioning module is also used to obtain a structured light reflection image obtained after the structured light image in each direction is irradiated onto the object to be measured and reflected, wherein there is more than one structured light image in each direction; the mask processing module is also used to fuse the target image area corresponding to multiple structured light reflection images into a target area modulation image in any direction, and perform mask image conversion on the target area modulation image to generate an initial mask image of the target area; and determine the overlapping area image between the initial mask images of the target area in all directions as the mask image of the target area.

8. A defect size measurement system, characterized in that: The method comprises a controller for executing the steps of the method according to any one of claims 1 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images