A method and apparatus for detecting surface defects of an object, three-dimensional scanner

Abstract

The application discloses an object surface defect detection scheme and belongs to the technical field of three-dimensional scanners. The method comprises the following steps: scanning the surface of a target object by using a three-dimensional scanner; during the scanning process, light sources on the three-dimensional scanner are exposed in turn at different angles according to a calibrated light source direction calibration sequence, and meanwhile, the three-dimensional scanner collects images of the surface of the target object to obtain a first image; a normal vector matrix and a reflectivity distribution matrix of the surface of the target object are acquired based on a light source direction matrix obtained when the light source direction is calibrated by using a photometric stereo method three-dimensional reconstruction technology; a gradient distribution matrix of the surface of the target object is generated based on the normal vector matrix; and concave-convex flaws and scratches existing on the surface of the target object are detected based on the gradient distribution matrix and the reflectivity distribution matrix. The object surface defect detection scheme provided by the application can comprehensively detect defects and accurately position the defects.

Description

Technical Field

[0001] This invention relates to the field of 3D scanner technology, and in particular to a method and apparatus for detecting defects on the surface of an object, and a 3D scanner. Background Technology

[0002] A 3D scanner is a scientific instrument used to detect and analyze the shape and appearance data of objects or environments in the real world. The collected data is often used for 3D reconstruction calculations to create digital models of real objects in the virtual world. The purpose of a 3D scanner is to create point clouds of the geometric surface of an object. These points can be used to interpolate the surface shape of the object; the denser the point cloud, the more accurate the model can be created. The process of creating a model based on a point cloud is called 3D reconstruction. If the 3D scanner can acquire the surface color, material maps can be further applied to the reconstructed surface, a process known as material imprinting.

[0003] Existing 3D scanners are limited by resolution and have difficulty detecting and identifying small bumps and scratches on the surface of objects, which ultimately affects the accuracy of the created 3D model. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for detecting defects on the surface of an object, as well as a 3D scanner, which can solve the problem that existing 3D scanners are unable to detect and identify small bumps and scratches on the surface of an object.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] This invention provides a method for detecting surface defects of an object, wherein the method is applied to a 3D scanner, and the method includes:

[0007] The surface of the target object is scanned using a 3D scanner. During the scanning process, the light source on the 3D scanner is rapidly exposed sequentially at different angles according to the calibrated light source direction and calibrated order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain the first image.

[0008] Using photometric stereoscopic 3D reconstruction technology, based on the light source direction matrix obtained when calibrating the light source direction, the normal vector matrix and reflectivity distribution matrix of the target object surface are obtained.

[0009] Generate the gradient distribution matrix of the target object surface based on the normal vector matrix;

[0010] Based on the gradient distribution matrix and the reflectance distribution matrix, the surface of the target object is detected for defects and scratches.

[0011] Optionally, before the step of scanning the surface of the target object using a 3D scanner, the method further includes:

[0012] By using a spatial curved surface object with known reflectivity, the direction of the light source on the 3D scanner is calibrated to obtain the light source direction matrix when the light source is rapidly and sequentially exposed at different angles.

[0013] Optionally, the step of calibrating the light source direction of the 3D scanner using a spatial curved surface object with known reflectivity to obtain the light source direction matrix for rapid sequential exposure at different angles includes:

[0014] The three-dimensional scanner is used to scan a spatial curved surface object with known reflectivity. The light source of the three-dimensional scanner is exposed rapidly and sequentially at different angles. At the same time, the three-dimensional scanner acquires images of the spatial curved surface object to obtain a second image.

[0015] The point cloud data of the spatial curved surface object is obtained based on the second image;

[0016] The surface normal vector matrix of the spatial curved object is determined based on the point cloud data;

[0017] Based on the grayscale matrix and normal vector matrix of the second image, the light source direction matrix is ​​obtained when the light source is rapidly and sequentially exposed at different angles on the 3D scanner.

[0018] Optionally, the step of obtaining the point cloud data of the spatial curved surface object based on the second image includes:

[0019] Determine the full-brightness image and the coded image in the second image;

[0020] The full-brightness image and the coded image are processed by multi-view imaging to obtain the point cloud data of the surface of the spatial curved object.

[0021] Optionally, the step of obtaining the light source direction matrix of the light source during rapid sequential exposure at different angles on the 3D scanner, based on the grayscale matrix and normal vector matrix of the second image, includes:

[0022] For any pixel illuminated by the light source, the surface normal vector at each pixel is solved based on the spatial surface equation of the preset reference object.

[0023] Based on the surface normal vector, reflected light, reflectivity at the pixel, normal vector at the illuminated location, and a set of preset equations, the unit light source incident vector at the pixel is calculated.

[0024] The light source direction matrix of the 3D scanner during rapid sequential exposure of the light source at different angles is obtained by using the unit light source incident vector at each pixel under the illumination of the light source.

[0025] Optionally, the step of generating the gradient distribution matrix of the target object surface based on the normal vector matrix includes:

[0026] Based on the normal vector matrix, the distribution of the image gradient in the X and Y directions is determined;

[0027] The gradient distribution of the image is obtained based on the distribution of the gradient in the X and Y directions;

[0028] The gradient distribution matrix of the target object surface is determined based on the image gradient distribution.

[0029] Optionally, the step of detecting surface imperfections and scratches on the target object based on the gradient distribution matrix and the reflectance distribution matrix includes:

[0030] The gradient distribution matrix, reflectance distribution matrix and image grayscale matrix acquired by the 3D scanner camera are fused to form multi-channel surface information of the target to be detected;

[0031] Based on the multi-channel surface information of the target object, defects existing on the surface of the target object are detected.

[0032] This invention also provides a surface defect detection device for objects, applied to a 3D scanner, the device comprising:

[0033] The control module is used to scan the surface of a target object using a 3D scanner. During the scanning process, the light source on the 3D scanner is rapidly exposed sequentially at different angles according to the calibrated light source direction and calibrated order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain a first image.

[0034] The acquisition module is used to acquire the normal vector matrix and reflectance distribution matrix of the target object surface based on the light source direction matrix obtained when calibrating the light source direction using photometric stereo method three-dimensional reconstruction technology.

[0035] A generation module is used to generate a gradient distribution matrix of the surface of the target object based on the normal vector matrix;

[0036] The detection module is used to detect the unevenness and scratches on the surface of the target object based on the gradient distribution matrix and the reflectance distribution matrix.

[0037] Optionally, the device further includes:

[0038] The calibration module is used to calibrate the light source direction on the 3D scanner using a spatial curved surface object with known reflectivity before the control module scans the surface of the target object, thereby obtaining the light source direction matrix when the light source is rapidly and sequentially exposed at different angles.

[0039] Optionally, the calibration module includes:

[0040] The first submodule is used to scan a spatial curved surface object with known reflectivity using the 3D scanner. The light source of the 3D scanner is exposed rapidly and sequentially at different angles. At the same time, the 3D scanner acquires images of the spatial curved surface object to obtain a second image.

[0041] The second submodule is used to obtain point cloud data of the surface of the spatial curved object based on the second image;

[0042] The third submodule is used to determine the surface normal vector matrix of the spatial curved object based on the point cloud data;

[0043] The fourth submodule is used to obtain the light source direction matrix of the light source when it is rapidly and sequentially exposed at different angles on the 3D scanner, based on the grayscale matrix and normal vector matrix of the second image.

[0044] Optionally, the second submodule is specifically used for:

[0045] The fully illuminated image and the encoded image in the second image are determined; the fully illuminated image and the encoded image are processed by multi-view imaging to obtain the point cloud data of the surface of the spatial curved object.

[0046] Optionally, the fourth submodule is specifically used for:

[0047] For any pixel illuminated by the light source, the surface normal vector at each pixel is solved based on the spatial surface equation of the preset reference object.

[0048] Based on the surface normal vector, reflected light, reflectivity at the pixel, normal vector at the illuminated location, and a set of preset equations, the unit light source incident vector at the pixel is calculated.

[0049] The light source direction matrix of the 3D scanner during rapid sequential exposure of the light source at different angles is obtained by using the unit light source incident vector at each pixel under the illumination of the light source.

[0050] Optionally, the generation module includes:

[0051] The fifth submodule is used to determine the distribution of the image gradient in the X and Y directions based on the normal vector matrix;

[0052] The sixth submodule is used to obtain the gradient distribution of the image based on the distribution of the gradient of the image in the X and Y directions;

[0053] The seventh submodule is used to determine the gradient distribution matrix of the target object surface based on the image gradient distribution.

[0054] Optionally, the detection module includes:

[0055] The eighth submodule is used to fuse the gradient distribution matrix, reflectance distribution matrix and image grayscale matrix acquired by the 3D scanner camera to form multi-channel surface information of the target to be detected;

[0056] The ninth submodule is used to detect defects on the surface of the target object based on the multi-channel target surface information.

[0057] This invention provides an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of any of the above-described methods for detecting surface defects of an object.

[0058] This invention provides a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of any of the above-described object surface defect detection methods.

[0059] The surface defect detection scheme provided in this invention combines the feature information such as the normal vector distribution, reflectivity distribution, and gradient distribution of the target object surface to detect defects such as unevenness and scratches on the target object surface, making the defect detection more comprehensive and the location more accurate. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the steps of a method for detecting surface defects of an object according to an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of the structure of a 3D scanner;

[0062] Figure 3 This is a schematic diagram showing the layout of the LED light source and camera in a 3D scanner;

[0063] Figure 4 This is a structural block diagram illustrating an object surface defect detection device according to an embodiment of this application. Detailed Implementation

[0064] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0065] Photometric stereoscopic inspection is a method that uses multiple light source directions to estimate surface geometry. It can reconstruct the normal vector of an object's surface and the reflectivity of different points on the surface. It is most effective in reconstructing continuous, smooth target surfaces. When using photometric stereoscopic inspection to reconstruct target surfaces, it is necessary to pre-calibrate the illumination directions of LED light sources at different angles. Typically, a high-brightness metal sphere is photographed to infer the illumination direction. The process is relatively cumbersome and requires the operator to have some experience.

[0066] Existing 3D scanners, limited by resolution, struggle to detect and identify minute imperfections and scratches. Furthermore, the laser beams emitted during scanning negatively impact the detection of these small imperfections and scratches. Photometric stereoscopic inspection can accurately reconstruct the surface normal and reflectivity of an object, thus enabling imperfection detection; however, the light source orientation calibration process is complex. This application creatively combines these two methods, using a 3D scanner for light direction calibration in photometric stereoscopic inspection, thereby enabling the 3D scanner to detect minute imperfections and scratches.

[0067] The surface defect detection scheme provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0068] As attached Figure 1 As shown, the object surface defect detection method of this application embodiment includes the following steps:

[0069] Step 101: Use a 3D scanner to scan the surface of the target object. During the scanning process, the light source on the 3D scanner is exposed rapidly and sequentially at different angles according to the calibrated light source direction and calibrated order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain the first image.

[0070] An exemplary 3D scanner schematic diagram is shown below. Figure 2 As shown, a 3D scanner includes: a light source such as an LED (Light Emitting Diode) light source, a camera, and a laser. The laser is used to scan the object to be scanned, the LED light source is used to emit light, and the camera is used to acquire images. A schematic diagram of the layout of the LED light source and camera in a 3D scanner is shown below. Figure 3 As shown, the LED light sources are evenly distributed around the camera. It should be noted that... Figure 3 This is merely an illustrative example; the actual number and size of the LED light sources are not subject to change in the actual implementation. Figure 3 The limitations in the design can be flexibly set by those skilled in the art. The LED light source can also be non-uniformly or symmetrically distributed.

[0071] In this embodiment of the application, before using a 3D scanner to detect defects on the surface of a target object, it is necessary to calibrate the direction of the light source on the 3D scanner using a spatial curved surface object with known reflectivity, so as to obtain the light source direction matrix L when the light source is exposed rapidly and sequentially at different angles.

[0072] The target object is the object whose surface defect is to be detected, and its reflectivity is unknown. When scanning the surface of the target object with unknown reflectivity using a 3D scanner, the order in which the LED light source on the 3D scanner is rapidly exposed at different angles should be the same as the order of light source orientation calibration. At the same time, the LED light source is attached to the 3D scanner, and the camera and the light source remain relatively stationary. Since the camera acquires images extremely quickly, while the 3D scanner scans the target object relatively slowly, it can be assumed that the light source orientation matrix L remains unchanged during the light source orientation calibration process and the actual scanning process.

[0073] Step 102: Using photometric stereoscopic 3D reconstruction technology, based on the light source direction matrix obtained when calibrating the light source direction, obtain the normal vector matrix and reflectivity distribution matrix of the target object surface.

[0074] The normal vector matrix and reflectance distribution matrix of the target object's surface can be obtained as follows:

[0075] Determine the light source direction matrix L, use a 3D scanner to scan the surface of a target object with unknown reflectivity, assume that the LED light source on the 3D scanner is exposed rapidly at different angles for f times, and at the same time the 3D scanner completes the image acquisition of the target object surface, and assume that the image resolution is w*h, then the grayscale matrix M of the target object surface image is f, and the columns are w*h.

[0076] The formula is M = ELNP, where M is the grayscale matrix of the target surface image with rows f and columns w*h, and E is the light source intensity matrix diag(e1, e2...e) of the LED light source after f rapid, sequential exposures at different angles. f L is the LED light source direction matrix with rows f and columns 3, N is the target object surface unit normal vector matrix with rows 3 and columns w*h, and P is the reflectivity distribution matrix with rows w*h and columns w*h, where M, E, and L are known quantities, and N and P are unknown quantities.

[0077] The formula M = ELNP can be expressed as M = L′N′, where L′ = EL and N′ = NP. Using the least squares method (the method is not unique), we can solve for N′, resulting in N′ = (L′... T L′) -1 L′ T

[0078] After solving for N′, we have N′ = NP.

[0079]

[0080] Therefore, ρ i (n ix +n iy +n iz )=n′ i , n′ i Since it is a unit normal vector, we have (n ix +n iy +n iz ) = 1,

[0081] Therefore:

[0082]

[0083] At this point, the unit normal vector matrix N and the reflectivity matrix P of the target object's surface have been solved.

[0084] Step 103: Generate the gradient distribution matrix of the target object surface based on the normal vector matrix.

[0085] A feasible method for generating the gradient distribution matrix of a target object's surface based on the normal vector matrix is ​​as follows:

[0086] First, based on the normal vector matrix, the distribution of the image gradient in the X and Y directions is determined;

[0087]

[0088] Secondly, based on the distribution of the image gradient in the X and Y directions, the gradient distribution of the image is obtained;

[0089]

[0090] Then, based on the image gradient distribution, the gradient distribution matrix G of the target object surface is determined.

[0091] Step 104: Based on the gradient distribution matrix and reflectance distribution matrix, detect the unevenness and scratches on the surface of the target object.

[0092] In practical implementation, the gradient distribution matrix, reflectance distribution matrix and grayscale matrix of the image acquired by the 3D scanner camera can be fused to form multi-channel surface information of the target to be detected; based on the multi-channel surface information of the target to be detected, defects existing on the surface of the target object can be detected.

[0093] The specific method for detecting defects on the surface of a target object based on multi-channel surface information can be flexibly set by those skilled in the art, and no specific limitations are imposed on this embodiment. Existing related schemes for detecting surface defects based on surface information can also be referenced.

[0094] In one optional embodiment, calibrating the light source direction of the 3D scanner using a spatial curved surface object with known reflectivity to obtain the light source direction matrix for rapid sequential exposure at different angles may include the following sub-steps:

[0095] Sub-step 1: Use a 3D scanner to scan a spatial curved surface object with known reflectivity. The light source of the 3D scanner is exposed rapidly and sequentially at different angles. At the same time, the 3D scanner acquires images of the spatial curved surface object to obtain a second image.

[0096] The number of second images is greater than or equal to 3.

[0097] Sub-step 2: Obtain point cloud data of the spatial curved object surface based on the second image;

[0098] One feasible approach is to determine the fully illuminated image and the coded image in the second image; and then process the fully illuminated image and the coded image through multi-view imaging to obtain point cloud data of the surface of the spatial curved object.

[0099] Sub-step 3: Determine the surface normal vector matrix of the spatial curved object based on the point cloud data;

[0100] Sub-step four: Based on the grayscale matrix and normal vector matrix of the second image, obtain the light source direction matrix when the light source is rapidly and sequentially exposed at different angles on the 3D scanner.

[0101] One feasible approach is as follows:

[0102] First, for any pixel illuminated by the light source, the surface normal vector at each pixel is solved based on the spatial surface equation of the preset reference object.

[0103] Let the equation of the spatial surface be Ax + By + Cz + D = 0, then the surface normal vector at any point is (A, B, ), which, when normalized, becomes

[0104] Secondly, based on the surface normal vector, reflected light, reflectivity at the pixel, normal vector at the illuminated location, and a set of preset equations, the unit light source incident vector at the pixel is calculated.

[0105] For Lambert reflection, the following holds: I∝cosθ I represents the reflected light captured by the camera, k is the scaling factor, θ is the angle between the unit normal vector n and the unit incident vector l at the illuminated point, e is the light intensity at that point, which can be considered as unit light intensity, ρ is the reflectivity at that point, and l is set to (l x , l y , l z ),but

[0106] Among them, I, ρ, Given that, we can derive the unit light source incident vector l = (l...) from three equations. These equations can then be used to calculate the unit light source incident vector l at that pixel. x , l y , l z ).

[0107] Finally, by using the unit light source incident vector at each pixel under the illumination of the light source, the light source direction matrix is ​​obtained when the light source is rapidly and sequentially exposed at different angles on the 3D scanner.

[0108] By applying the above method of calculating the incident vector of a unit light source at each pixel to the calculation of the incident vector of a unit light source at multiple pixels under multiple illumination angles, the light source direction matrix L can be obtained from the grayscale matrix, normal vector matrix, and reflectivity of the target surface image.

[0109] The surface defect detection method provided in this application combines the feature information such as the normal vector distribution, reflectivity distribution, and gradient distribution of the target object surface to detect defects such as unevenness and scratches on the target object surface, making the defect detection more comprehensive and the location more accurate.

[0110] The method for detecting surface defects of objects provided in this application will be described below with reference to a specific embodiment.

[0111] The surface defect detection method for objects in this application mainly includes two parts: 3D scanner LED light source angle calibration and formal scanning after light source calibration to detect surface defects of the target object.

[0112] The purpose of LED light source orientation calibration is to obtain the light source orientation matrix during rapid sequential exposure of the LED light source at different angles, including the following steps:

[0113] S1: Use a 3D scanner to scan a metal sphere with reflectivity ρ. The LED light source on the 3D scanner rapidly and sequentially exposes the surface of the metal sphere at different angles, while the 3D scanner simultaneously acquires images of the metal sphere's surface, obtaining at least three image data points.

[0114] The three image data constitute the second image. In this embodiment, a metal sphere is used as a reference object for illustration. In actual implementation, the reference object is not limited to a metal sphere; it can be any other suitable spatial surface object, which can be selected by those skilled in the art according to actual needs.

[0115] The scanner captures images of the surface of the metal sphere using a camera. The number of images captured can be any number greater than or equal to 3, such as 3, 4, or 5.

[0116] S2: Obtain point cloud data of the metal sphere surface from the image acquired by the 3D scanner, and then obtain the surface normal vector matrix N of the metal sphere. q ;

[0117] Specifically, a fully illuminated image and an encoded image of the metal sphere's surface are acquired, and multi-view imaging is used to obtain point cloud data of the metal sphere's surface. Based on the point cloud data, the normal vector matrix N of the metal sphere's surface is obtained. q ;

[0118] S3: Based on the grayscale matrix M of the metal sphere surface image q and normal vector matrix N q The light source direction matrix L and the light intensity matrix E are obtained when the LED light source on the 3D scanner is exposed rapidly at different angles.

[0119] The specific implementation steps of S3 can be as follows:

[0120] Let the equation of the spatial surface be Ax + By + Cz + D = 0. Then the surface normal vector at any point is (A, B, C), which, after normalization, becomes...

[0121] For Lambert reflection, the following holds: I∝cosθ I represents the reflected light captured by the camera, k is the scaling factor, θ is the angle between the unit normal vector n and the unit incident vector l at the illuminated point, e is the light intensity at that point, which can be considered as unit light intensity, ρ is the reflectivity at that point, and l is set to (l x , l y , l z ),but

[0122] Among them, I, ρ, Given that, we can derive the incident vector of the unit light source at that point, l = (l_inc.), from three equations. x , l y , l z Similarly, the grayscale matrix M of the metal sphere surface image... q and normal vector matrix N q The light source direction matrix L is derived from the reflectivity ρ.

[0123] Thus, the LED light source angle calibration of the 3D scanner was completed by executing S1-S3. The specific procedure for formal scanning to detect surface defects on the target object after light source calibration is as follows:

[0124] S4: When using a 3D scanner to scan the surface of a target object with unknown reflectivity, the order in which the LED light source on the 3D scanner is rapidly exposed at different angles should be the same as the order of light source orientation calibration. At the same time, the LED light source is attached to the 3D scanner, and the camera and the light source remain relatively stationary. Since the camera acquires images extremely quickly, while the target object being scanned by the 3D scanner moves relatively slowly, it can be assumed that the light source orientation calibration process and the light source orientation matrix L remain unchanged during the actual scanning process.

[0125] S5: The light source intensity matrix E is solved by minimizing the alternation of matrices, and the normal vector matrix N and reflectivity distribution matrix P of the target surface to be scanned are obtained by three-dimensional reconstruction using the photometric stereo method.

[0126] The specific implementation steps of S5 are as follows;

[0127] After determining the light source direction matrix L and the light source intensity matrix E, a 3D scanner is used to scan the surface of a target object with unknown reflectivity. Assume that the LED light source on the 3D scanner is rapidly exposed f times at different angles, and at the same time, the 3D scanner completes the image acquisition of the target object surface. Assume that the image resolution is w*h, then the grayscale matrix M of the target object surface image is f, and the columns are w*h.

[0128] The formula is M = ELNP, where M is the grayscale matrix of the target object surface image with rows f and columns w*h, and E is the light source intensity matrix diag(e1, e2...e) of the LED light source after f rapid, sequential exposures at different angles. f L is the LED light source direction matrix with rows f and columns 3, N is the target object surface unit normal vector matrix with rows 3 and columns w*h, and P is the reflectivity distribution matrix with rows w*h and columns w*h, where M, E, and L are known quantities, and N and P are unknown quantities.

[0129] The formula M = ELNP can be expressed as M = L′N′, where L′ = EL and N′ = NP. Using the least squares method (the method is not unique), we can solve for N′, resulting in N′ = (L′... T L′) -1 L′ T

[0130] After solving for N′, we have N′ = NP.

[0131]

[0132] Therefore there is n′ i Since it is a unit normal vector, we have (n ix +n iy +n iz ) = 1,

[0133] Therefore:

[0134]

[0135] At this point, the unit normal vector matrix N and the reflectivity matrix P of the target object's surface have been solved.

[0136] S6: Obtain the gradient distribution matrix G of the target surface to be scanned through the normal vector matrix.

[0137] The specific implementation steps of S6 are as follows:

[0138] The gradient distribution of the image in the X and Y directions can be obtained from the normal vector matrix N obtained above:

[0139] have,

[0140] The gradient distribution of the image can be obtained from the gradient distribution in the X and Y directions described above.

[0141] have Thus, the gradient distribution matrix G on the surface of the target object is obtained.

[0142] S7: Detects surface imperfections and scratches on target objects by using gradient distribution and reflectivity distribution.

[0143] When detecting surface imperfections and segmentation of a target object, the gradient distribution matrix, reflectance distribution matrix, and grayscale matrix acquired by the camera can be fused to form multi-channel target object surface information. This step includes normalization processing, which normalizes the gradient distribution matrix, reflectance distribution matrix, and grayscale matrix acquired by the camera. Then, using the fused multi-channel target object surface information, defect detection is performed on the target object surface. The specific method used for detection can be a traditional method or a deep learning-based method.

[0144] A feasible method for detecting surface defects and scratches on a target object by using fused multi-channel target object surface information is as follows:

[0145] Threshold segmentation is performed on the reflectance distribution image transformed from the reflectance distribution matrix P;

[0146] Use the connected component labeling function to label the regions of bumps and scratches as connected components.

[0147] Find the geometric center point of the area with bumps and scratches and the farthest distance from the center of the area;

[0148] Using the aforementioned geometric center point and farthest distance, mark the areas containing bumps, imperfections, and scratches in the reflectivity distribution image.

[0149] Threshold segmentation is performed on the gradient distribution image transformed from the gradient distribution image G;

[0150] Use the connected component labeling function to label the regions of bumps and scratches as connected components.

[0151] Find the geometric center point of the area with bumps and scratches and the farthest distance from the center of the area;

[0152] Using the geometric center point and the furthest distance described above, mark the areas containing bumps, blemishes, and scratches in the gradient distribution image.

[0153] The original image displays the union of the regions marked with bumps and scratches in the reflectance distribution image and the regions marked with bumps and scratches in the gradient distribution image.

[0154] The 3D scanner has now completed its inspection of bumps, dents, and scratches.

[0155] Figure 4 The structural block diagram of an object surface defect detection device according to an embodiment of this application is shown.

[0156] The object surface defect detection device provided in this application embodiment is applied to a 3D scanner, and the object surface defect detection device includes the following functional modules:

[0157] Control module 401 is used to scan the surface of a target object using a 3D scanner. During the scanning process, the light source on the 3D scanner is rapidly exposed sequentially at different angles according to the calibrated light source direction calibration order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain a first image.

[0158] The acquisition module 402 is used to acquire the normal vector matrix and reflectance distribution matrix of the target object surface based on the light source direction matrix obtained when calibrating the light source direction using photometric stereo method three-dimensional reconstruction technology.

[0159] The generation module 403 is used to generate the gradient distribution matrix of the surface of the target object based on the normal vector matrix;

[0160] The detection module 404 is used to detect the unevenness and scratches on the surface of the target object based on the gradient distribution matrix and the reflectance distribution matrix.

[0161] Optionally, the device further includes:

[0162] The calibration module is used to calibrate the light source direction on the 3D scanner using a spatial curved surface object with known reflectivity before the control module scans the surface of the target object, thereby obtaining the light source direction matrix when the light source is rapidly and sequentially exposed at different angles.

[0163] Optionally, the calibration module includes:

[0164] The first submodule is used to scan a spatial curved surface object with known reflectivity using the 3D scanner. The light source of the 3D scanner is exposed rapidly and sequentially at different angles. At the same time, the 3D scanner acquires images of the spatial curved surface object to obtain a second image.

[0165] The second submodule is used to obtain point cloud data of the surface of the spatial curved object based on the second image;

[0166] The third submodule is used to determine the surface normal vector matrix of the spatial curved object based on the point cloud data;

[0167] The fourth submodule is used to obtain the light source direction matrix of the light source when it is rapidly and sequentially exposed at different angles on the 3D scanner, based on the grayscale matrix and normal vector matrix of the second image.

[0168] Optionally, the second submodule is specifically used for:

[0169] The fully illuminated image and the encoded image in the second image are determined; the fully illuminated image and the encoded image are processed by multi-view imaging to obtain the point cloud data of the surface of the spatial curved object.

[0170] Optionally, the fourth submodule is specifically used for:

[0171] For any pixel illuminated by the light source, the surface normal vector at each pixel is solved based on the spatial surface equation of the preset reference object.

[0172] Based on the surface normal vector, reflected light, reflectivity at the pixel, normal vector at the illuminated location, and a set of preset equations, the unit light source incident vector at the pixel is calculated.

[0173] The light source direction matrix of the 3D scanner during rapid sequential exposure of the light source at different angles is obtained by using the unit light source incident vector at each pixel under the illumination of the light source.

[0174] Optionally, the generation module includes:

[0175] The fifth submodule is used to determine the distribution of the image gradient in the X and Y directions based on the normal vector matrix;

[0176] The sixth submodule is used to obtain the gradient distribution of the image based on the distribution of the gradient of the image in the X and Y directions;

[0177] The seventh submodule is used to determine the gradient distribution matrix of the target object surface based on the image gradient distribution.

[0178] Optionally, the detection module includes:

[0179] The eighth submodule is used to fuse the gradient distribution matrix, reflectance distribution matrix and image grayscale matrix acquired by the 3D scanner camera to form multi-channel surface information of the target to be detected;

[0180] The ninth submodule is used to detect defects on the surface of the target object based on the multi-channel target surface information.

[0181] The object surface defect detection device provided in this application combines the characteristic information such as the normal vector distribution, reflectivity distribution, and gradient distribution of the target object surface to detect defects such as unevenness and scratches on the target object surface, making the defect detection more comprehensive and the positioning more accurate.

[0182] In the embodiments of this application Figure 4 The surface defect detection device shown can be a device, or it can be a component, integrated circuit, or chip in a 3D scanner. (This is from an embodiment of the present application.) Figure 4 The surface defect detection device shown can be a device with an operating system.

[0183] The embodiments provided in this application Figure 4 The surface defect detection device shown can achieve Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0184] Optionally, embodiments of this application also provide a 3D scanner, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When executed by the processor, the program or instructions implement the various processes of the above-described object surface defect detection method embodiments and achieve the same technical effects. To avoid repetition, further details are omitted here. Furthermore, the 3D scanner also includes light sources such as LED light sources and cameras.

[0185] It should be noted that the electronic device in this application embodiment includes the server described above.

[0186] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described object surface defect detection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0187] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0188] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described method embodiment for removing surface defects of objects, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0189] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0190] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0191] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting defects on the surface of an object, characterized in that, Applied to a 3D scanner, the method includes: The surface of the target object is scanned using a 3D scanner. During the scanning process, the light source on the 3D scanner is rapidly exposed sequentially at different angles according to the calibrated light source direction and calibrated order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain the first image. Using photometric stereoscopic 3D reconstruction technology, based on the light source direction matrix obtained when calibrating the light source direction, the normal vector matrix and reflectivity distribution matrix of the target object surface are obtained. Generate the gradient distribution matrix of the target object surface based on the normal vector matrix; Based on the gradient distribution matrix and the reflectance distribution matrix, the surface of the target object is detected for unevenness and scratches; before the step of scanning the surface of the target object with a 3D scanner, the method further includes: scanning a spatial curved surface object with known reflectance using the 3D scanner, the light source of the 3D scanner being rapidly and sequentially exposed at different angles, and the 3D scanner simultaneously acquiring an image of the spatial curved surface object to obtain a second image; The point cloud data of the spatial curved surface object is obtained based on the second image; The surface normal vector matrix of the spatial curved object is determined based on the point cloud data; Based on the grayscale matrix and normal vector matrix of the second image, the light source direction matrix is ​​obtained when the light source is rapidly and sequentially exposed at different angles on the 3D scanner.

2. The method according to claim 1, characterized in that, The step of obtaining the point cloud data of the surface of the spatial curved object based on the second image includes: Determine the full-brightness image and the coded image in the second image; The full-brightness image and the coded image are processed by multi-view imaging to obtain the point cloud data of the spatial curved object surface.

3. The method according to claim 1, characterized in that, The step of obtaining the light source direction matrix of the 3D scanner during rapid sequential exposure at different angles, based on the grayscale matrix and normal vector matrix of the second image, includes: For any pixel illuminated by the light source, the surface normal vector at each pixel is solved based on the spatial surface equation of the preset reference object. Based on the surface normal vector, reflected light, reflectivity at the pixel, normal vector at the illuminated location, and a set of preset equations, the unit light source incident vector at the pixel is calculated. The light source direction matrix of the 3D scanner during rapid sequential exposure of the light source at different angles is obtained by using the unit light source incident vector at each pixel under the illumination of the light source.

4. The method according to claim 1, characterized in that, The step of generating the gradient distribution matrix of the target object surface based on the normal vector matrix includes: Based on the normal vector matrix, the distribution of the image gradient in the X and Y directions is determined; The gradient distribution of the image is obtained based on the distribution of the gradient in the X and Y directions; The gradient distribution matrix of the target object surface is determined based on the image gradient distribution.

5. The method according to claim 1, characterized in that, The step of detecting surface imperfections and scratches on the target object based on the gradient distribution matrix and the reflectance distribution matrix includes: The gradient distribution matrix, reflectance distribution matrix and grayscale matrix acquired by the 3D scanner camera are fused to form multi-channel surface information of the target to be detected. Based on the multi-channel surface information of the target object, defects existing on the surface of the target object are detected.

6. A device for detecting surface defects of an object, characterized in that, The device, used in a 3D scanner, includes: The control module is used to scan the surface of a target object using a 3D scanner. During the scanning process, the light source on the 3D scanner is rapidly exposed sequentially at different angles according to the calibrated light source direction and calibrated order. At the same time, the 3D scanner acquires images of the surface of the target object to obtain a first image. The acquisition module is used to acquire the normal vector matrix and reflectance distribution matrix of the target object surface based on the light source direction matrix obtained when calibrating the light source direction using photometric stereo method three-dimensional reconstruction technology. A generation module is used to generate a gradient distribution matrix of the surface of the target object based on the normal vector matrix; A detection module is used to detect surface imperfections and scratches on the target object based on the gradient distribution matrix and the reflectance distribution matrix; wherein, the device further includes: The calibration module is used to calibrate the light source direction on the 3D scanner using a spatial curved surface object with known reflectivity before the control module scans the surface of the target object, so as to obtain the light source direction matrix when the light source is rapidly and sequentially exposed at different angles. The calibration module includes: The first submodule is used to scan a spatial curved surface object with known reflectivity using the 3D scanner. The light source of the 3D scanner is exposed rapidly and sequentially at different angles. At the same time, the 3D scanner acquires images of the spatial curved surface object to obtain a second image. The second submodule is used to obtain point cloud data of the surface of the spatial curved object based on the second image; The third submodule is used to determine the surface normal vector matrix of the spatial curved object based on the point cloud data; The fourth submodule is used to obtain the light source direction matrix of the light source when it is rapidly and sequentially exposed at different angles on the 3D scanner, based on the grayscale matrix and normal vector matrix of the second image.

7. A three-dimensional scanner, comprising a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the object surface defect detection method as described in any one of claims 1-5.

Images