A 3D measurement method for high-reflection surfaces using structured light based on the main and auxiliary dual-view multi-gray-level projection

Abstract

The present invention discloses a three-dimensional measurement method for high-reflection surface structured light based on main and auxiliary dual-view multi-gray-level projection. Based on the light-reflecting characteristics of the high-reflection surface, a three-dimensional reconstruction system for structured light with main and auxiliary dual-views is established. By additionally introducing an auxiliary reconstruction view, it is possible to avoid image overexposure caused by specular reflection during the single-projection reconstruction process. For the regions where the image is overexposed in the main reconstruction view, image reconstruction in the auxiliary view is used for compensation, enabling more points to be reconstructed in a single view, improving the measurement speed and efficiency. At the same time, the method of sharing the same projector also saves hardware costs well. Combining with the multi-gray-level projection method, by gradually reducing the projection intensity, points that are overexposed in both the main and auxiliary dual-views are reconstructed, thereby reconstructing a complete high-reflection surface. Meanwhile, a method for discriminating overexposed points in dual-views is proposed to determine whether the point is overexposed in both the main and auxiliary views.

Description

Technical Field

[0001] The present invention relates to the field of optical measurement, and more specifically, to a three-dimensional measurement method of structured light for high-reflection surfaces based on main and auxiliary dual-view multi-gray-level projection. Background Art

[0002] In the real world, all objects perceived by humans are three-dimensional. In the past few decades, with the rapid development of technology, three-dimensional shape measurement technology has found more and more applications in fields such as industrial manufacturing, biomedicine, national defense security, animation production, virtual reality, and 3D printing. Due to the advantages of structured light technology such as full-field scanning, high resolution, and high precision, and thanks to the rapid development of DLP (Digital Light Processing) technology, the projected structured light patterns can be flexibly selected through computer programming, maximizing the measurement accuracy and measurement speed. These advantages make it the most promising three-dimensional data acquisition method in current engineering applications.

[0003] However, there are still some problems in the application of structured light three-dimensional measurement technology. For example, the ambient light during measurement is required to be limited within a certain range, the object to be measured must be a diffuse reflection surface, and the surface reflectivity change range is not large. When the input intensity exceeds the photosensitive range of the camera sensor, overexposure will occur in the image, and underexposure will occur when the projection intensity is reduced. Both of these situations will lead to obvious errors in the extracted phase map, thus resulting in the measurement of the three-dimensional shape.

[0004] Currently, there are mainly two three-dimensional measurement schemes for high-reflection surfaces using structured light: one is to design a complex optical system, such as using optical methods such as multi-angle projection, multi-angle shooting, and multi-spectral to eliminate overexposure; the other is to adopt high dynamic range imaging (HDRI) technology, adjust the camera exposure or projection intensity, and collect multiple frames of images to be fused into an HDR image to eliminate overexposure. Both of these methods can compensate for the overexposed area and solve the overexposure problem to a certain extent, but each has certain defects. Among them, the complex optical system method requires precise design of the system to achieve an ideal effect, the structure is relatively complex, and at the same time, the system cost is high; while the HDRI technology has a long projection sequence, resulting in too long measurement time and low measurement efficiency; in addition, the above methods have problems that the pixel points near the specular reflection angle cannot be completely reconstructed, and the integrity of point cloud reconstruction needs to be improved.

[0005] Summary of the Invention

[0006] To solve the deficiencies of the prior art, the purpose of the present invention is to provide a three-dimensional measurement method of structured light for high-reflection surfaces based on main and auxiliary dual-view multi-gray-level projection:

[0007] 1) Based on the light reflection characteristics of a highly reflective surface, a three-dimensional reconstruction system of structured light with a main and auxiliary dual-view structure is established. Compared with the traditional binocular structured light system, this system is composed of two monocular reconstruction systems sharing one projector. By introducing an additional auxiliary reconstruction view, it is possible to avoid image overexposure caused by specular reflection during a single projection reconstruction process. For the areas with overexposed images in the main reconstruction view, the image reconstruction in the auxiliary view is used for compensation, enabling more points to be reconstructed in a single view, improving the measurement speed and efficiency. At the same time, the method of sharing the same projector also saves hardware costs well.

[0008] 2) Considering the complexity of the surface shape of the measured object, relying solely on two reconstruction views cannot ensure that all overexposed points on the surface can be complementarily reconstructed, and adding additional reconstruction views will increase hardware costs. Therefore, this method combines the multi-gray-level projection method on the basis of this system. By gradually reducing the projection intensity, the points that are overexposed in both the main and auxiliary dual-views are reconstructed, and then a complete highly reflective surface is reconstructed. At the same time, a method for discriminating overexposed points in the dual-view is proposed to determine whether the point is overexposed in both the main and auxiliary views, and during the reconstruction process, the points that are repeatedly reconstructed in the main and auxiliary dual-views are deleted.

[0009] The present invention combines the advantages of the above two methods to achieve efficient and high-precision three-dimensional measurement of highly reflective surfaces. Compared with the traditional single-view method, the proposed method can reconstruct more points at each projection intensity, especially for the pixels around the specular reflection angle, reducing the number of required projection intensities to save time consumption. First, a three-dimensional measurement method of structured light for highly reflective surfaces based on a main and auxiliary dual-view multi-gray-level projection is disclosed, including:

[0010] To achieve the above objectives, the technical methods adopted in the present invention are as follows:

[0011] A structured light three-dimensional measurement method for highly reflective surfaces based on a main and auxiliary dual-view multi-gray-level projection is disclosed, including:

[0012] Construct a main and auxiliary dual-view structured light reconstruction system;

[0013] Calibrate the structured light reconstruction system to obtain the internal and external parameter matrices of the main-view camera, auxiliary-view camera, and projector;

[0014] Project a uniform grayscale image onto the surface of the object to be measured. The main-view camera and the auxiliary-view camera simultaneously capture an image. After binarizing the image, the overexposure masks of the main-view camera and the auxiliary-view camera are obtained;

[0015] Project structured light sinusoidal phase-shift fringes onto the surface of the object to be measured. Use the phase-shift method to solve for the phase to obtain the absolute phase of the corresponding object surface under the perspectives of the main-view camera and the auxiliary-view camera. Calculate the spatial three-dimensional coordinates of the pixel points in the overexposed area of the overexposure mask for the auxiliary-view camera by using the internal parameter matrices of the main-view camera, the auxiliary-view camera, the projector, and the external parameter matrices of the main-view camera, the auxiliary-view camera, and the projector. Reproject the three-dimensional space points corresponding to the pixel points in the non-overexposed area of the auxiliary camera to the overexposure mask of the main-view camera image by using the internal and external parameter matrices of the main-view camera to obtain the overall overexposure judgment mask;

[0016] Perform a de-duplication process on the overall overexposure judgment mask to obtain the three-dimensional reconstruction point cloud corresponding to the remaining points;

[0017] Judge whether there are still overexposed points in the three-dimensional reconstruction points corresponding to the remaining points that have not been processed. If there are, reduce the projection intensity and repeat the above projection and acquisition process and calculate the overexposure masks under the corresponding intensities. Reconstruct the non-overexposed points and store the results. If not, it means that all overexposed points have been reconstructed;

[0018] Merge the three-dimensional reconstruction point clouds under each projection intensity stored and output the final reconstruction result.

[0019] As a further improvement, the main-auxiliary dual-view structured light reconstruction system of the present invention includes a main-view camera, an auxiliary-view camera, and a projector. The object to be measured is placed stationary under the lens of the projector. The main-view camera and the auxiliary-view camera are placed on both sides of the projector. The distances between the main-view camera and the auxiliary-view camera and the projector are respectively in the range of 100 mm to 150 mm, and the included angles with the projector are respectively in the range of 10° to 30°;

[0020] As a further improvement, for the step of reprojecting the three-dimensional space points corresponding to the pixel points in the non-overexposed area of the auxiliary-view camera to the overexposure mask of the main-view camera image by using the internal and external parameter matrices of the main-view camera to obtain the overall overexposure judgment mask, specifically:

[0021] Use the internal and external parameter matrices K m and T m of the main-view camera to reproject a point P auxiliary (u a , v a ) in the non-overexposed area of the auxiliary-view camera to the corresponding three-dimensional space point P object (x, y, z) to the overexposure mask of the main-view camera image:

[0022] s c [u′ a ,v′ a ,1] T =K m T m [x,y,z,1] T

[0023] where (u′ a ,v′ a ) are the pixel coordinates calculated by reprojection, s c is the scale factor. Considering that the calculated coordinate values contain decimal parts, the coordinates are rounded to integers by the method of rounding:

[0024] [u″ a ,v″ a T =Round([u′ a ,v″ a T )

[0025] where (u″ a ,v″ a ) are the obtained mapped point coordinates. By performing the above reprojection operation on the non-overexposed pixels in all auxiliary view cameras, the overall overexposure judgment mask M′ k can be obtained, which can be expressed as:

[0026]

[0027] As a further improvement, on the basis of obtaining the total overexposure mask, the mask is further processed to remove duplicates, specifically:

[0028] Remove the pixels that have not been overexposed in the (k - 1)-th projection process from the overexposure mask obtained in the k-th projection:

[0029]

[0030] where M′ k-1 、M′ k are the total overexposure masks obtained in the (k - 1)-th and k-th projection processes respectively, is the total overexposure mask after duplicate removal in the k-th projection process. The points with a value of 255 in the mask are the points that can be reconstructed in the k-th projection process.

[0031] Compared with the prior art, the beneficial effects achieved by the present invention:

[0032] ​​1. Simple setup and low cost: The proposed main and auxiliary dual-view structured light reconstruction system in the present invention realizes dual-angle complementary reconstruction of the object to be measured by sharing the same projector with two cameras, avoiding the use of more complex system designs, with a simpler structure and lower equipment cost.

[0033] 2. High reconstruction efficiency: In traditional multi-gray-level projection reconstruction methods, when processing pixel points near the specular reflection angle, it is necessary to reduce the intensity projection multiple times, consuming a large amount of time. The proposed method combines the advantages of dual-view reconstruction, and uses the auxiliary view complementary reconstruction method for pixel points near the specular reflection angle to quickly compensate for a large number of overexposed pixels and improve the reconstruction efficiency of a single reconstruction process. At the same time, this method can reduce the number of projections used, thereby improving the overall reconstruction measurement speed.

[0034] 3. High reconstruction integrity: In the face of some strongly reflective surfaces, no matter how the projection intensity is adjusted for pixel points near the reflection angle, overexposure will occur, resulting in holes in the reconstructed point cloud and affecting the measurement accuracy. The method of the present invention can well solve this problem by combining dual-angle complementary reconstruction and multi-gray-level projection methods. By avoiding the strong specular reflection light from two angles and gradually reducing the projection intensity for complementary reconstruction on this basis, the integrity of the three-dimensional reconstruction of the overall highly reflective surface is improved, and the measurement accuracy is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the structure of the main and auxiliary dual-view structured light three-dimensional measurement system of the present invention.

[0036] In the figure, 1 is the main-view monocular structured light system; 2 is the auxiliary-view monocular structured light system; 3 is the projector; 4 is the main-view camera; 5 is the auxiliary-view camera; 6 is the object to be measured;

[0037] Figure 2 It is the data flow diagram of the method of the present invention;

[0038] Figure 3 It is a schematic diagram of the pixel reprojection of the auxiliary-view camera. DETAILED DESCRIPTION OF THE INVENTION

[0039] The following further illustrates the technical solutions of the present invention through specific embodiments in conjunction with the accompanying drawings of the specification:

[0040] The purpose of the present invention is to propose a three-dimensional structured light measurement method for highly reflective surfaces based on main and auxiliary dual-view multi-gray-level projection in view of the poor reconstruction efficiency and reconstruction quality near the specular reflection angle of existing three-dimensional structured light measurement methods for highly reflective surfaces.

[0041] Figure 1Schematically shows the configured master-slave dual-view structured light three-dimensional measurement system. This system consists of two monocular structured light reconstruction systems sharing the same projector 3, including a master-view camera 4, an auxiliary-view camera 5, and a projector 3. The master-view camera 4 and the projector 3 form the master-view monocular structured light system 1, and the auxiliary-view camera 5 and the projector 3 form the auxiliary-view monocular structured light system 2. The object to be measured 6 is placed stationary below the projector 3. The master-view camera 4 and the auxiliary-view camera 5 are placed on both sides of the projector 3. The distance range between the camera and the projector 3 is from 100 mm to 150 mm, and the angle range between the camera and the projector 3 is from 10° to 30°; from 100 mm to 150 mm, and the angle range between the camera and the projector 3 is from 10° to 30°. This configuration enables the object to be measured 6 to be within the common field of view of the master and auxiliary-view cameras 5. At the same time, it enables the overexposed points near the specular reflection angle in the master-view camera 4 to avoid the specular reflection angle during the acquisition process of the auxiliary-view camera 5 and not generate overexposure, thereby achieving complementary reconstruction to solve the problem of overexposure in the image of the master-view camera 4.

[0042] Figure 2 is the data flow diagram of the present invention.

[0043] The specific implementation method of the present invention is as follows:

[0044] Step 1: Before measurement, calibrate the structured light system. Collect checkerboard images to calibrate the camera. Project horizontal and vertical phase-shifted fringes through the projector 3, and the camera collects phase-shifted images and solves the phase to obtain the absolute phase. Calibrate the projector 3 to obtain the internal and external parameter matrices of the master and auxiliary-view cameras 5 and the projector 3.

[0045] Step 2: First, project a uniform gray-scale image onto the surface of the object to be measured 6 (when first projected, the gray scale of this image is the maximum intensity value 255). The master and auxiliary-view cameras 5 simultaneously collect an image. After binarizing the image, an overexposure mask M main and M auxiliary are obtained. The overexposure mask can be expressed as:

[0046]

[0047] where M k is the overexposure mask obtained from the k-th projection, is the intensity at the pixel (u c , v c ) in the image collected by the camera, and I thresholdis the overexposure judgment threshold (usually set to a value slightly lower than 255 to reduce the influence of noise). Pixels with a median value of 255 in the overexposure mask are underexposed pixels, and 3D reconstruction can be performed on the current intensity through the structured light phase-shift method. Pixels with a value of 0 are overexposed pixels and cannot be reconstructed due to the lack of phase information;

[0048] Step three, as Figure 3 shown, the projector 3 projects the structured light sinusoidal phase-shift fringes onto the object to be measured 6, and uses the phase-shift method to solve the phase to obtain the absolute phase of the corresponding object surface under the main and auxiliary viewpoints Calculate the spatial three-dimensional coordinates of the pixels in the underexposed area of the camera overexposure mask in the auxiliary viewpoint through the camera internal parameter matrix, projector 3 internal parameter matrix, and the external parameter matrices of the two obtained in step one (that is, the points with a median value of 255 in M k . Use the internal and external parameter matrices K m and T m of the main-viewpoint camera 4 to re-project a point P auxiliary (u a , v a ) in the underexposed area of the auxiliary-viewpoint camera 5 to the corresponding three-dimensional space point P object (x, y, z) in the overexposure mask of the main-viewpoint camera 4 image:

[0049] s c [u′ a , v′ a , 1] T = K m T m [x, y, z, 1] T

[0050] where (u′ a , v′ a ) is the pixel coordinate calculated by re-projection, and s c is the scale factor. Considering that the calculated coordinate values contain decimal parts, the coordinates are rounded to integers using the rounding method:

[0051] [u″ a , v″ a T = Round([u′ a , v′ a T )

[0052] where (u″ a , v″ a ) is the obtained mapping point coordinate. By performing the above re-projection operation on all underexposed pixels in the auxiliary-viewpoint camera 5, the overall overexposure judgment mask M′ k can be obtained, which can be expressed as:

[0053]

[0054] The overall overexposure mask obtained at this time includes all the pixels that can be reconstructed in the primary and secondary viewing angles under the projection intensity (the value is 255);

[0055] Step 4: Based on the total overexposure mask, the mask is further de-duplicated. Considering that when the mask calculation is repeated with reduced intensity, a large number of points are repeatedly statistically calculated, thereby reducing the calculation efficiency, the pixel points that are not overexposed in the k-1th projection process are removed from the overexposure mask obtained in the kth time:

[0056]

[0057] Where M′ k-1 , M′ k are the total overexposure masks obtained in the k-1th and kth projection processes, respectively. is the total overexposure mask after deduplication in the k-th projection process. The points with a median value of 255 in the mask are the points that can be reconstructed in the k-th projection process. The corresponding 3D reconstruction points calculated in step 2 of these points are transmitted to the industrial computer;

[0058] Step 5: Determine whether there are any over-exposure points that have not been processed, that is, search M′ k Is there any pixel with a value of 0 in the image? If so, reduce the projection intensity and repeat steps 2 to 4. If not, it means that all overexposed points have been reconstructed and go to the next step.

[0059] Step 6: Merge the three-dimensional reconstruction results at each projection intensity stored in step 4 and output the final reconstruction result.

[0060] Finally, it should be noted that the above examples are only specific embodiments of the present invention. Obviously, the present invention is not limited to the above examples, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A three-dimensional measurement method for high-reflection surface structured light based on main and auxiliary dual-view multi-gray-level projection, characterized in that, it includes: Constructing a main and auxiliary dual-view structured light reconstruction system; Calibrating the structured light reconstruction system to obtain the internal and external parameter matrices of the main-view camera (4), the auxiliary-view camera (5), and the projector (3); Projecting a uniform gray image onto the surface of the object to be measured (6), and the main-view camera (4) and the auxiliary-view camera (5) simultaneously acquire an image. After binarizing the image, the overexposure masks of the main-view camera (4) and the auxiliary-view camera (5) are obtained; Projecting structured light sinusoidal phase-shift fringes onto the surface of the object to be measured (6), using the phase-shift method to solve the phase to obtain the absolute phase of the corresponding object surface under the main-view camera (4) and the auxiliary-view camera (5). By obtaining the internal parameter matrices of the main-view camera (4), the auxiliary-view camera (5), the internal parameter matrix of the projector (3), and the external parameter matrices of the main-view camera (4), the auxiliary-view camera (5), and the projector, calculate the spatial three-dimensional coordinates of the pixel points in the overexposed area of the overexposure mask of the auxiliary-view camera (5). Use the internal and external parameter matrices of the main-view camera (4) to reproject the three-dimensional space points corresponding to the pixel points in the non-overexposed area of the auxiliary camera to the overexposure mask of the main-view camera (4) image to obtain the overall overexposure judgment mask; Perform de-duplication processing on the overall overexposure judgment mask to obtain the three-dimensional reconstruction point cloud corresponding to the remaining points; Judge whether there are still overexposed points in the three-dimensional reconstruction points corresponding to the remaining points that have not been processed. If there are, reduce the projection intensity and repeat the above projection acquisition process and calculate the overexposure masks under the corresponding intensities. Reconstruct and store the results for the non-overexposed points. If not, it means that all overexposed points have been reconstructed; Merge the three-dimensional reconstruction point clouds under each stored projection intensity and output the final reconstruction result.

2. The three-dimensional measurement method for high-reflection surface structured light based on main and auxiliary dual-view multi-gray-level projection according to claim 1, characterized in that, the main and auxiliary dual-view structured light reconstruction system includes a main-view camera (4), an auxiliary-view camera (5), and a projector (3). The object to be measured (6) is placed under the lens of the projector (3) and remains stationary. The main-view camera (4) and the auxiliary-view camera (5) are placed on both sides of the projector (3). The distances between the main-view camera (4) and the auxiliary-view camera (5) and the projector (3) are in the range of 100 mm to 150 mm, and the included angles with the projector (3) are in the range of 10° to 30° respectively.

3. The three-dimensional measurement method for high-reflection surface structured light based on main and auxiliary dual-view multi-gray-level projection according to claim 1 or 2, characterized in that, the step of using the internal and external parameter matrices of the main-view camera (4) to reproject the three-dimensional space points corresponding to the pixel points in the non-overexposed area of the auxiliary-view camera (5) to the overexposure mask of the main-view camera (4) image to obtain the overall overexposure judgment mask is specifically: Using the internal and external parameter matrices of the main perspective camera (4) and Reproject the three-dimensional space point corresponding to a point in the non-overexposed area of the auxiliary perspective camera (5) onto the overexposure mask of the image of the main perspective camera (4): ​ ； wherein is the pixel coordinate calculated by reprojection, is the scale factor. Considering that the calculated coordinate value contains a decimal part, the rounding method is used to convert the coordinate into an integer: ； Among them are the obtained mapped point coordinates. By performing the above reprojection operation on the non-overexposed pixels in all the auxiliary view cameras (5), the overall overexposure judgment mask can be obtained , which can be expressed as: 。 4. The three-dimensional measurement method for high-reflection surface structured light based on main and auxiliary dual-view multi-gray-level projection according to claim 3, characterized in that, the de-duplication processing of the overall overexposure judgment mask is specifically: Remove the pixel points that were not overexposed during the (k - 1)-th projection process from the overexposure mask obtained in the k-th time: ； wherein and are the total overexposed masks obtained during the (k - 1)-th and k-th projection processes respectively, is the total overexposed mask after duplicate removal during the k-th projection process, and the points with a mask median value of 255 are the points that can be reconstructed during the k-th projection process.

Images