Line structured light three-dimensional reconstruction method and system suitable for high and low reflection surfaces

Abstract

The invention discloses a line structured light three-dimensional reconstruction method and system suitable for high and low reflection surfaces, relates to the technical field of three-dimensional reconstruction, and improves the three-dimensional reconstruction effect. Comprising the following steps: acquiring internal parameters and external parameters of a binocular system; obtaining a laser plane equation based on the calculated point cloud fitting plane of the plurality of light strips; obtaining detected object images shot by the left monocular camera and the right monocular camera, and detecting the light strip center point position of the detected object images; point clouds of the left camera and the right camera are reconstructed according to the internal reference of the monocular camera and the linear laser plane equation, and the point cloud of the right camera is aligned to the coordinate system of the left camera; selecting an optimal point according to the light strip width, the peak value and the nearest neighbor point distance; carrying out point cloud fine alignment; and aligning the optimal point in the reconstructed point cloud of the left camera and the optimal point in the reconstructed point cloud of the right camera to the point cloud of the left camera, and combining the optimal point and the point cloud as final point cloud for three-dimensional reconstruction. By means of the method, the exposure of the low-reflection surface is improved, overexposure of the high-reflection surface is restrained, and meanwhile the three-dimensional reconstruction effect is improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a line structured light three-dimensional reconstruction method and system suitable for high and low reflective surfaces. Background Art

[0002] Monocular and binocular 3D reconstruction methods based on structured light are characterized by high measurement accuracy, fast speed, and a wide range of adaptability, making them widely used measurement methods in the 3D reconstruction field. However, structured light 3D reconstruction methods also have difficulties, such as overexposure of highly reflective surfaces, resulting in distortion of the structured light image and a significant decrease in measurement accuracy, and underexposure of low-reflectivity surfaces, which results in ineffective structured light imaging and missing point clouds. This greatly reduces the practicality of structured light 3D reconstruction.

[0003] In recent years, many researchers have conducted extensive research on these difficulties and proposed many methods to deal with the overexposure problem of imaging high-reflectivity surfaces:

[0004] For example, Chinese patent CN115876124A, "Method and Apparatus for 3D Reconstruction of Highly Reflective Surfaces Based on a Polarized Structured Light Camera," proposes a method that utilizes a polarized camera to capture multiple images of a highly reflective surface at different polarization angles, using a projector equipped with a polarizer to project a pattern onto the surface. This method utilizes the characteristics of a polarized camera, which can produce four images of varying brightness in a single image, thereby reconstructing a four-point cloud. This method then fuses and complements the point clouds of high-reflectivity areas based on image brightness, addressing the difficulty of reconstructing high-reflectivity areas. However, this method requires the use of a polarized camera for imaging; otherwise, four images of varying brightness cannot be obtained simultaneously, leading to high costs.

[0005] Chinese patent CN113554575A, "A method for removing highlights from highly reflective surfaces based on polarization principles," adds a polarizer to the light source and camera, then rotates the polarizer to find the optimal polarization angle based on quantitative indicators. The polarizer's ability to suppress high-reflectivity exposure at the appropriate angle allows for three-dimensional reconstruction of high-reflectivity surfaces. However, the introduction of the polarizer reduces the brightness of low-reflectivity surfaces, making it difficult to capture structured light from low-reflectivity surfaces, resulting in missing point clouds.

[0006] Chinese patent CN111307065A, "Rail Profile Detection Method, System, and Apparatus Based on Polarization Spectroscopy," uses a polarization prism to split a light source into two polarization angles: S and P. The resulting images are then fused and complemented to address the issue of high-reflection overexposure. However, this method requires pixel-level alignment of the images. However, due to the different imaging optical paths of the two cameras (e.g., parallax, optical distortion, or installation position deviations), pixel-level alignment is difficult to achieve, resulting in ineffective image fusion and increased difficulty in 3D reconstruction. Summary of the Invention

[0007] Based on this, in order to solve the above technical problems, a line structured light 3D reconstruction method and system suitable for high and low reflective surfaces is provided to solve the problems of high cost, lack of point cloud and high difficulty of existing 3D reconstruction methods.

[0008] In a first aspect, a line structured light 3D reconstruction method applicable to high and low reflective surfaces is provided, the method comprising:

[0009] Calculating intrinsic and extrinsic parameters of a binocular system in a line structured light 3D reconstruction device; the intrinsic and extrinsic parameters of the binocular system are obtained by calibrating images of a calibration plate captured simultaneously by the left and right cameras of the binocular system; the line structured light 3D reconstruction device includes left and right monocular cameras, a line laser, and a polarizing film disposed in front of the right monocular camera;

[0010] Calculating the monocular camera intrinsic parameters and multiple light strip point clouds based on multiple calibration plate and light strip images obtained by using a monocular camera, and obtaining the laser plane equation based on the multiple light strip point clouds fitted with a plane; the calibration plate and light strip images are calibration plate images with light strips taken by the left and right monocular cameras;

[0011] Obtain images of the object under test taken by the left and right monocular cameras, and detect the center position of the light stripe of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively based on the center position of the light stripe, the internal parameters of the monocular camera, and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the expected effect, and then photographing the object under test with the left and right monocular cameras;

[0012] By traversing the image of the object captured by the monocular camera row by row, the light stripe width, peak value and nearest neighbor distance of each row are calculated, and the optimal point in the point cloud reconstructed by the left and right cameras is retained according to the light stripe width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the same row of the point cloud reconstructed by the right camera;

[0013] Obtaining a point pair with a nearest neighbor point and a nearest neighbor distance less than a preset value from the optimal point in the point cloud reconstructed by the left and right cameras, and calculating the optimal rotation and translation to align the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera;

[0014] The point clouds of the optimal point in the left camera reconstructed point cloud and the optimal point in the right camera reconstructed point cloud aligned to the point cloud of the left camera are merged together as the final point cloud for three-dimensional reconstruction.

[0015] In the above solution, optionally, the step of calculating the intrinsic parameters of the monocular camera based on the multiple calibration plates and light bar images acquired by the monocular camera includes:

[0016] Performing epipolar correction on multiple calibration plate and light bar images captured by a monocular camera according to an epipolar correction mapping matrix; the epipolar correction mapping matrix is ​​calculated based on the intrinsic and extrinsic parameters of the binocular system;

[0017] The monocular camera is calibrated using Zhang's calibration method based on the calibrated calibration plate and light bar image to obtain the monocular camera intrinsic parameters.

[0018] In the above solution, further optionally, calculating the point clouds of the plurality of light stripes includes:

[0019] Based on the calibrated calibration plate and light strip image, the monocular camera is calibrated using Zhang's calibration method to obtain the rotation and translation matrices Ri and Ti from the world coordinate system to the camera coordinate system corresponding to different postures of the calibration plate;

[0020] The plane equation of the plane to which the calibration plate belongs in different postures in the camera coordinate system is calculated according to the rotation and translation matrices Ri and Ti;

[0021] The center positions of the light stripes of the plurality of calibration plates and light strip images are detected, and the plurality of light stripe point clouds are calculated in combination with the corresponding calibration plate plane equations.

[0022] In the above solution, further optionally, the center position points of the light stripes of the plurality of calibration plates and light strip images are detected by using a Steger algorithm or a grayscale centroid method.

[0023] In the above solution, optionally, reconstructing the point clouds of the left and right cameras respectively according to the center position of the light strip, the intrinsic parameters of the monocular camera, and the line laser plane equation includes:

[0024] Convert the pixel coordinates of the center of the light strip to the camera coordinate system according to the intrinsic parameters of the monocular camera;

[0025] A straight line is determined based on the center point of the light strip in the camera coordinate system and the origin of the camera coordinate system, and the three-dimensional coordinates of the measured point are determined based on the straight line and the laser plane equation, thereby reconstructing the point clouds of the left and right cameras.

[0026] In the above scheme, optionally, the calculation of the light strip width of each row includes: performing Gaussian filtering on the image of the object to be measured, and counting the sum of the pixel value differences between all pixels within the neighborhood radius of the same row centered on the current pixel coordinate and the current pixel according to a preset neighborhood radius, and comparing with the preset value to determine whether the current pixel is in the light strip area. After the judgment is completed, the number of pixel clusters connected in the same row and in the light strip area is the number of light strips, and the number of pixels in each cluster is the width of each light strip.

[0027] In the above solution, optionally, the calculating the optimal rotation and translation further includes:

[0028] The point pairs having the nearest neighbor points and the nearest neighbor distances less than a preset value among the optimal points in the point clouds reconstructed by the left and right cameras are stored in the left target point set and the right point set to be aligned respectively;

[0029] Calculate the centroid of the left target point set and the right point set to be aligned;

[0030] A centroid operation is performed on the left target point set and the right point set to be aligned, a covariance matrix is ​​calculated based on the left target point set and the right point set to be aligned after the centroid operation is performed, an SVD decomposition is performed on the covariance matrix, an optimal rotation is calculated based on the SVD decomposition result, and an optimal translation is calculated based on the optimal rotation.

[0031] In a second aspect, a line structured light 3D reconstruction system suitable for high and low reflective surfaces is provided, characterized in that the system comprises:

[0032] The binocular system's intrinsic and extrinsic parameter acquisition module is used to calculate the binocular system's intrinsic and extrinsic parameters in a line structured light 3D reconstruction device. The binocular system's intrinsic and extrinsic parameters are obtained by calibrating images of a calibration plate captured simultaneously by the binocular system's left and right cameras. The line structured light 3D reconstruction device includes left and right monocular cameras, a line laser, and a polarizing film in front of the right monocular camera.

[0033] Laser plane equation calculation module: used to calculate the monocular camera internal parameters and multiple light strip point clouds based on the multiple calibration plate and light strip images captured by the monocular camera, and to obtain the laser plane equation based on the plane fitting of the multiple light strip point clouds; the calibration plate and light strip images are calibration plate images with light strips captured by the left and right monocular cameras;

[0034] Point cloud reconstruction module: used to obtain the images of the object under test taken by the left and right monocular cameras, detect the center point position of the light strip of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively based on the center position of the light strip, the internal parameters of the monocular camera and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the desired effect, and then taking pictures of the object under test with the left and right monocular cameras;

[0035] Point cloud fusion module: used to traverse the image of the object captured by the monocular camera row by row, calculate the light stripe width, peak value and nearest neighbor distance of each row, and select and retain the optimal point in the point cloud reconstructed by the left and right cameras based on the light stripe width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the same row of the point cloud reconstructed by the right camera;

[0036] Point cloud alignment module: used to obtain point pairs with nearest neighbor points and a nearest neighbor distance less than a preset value in the optimal points in the point clouds reconstructed by the left and right cameras, calculate the optimal rotation and translation, and align the optimal points in the point cloud reconstructed by the right camera to the point cloud of the left camera;

[0037] The final point cloud determination module is used to merge the point clouds of the point cloud reconstructed by the left camera and the point cloud reconstructed by the right camera, which are aligned to the point cloud of the left camera, as the final point cloud for three-dimensional reconstruction.

[0038] In a third aspect, a computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the line structured light three-dimensional reconstruction method applicable to high and low reflective surfaces described in the first aspect.

[0039] In a fourth aspect, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of a line structured light three-dimensional reconstruction method applicable to high and low reflective surfaces as described in the first aspect.

[0040] This application has at least the following beneficial effects:

[0041] This application increases the exposure of low-reflective surfaces by increasing the brightness of the light source, allowing a left-side monocular system without a polarizer to effectively image and reconstruct low-reflective surfaces. The problem of easy overexposure caused by increasing the brightness of the light source is solved by the right camera. The right-side monocular structured light system with a polarizer can suppress overexposure of high-reflective areas through the polarizer. By adjusting the angle of the polarizer, the degree of suppression of high-reflective surfaces can be flexibly controlled, ensuring that the details of the high-reflective areas can be clearly captured and reconstructed, achieving effective three-dimensional reconstruction of high-reflective surfaces. By adjusting the angle of the polarizer, the degree of high-reflective suppression can be adjusted to achieve three-dimensional reconstruction of high-reflective surfaces without affecting the imaging of low-reflective surfaces by the other camera, thereby achieving different imaging and fusion effects and more flexible applications. This application achieves a complementary effect of high-reflective and low-reflective surface point clouds and left and right blind spot point clouds by fusing left and right perspective point clouds, making the reconstructed point cloud more complete. The fusion and complementarity between high-reflective point clouds, low-reflective point clouds, and point clouds imaged at different angles can reduce point clouds lost due to occlusion, high-reflective surfaces, and low-reflective surfaces, improving the integrity and robustness of the three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic flow chart of a line structured light 3D reconstruction method applicable to high and low reflective surfaces provided in one embodiment of the present application;

[0043] Figure 2 A schematic diagram of a specific process of a line structured light 3D reconstruction method applicable to high and low reflective surfaces is provided for one embodiment of the present application;

[0044] Figure 3 This is a structural diagram of a three-dimensional reconstruction system according to an embodiment of the present application;

[0045] Figure 4 The existing technology causes the left camera to be overexposed due to the high reflection when the structured light hits the stainless steel muzzle of the heat gun.

[0046] Figure 5 for Figure 4 At the same time, the imaging is performed by the right camera of the line structured light 3D reconstruction device of the present application after the high reflection is suppressed by the polarizer;

[0047] Figure 6 For this application, the hot air gun point cloud for monocular 3D reconstruction was performed using the left camera;

[0048] Figure 7 This application uses the left camera for monocular 3D reconstruction using the hot air gun point cloud triangulation surface;

[0049] Figure 8 For this application, a hot air gun point cloud with a right camera with a polarizer was used for monocular 3D reconstruction;

[0050] Figure 9 For this application, the surface obtained by triangulating the hot air gun point cloud using the right camera with a polarizer for monocular 3D reconstruction;

[0051] Figure 10 To use the present application's line structured light 3D reconstruction method for high and low reflective surfaces to fuse hot air gun point clouds;

[0052] Figure 11 The surface obtained by triangulating the hot air gun point cloud by using a line structured light 3D reconstruction method suitable for high and low reflective surfaces in this application;

[0053] Figure 12 This is because the left camera is overexposed due to high reflectivity when the structured light is used to illuminate the object being measured.

[0054] Figure 13 for Figure 12At the same time as imaging, the imaging of the right camera in the line structured light 3D reconstruction device of the present application is performed after the polarizer suppresses high reflection;

[0055] Figure 14 This is the image of the left camera when the structured light of this application hits the low-reflective surface of the object being measured;

[0056] Figure 15 for Figure 14 While imaging, the underexposure phenomenon caused by the reduction in imaging brightness of the right camera after passing through the polarizer in the line structured light 3D reconstruction device of the present application is utilized;

[0057] Figure 16 This application uses the left camera for monocular 3D reconstruction of the camera point cloud;

[0058] Figure 17 This is the surface obtained by triangulating the camera point cloud for monocular 3D reconstruction using the left camera in this application;

[0059] Figure 18 For this application, the camera point cloud of the monocular 3D reconstruction is performed using the right camera with a polarizer;

[0060] Figure 19 The surface obtained by triangulating the camera point cloud for monocular 3D reconstruction using the right camera with a polarizer for this application;

[0061] Figure 20 The camera point cloud fused for this application;

[0062] Figure 21 The surface obtained by triangulating the camera point cloud fused for this application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] In one embodiment, Figure 1 and Figure 2 As shown, a line structured light 3D reconstruction method applicable to high and low reflective surfaces is provided, the method comprising:

[0065] Step S1: Calculate the intrinsic and extrinsic parameters of the binocular system in the line structured light 3D reconstruction device; the intrinsic and extrinsic parameters of the binocular system are obtained by calibration based on the calibration plate images simultaneously captured by the left and right cameras of the binocular system; the line structured light 3D reconstruction device includes left and right monocular cameras, a line laser, and a polarizing film is set in front of the right monocular camera.

[0066] In step S1, the left and right cameras are regarded as a binocular system and the same calibration plate image is taken at the same time. The calibration plate posture is changed and multiple sets of images are taken multiple times. The Zhang calibration method is used to calibrate the binocular system to obtain the internal and external parameters. The internal parameters are the focal length f, the distortion parameter , pixel coordinate origin offset The external parameters include the rotation matrix Rc and translation matrix Tc from the right camera to the left camera. At the same time, the epipolar correction mapping matrix is ​​calculated based on the internal and external parameters for subsequent epipolar correction of the image.

[0067] Step S2: Calculate the monocular camera intrinsic parameters and multiple light strip point clouds based on the multiple calibration plate and light strip images obtained using the monocular camera, and obtain the laser plane equation based on the plane fitting of the multiple light strip point clouds; the calibration plate and light strip images are calibration plate images with light strips taken by the left and right monocular cameras.

[0068] like Figure 3 This is the structural diagram of the 3D reconstruction system. The laser light source is turned on to illuminate the calibration plate so that the calibration plate and the light bar appear in the field of view of the monocular camera at the same time. The calibration plate's posture and position are continuously adjusted, and the monocular camera is used to capture multiple images of the calibration plate and the light bar and perform epipolar correction. Based on the corrected images, the monocular camera is calibrated using Zhang's calibration to obtain the monocular camera's intrinsic parameters and the rotation and translation matrix from the world coordinate system to the camera coordinate system corresponding to different postures of the calibration plate. and ,according to and Calculate the plane equation of the calibration plate in different postures in the camera coordinate system. The formula is as follows:

[0069]

[0070] in, is the plane equation of the calibration plate in the camera coordinate system The coefficient of Represents the plane equation of the calibration plate in the world coordinate system determined by the calibration plate The coefficient of and Represents the rotation and translation matrix from the world coordinate system to the camera coordinate system under the i-th calibration plate pose.

[0071] Detect the center positions of the light stripes in multiple calibrated images and calculate the point clouds of the light stripes based on the corresponding calibration plate plane equations. Common detection algorithms such as the Steger algorithm or the grayscale centroid method can be used to detect the center positions of the light stripes.

[0072] Collect multiple light strip point clouds and fit the plane to get the line laser light plane equation. The fitting method can use the commonly used RANSAC fitting algorithm. Use the same method to calibrate the light plane equations of the left and right cameras in their respective coordinate systems. .

[0073] Step S3: Obtain the images of the object under test taken by the left and right monocular cameras, and detect the center point position of the light strip of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively according to the center position of the light strip, the internal parameters of the monocular camera and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the expected effect, and then taking the images of the object under test by the left and right monocular cameras.

[0074] Specifically, in step S3, an image of the object is captured, the coordinates of the light strip center are detected, and monocular 3D reconstruction is performed. Structured light is emitted to illuminate the object, and the left and right cameras simultaneously capture images of the object. The captured images are then used to detect the center position of the light strip. Monocular 3D reconstruction is performed based on the principle of triangulation based on the light strip center position, camera intrinsic parameters, and the line laser plane equation.

[0075] The monocular 3D reconstruction process is as follows: let the pixel coordinates of the center point of a certain light strip be , the camera focal length is f, and the pixel size is 、 , the offset from the image coordinate system to the pixel coordinate system is , the pixel coordinates (in pixels) can be converted to the camera coordinate system (in mm). The conversion formula is as follows:

[0076]

[0077] ( , , ) and the origin of the camera coordinate system to determine a straight line l, which is aligned with the laser plane The intersection point is the coordinate P of the measured point in the camera coordinate system ( , , ). The formula for calculating the coordinates of point P is as follows:

[0078] in,

[0079] The left and right monocular systems are reconstructed to obtain the left and right camera reconstruction point clouds respectively. The right camera reconstruction point cloud is converted into the external parameters obtained by binocular calibration in step 2. and Transform to the left camera coordinate system. The formula is as follows:

[0080]

[0081] Step S4: By traversing the image of the object taken by the monocular camera row by row, the light bar width, peak value and nearest neighbor distance of each row are calculated, and the optimal point in the point cloud reconstructed by the left and right cameras is retained according to the light bar width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the same row of the point cloud reconstructed by the right camera.

[0082] Specifically, the image is traversed row by row, and the point cloud fusion and complementation are performed by combining the light strip width, peak value, and the nearest neighbor distance reconstructed in the same row of the image. The light strip width is the width of the detected light strip area, and the peak value is the pixel value of the nearest neighbor pixel of the detected light strip center position point. The nearest neighbor distance reconstructed in the same row is: there may be multiple light strip center position points on the same pixel row in the image taken by the camera. These center position points can be reconstructed to obtain multiple three-dimensional points to form a point set. The point sets of the pixel rows with the same row index in the left and right camera images are represented as the left point set and the right point set. The points in the left point set find the nearest neighbor points in the right point set and the distance calculated is represented as the nearest neighbor distance. Based on these indicators, point cloud fusion is performed according to the strategy shown in Table 1:

[0083] Table 1

[0084]

[0085] It should be noted that when performing monocular structured light 3D reconstruction using the left and right cameras, the left and right images undergo epipolar correction, meaning that the corrected images are row-aligned. The reconstructed points contain information about peak values, line widths, and row indexes. The strategy described above traverses the points row by row and selects the points to retain between each point in the left point set and its nearest neighbor. When the left point set contains no points but the right point set contains some, the points in the right point set are retained. In the table, T indicates true, and F indicates false. Thresholds V1 and V2 are related to the actual light bar width. When adjusting the light source focal length to change the light bar width, the thresholds should be adjusted accordingly. This should be determined based on actual results. Threshold M is related to the measured object distance; a larger threshold should be set for a larger object distance, depending on actual results. The right reconstructed point cloud retained by this strategy is denoted as CloudR, and the left reconstructed point cloud retained by this strategy is denoted as CloudL.

[0086] That is, first, for each point in the left camera point cloud, find the nearest neighbor points in the right camera point cloud that belong to the same row and combine them into a pair of points. For each pair of points, select an optimal point according to the following rules:

[0087] For each point pair, the point belonging to the left camera point cloud is recorded as the left point, and the point belonging to the right camera point cloud is recorded as the right point.

[0088] (1) When the peak value, light bar width and nearest neighbor of the left and right points are all less than their respective thresholds, the left point is selected as the optimal point of the point pair.

[0089] (2) The peak value of the left point is less than the threshold V1, the light bar width is greater than the threshold V2, and the nearest neighbor distance is less than the threshold M. The peak value of the right point is less than the threshold V1, and the light bar width is less than the threshold V2. At this time, the right point is selected as the optimal point of the point pair.

[0090] (3) The peak value of the left point is less than the threshold V1, the light bar width is less than the threshold V2, and the distance to the nearest neighbor is greater than the threshold M. At this time, as long as the right point exists, the right point is selected as the optimal point of the point pair.

[0091] (4) The peak value of the left point is greater than the threshold V1, the distance to the nearest neighbor is less than the threshold M, and the peak value of the right point is less than the threshold V1. At this time, the right point is selected as the optimal point of the point pair.

[0092] (5) The peak value of the left point is greater than the threshold V1, the distance to the nearest neighbor is less than the threshold M, and the peak value of the right point is greater than the threshold V1. At this time, the left point is selected as the optimal point of the point pair.

[0093] (6) The peak value of the left point is less than the threshold V1, the distance to the nearest neighbor is greater than the threshold M, the peak value of the right point is less than the threshold V1, and the light bar width is less than the threshold V2. At this time, the right point is selected as the optimal point of the point pair.

[0094] (7) If there is a point on the left side and no point on the right side in the same row, the point on the left side is the optimal point.

[0095] (8) If there is a point on the right side and no point on the left side in the same row, then the point on the right side is the optimal point.

[0096] Light streak area detection method: First, perform Gaussian filtering on the image, then set the neighborhood radius to count the sum of the pixel value differences between all pixels in the same row of the neighborhood radius centered on the current pixel coordinate and the current pixel, and finally set a threshold to determine whether the pixel is a light streak area. The calculation formula is as follows:

[0097]

[0098] in Indicates the pixel value at row , column col . The neighborhood radius ξ (in pixels) depends on the actual width of the laser line stripe (mm). The wider the stripe, the larger the neighborhood radius. The neighborhood radius and threshold are determined based on actual test results.

[0099] Step S5: Obtain a point pair in the optimal point in the point cloud reconstructed by the left and right cameras, which has a nearest neighbor point and a nearest neighbor distance less than a preset value, and calculate the optimal rotation and translation to align the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera.

[0100] Specifically, point cloud is precisely aligned. In the process of collecting step S4, there are point pairs with nearest neighbor points and nearest neighbor distance less than M, which are stored in the left target point set and the right point set to be aligned. The left target point set is , the right set of points to be aligned is recorded as , and The points of are one-to-one corresponding. Align to , we need to calculate the optimal rotation and optimal translation , first calculate and The center of mass , :

[0101]

[0102] Then perform the centroid removal operation:

[0103]

[0104] Calculate the covariance matrix H:

[0105]

[0106] The SVD decomposition of H is:

[0107]

[0108] According to the SVD decomposition results, the optimal rotation can be obtained:

[0109]

[0110] Based on the optimal rotation, the optimal translation can be calculated:

[0111]

[0112] Finally, the right point set CloudR retained in step S4 is transformed and aligned to the left point cloud according to the optimal rotation and optimal translation.

[0113]

[0114] Step S6: aligning the optimal point in the point cloud reconstructed by the left camera and the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera and merging them together as the final point cloud for 3D reconstruction.

[0115] The retained point set will be selected as the final output point cloud.

[0116] In the above-mentioned line structured light 3D reconstruction method applicable to high- and low-reflective surfaces, by increasing the brightness of the light source to increase the exposure of the low-reflective surface, the left monocular system without the addition of a polarizer can effectively image and 3D reconstruct the low-reflective surface. The problem of easy overexposure caused by increasing the brightness of the light source is solved by the right camera. The right monocular structured light system with the addition of a polarizer can suppress the overexposure of high-reflective light through the polarizer, thereby effectively reconstructing the high-reflective surface in 3D. By adjusting the angle of the polarizer, the degree of high-reflective suppression can be adjusted to achieve 3D reconstruction of high-reflective surfaces without affecting the imaging of the low-reflective surface by the other camera, thereby achieving different imaging and fusion effects and more flexible application. This application achieves the complementary effect of high-reflective and low-reflective surface point clouds and the left and right blind spot point clouds by fusing the left and right viewpoint point clouds, making the reconstructed point cloud more complete. The fusion and complementation between high-reflective point clouds, low-reflective point clouds, and point clouds imaged at different angles can reduce point clouds lost due to occlusion, high-reflective surfaces, and low-reflective surfaces, thereby improving the completeness and robustness of the 3D reconstruction.

[0117] Specific as Figure 4-Figure 21 This application discloses a line structured light 3D reconstruction method for high and low reflective surfaces, which is used to image and point cloud changes of the object under imaging by left and right monocular cameras.

[0118] In one embodiment, a line structured light 3D reconstruction system suitable for high and low reflective surfaces is provided, the system comprising:

[0119] The module for acquiring the intrinsic and extrinsic parameters of the binocular system is used to obtain the intrinsic and extrinsic parameters of the binocular system by performing calibration calculations on the calibration plate images taken simultaneously by the left and right cameras of the binocular system.

[0120] Laser plane equation calculation module: used to calculate the monocular camera internal parameters and multiple light strip point clouds based on the multiple calibration plate and light strip images captured by the monocular camera, and to obtain the laser plane equation based on the plane fitting of the multiple light strip point clouds; the calibration plate and light strip images are calibration plate images with light strips captured by the left and right monocular cameras;

[0121] Point cloud reconstruction module: used to obtain the images of the object under test taken by the left and right monocular cameras, detect the center point position of the light strip of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively based on the center position of the light strip, the internal parameters of the monocular camera and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the desired effect, and then taking pictures of the object under test with the left and right monocular cameras;

[0122] Point cloud fusion module: used to traverse the image of the object captured by the monocular camera row by row, calculate the light stripe width, peak value and nearest neighbor distance of each row, and select the optimal point in the point cloud reconstructed by the left and right cameras based on the light stripe width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the point cloud reconstructed by the right camera;

[0123] Point cloud alignment module: used to obtain point pairs with nearest neighbor points and a nearest neighbor distance less than a preset value in the optimal points in the point clouds reconstructed by the left and right cameras, calculate the optimal rotation and translation, and align the optimal points in the point cloud reconstructed by the right camera to the point cloud of the left camera;

[0124] The final point cloud determination module is used to align the optimal point in the point cloud reconstructed by the left camera and the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera and merge them together as the final point cloud for three-dimensional reconstruction.

[0125] For the specific definition of a line structured light 3D reconstruction system suitable for high and low reflective surfaces, please refer to the definition of a line structured light 3D reconstruction method suitable for high and low reflective surfaces above, which will not be repeated here. The various modules in the above-mentioned line structured light 3D reconstruction system suitable for high and low reflective surfaces can be implemented in whole or in part through software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0126] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above-mentioned line structured light three-dimensional reconstruction method applicable to high and low reflective surfaces is implemented.

[0127] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored, which involves all or part of the processes in the above-mentioned embodiment method.

[0128] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0129] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A line structured light 3D reconstruction method suitable for high and low reflective surfaces, characterized in that: The method comprises: Calculating intrinsic and extrinsic parameters of a binocular system in a line structured light 3D reconstruction device; the intrinsic and extrinsic parameters of the binocular system are obtained by calibrating images of a calibration plate captured simultaneously by the left and right cameras of the binocular system; the line structured light 3D reconstruction device includes left and right monocular cameras, a line laser, and a polarizing film disposed in front of the right monocular camera; Calculating the monocular camera intrinsic parameters and multiple light strip point clouds based on multiple calibration plate and light strip images obtained by using a monocular camera, and obtaining the laser plane equation based on the multiple light strip point clouds fitted with a plane; the calibration plate and light strip images are calibration plate images with light strips taken by the left and right monocular cameras; Obtain images of the object under test taken by the left and right monocular cameras, and detect the center position of the light stripe of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively based on the center position of the light stripe, the internal parameters of the monocular camera, and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the expected effect, and then photographing the object under test with the left and right monocular cameras; By traversing the image of the object captured by the monocular camera row by row, the light stripe width, peak value and nearest neighbor distance of each row are calculated, and the optimal point in the point cloud reconstructed by the left and right cameras is retained according to the light stripe width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the same row of the point cloud reconstructed by the right camera; Obtaining a point pair with a nearest neighbor point and a nearest neighbor distance less than a preset value from the optimal point in the point cloud reconstructed by the left and right cameras, and calculating the optimal rotation and translation to align the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera; The point clouds of the optimal point in the left camera reconstructed point cloud and the optimal point in the right camera reconstructed point cloud aligned to the point cloud of the left camera are merged together as the final point cloud for three-dimensional reconstruction.

2. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 1, characterized in that: The method of calculating the intrinsic parameters of the monocular camera according to the multiple calibration plates and light bar images obtained by taking the monocular camera comprises: Performing epipolar correction on multiple calibration plate and light bar images captured by a monocular camera according to an epipolar correction mapping matrix; the epipolar correction mapping matrix is ​​calculated based on the intrinsic and extrinsic parameters of the binocular system; The monocular camera is calibrated using Zhang's calibration method based on the calibrated calibration plate and light bar image to obtain the monocular camera intrinsic parameters.

3. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 2, characterized in that: The calculating of the plurality of light streak point clouds comprises: Based on the calibrated calibration plate and light strip image, the monocular camera is calibrated using Zhang's calibration method to obtain the rotation and translation matrices Ri and Ti from the world coordinate system to the camera coordinate system corresponding to different postures of the calibration plate; The plane equation of the plane to which the calibration plate belongs in different postures in the camera coordinate system is calculated according to the rotation and translation matrices Ri and Ti; The center positions of the light stripes of the plurality of calibration plates and light strip images are detected, and the plurality of light stripe point clouds are calculated in combination with the corresponding calibration plate plane equations.

4. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 3, characterized in that: The center position of the light stripe of the plurality of calibration plates and light stripe images is detected by using the Steger algorithm or the grayscale centroid method.

5. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 1, characterized in that: The step of reconstructing the point clouds of the left and right cameras respectively according to the center position of the light strip, the internal parameters of the monocular camera, and the line laser plane equation includes: Convert the pixel coordinates of the center of the light strip to the camera coordinate system according to the intrinsic parameters of the monocular camera; A straight line is determined based on the center point of the light strip in the camera coordinate system and the origin of the camera coordinate system, and the three-dimensional coordinates of the measured point are determined based on the straight line and the laser plane equation, thereby reconstructing the point clouds of the left and right cameras.

6. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 1, characterized in that: The calculation of the light strip width of each row includes: performing Gaussian filtering on the image of the object to be measured, counting the sum of the pixel value differences between all pixels within the neighborhood radius of the same row centered on the current pixel coordinate and the current pixel according to a preset neighborhood radius, and comparing with the preset value to determine whether the current pixel is in the light strip area. After the determination is completed, the number of pixel clusters connected in the same row and in the light strip area is the number of light strips, and the number of pixels in each cluster is the width of each light strip.

7. The line structured light 3D reconstruction method applicable to high and low reflective surfaces according to claim 1, characterized in that: The calculating of the optimal rotation and translation further comprises: The point pairs having the nearest neighbor points and the nearest neighbor distances less than a preset value among the optimal points in the point clouds reconstructed by the left and right cameras are stored in the left target point set and the right point set to be aligned respectively; Calculate the centroid of the left target point set and the right point set to be aligned; A centroid operation is performed on the left target point set and the right point set to be aligned, a covariance matrix is ​​calculated based on the left target point set and the right point set to be aligned after the centroid operation is performed, an SVD decomposition is performed on the covariance matrix, an optimal rotation is calculated based on the SVD decomposition result, and an optimal translation is calculated based on the optimal rotation.

8. A line structured light 3D reconstruction system suitable for high and low reflective surfaces, characterized in that: The system comprises: The module for acquiring the intrinsic and extrinsic parameters of the binocular system is used to obtain the intrinsic and extrinsic parameters of the binocular system by performing calibration calculations on the calibration plate images taken simultaneously by the left and right cameras of the binocular system. Laser plane equation calculation module: used to calculate the monocular camera internal parameters and multiple light strip point clouds based on the multiple calibration plate and light strip images captured by the monocular camera, and to obtain the laser plane equation based on the plane fitting of the multiple light strip point clouds; the calibration plate and light strip images are calibration plate images with light strips captured by the left and right monocular cameras; Point cloud reconstruction module: used to obtain the images of the object under test taken by the left and right monocular cameras, detect the center point position of the light strip of the image of the object under test; reconstruct the point clouds of the left and right cameras respectively based on the center position of the light strip, the internal parameters of the monocular camera and the line laser plane equation, and align the right camera point cloud to the left camera coordinate system using the external parameters of the binocular system; the images of the object under test taken by the left and right monocular cameras are obtained by adjusting the angle of the polarizer in the system until the high reflection suppression achieves the desired effect, and then taking pictures of the object under test with the left and right monocular cameras; Point cloud fusion module: used to traverse the image of the object captured by the monocular camera row by row, calculate the light stripe width, peak value and nearest neighbor distance of each row, and select the optimal point in the point cloud reconstructed by the left and right cameras based on the light stripe width, peak value and nearest neighbor distance. The nearest neighbor distance is the distance between a point in the point cloud reconstructed by the left camera and its nearest neighbor in the point cloud reconstructed by the right camera; Point cloud alignment module: used to obtain point pairs with nearest neighbor points and a nearest neighbor distance less than a preset value in the optimal points in the point clouds reconstructed by the left and right cameras, calculate the optimal rotation and translation, and align the optimal points in the point cloud reconstructed by the right camera to the point cloud of the left camera; The final point cloud determination module is used to align the optimal point in the point cloud reconstructed by the left camera and the optimal point in the point cloud reconstructed by the right camera to the point cloud of the left camera and merge them together as the final point cloud for three-dimensional reconstruction.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images