A method and apparatus for fusion of monocular and binocular structured light 3D data

Abstract

This invention discloses a method and apparatus for fusion of monocular and binocular structured light 3D data. The method includes: S1, individually calibrating a binocular structured light 3D reconstruction system, a left monocular structured light 3D reconstruction system, and a right monocular structured light 3D reconstruction system; S2, jointly calibrating the individually calibrated binocular, left monocular, and right monocular structured light 3D reconstruction systems; S3, dividing the view area into region B and region M based on the views captured by the left and right cameras; and performing 3D reconstruction of the divided regions based on the binocular, left monocular, and right monocular structured light 3D reconstruction systems to obtain 3D point cloud data; S4, performing rigid body transformation on the 3D point cloud data and merging them to obtain fused monocular and binocular 3D point cloud data. This invention solves the problems of missing and hole-like points in the binocular structured light 3D reconstruction data caused by the difference in the field of view of the left and right cameras.

Description

Technical Field

[0001] This invention belongs to the field of machine vision measurement technology, specifically relating to a method and apparatus for fusion of monocular and binocular structured light three-dimensional data. Background Technology

[0002] Binocular structured light 3D measurement is based on the theory of binocular vision measurement, eliminating the need for projector calibration. The added structured light also avoids the problem of difficulty in matching areas with weak or repetitive textures. Compared to traditional binocular measurement, it offers higher accuracy and stronger resistance to interference from other environmental factors than monocular structured light measurement. Its applications are more widespread. However, binocular measurement systems rely on matching "corresponding point pairs" between two cameras, calculating 3D coordinates based on the geometric relationships obtained from calibration. Therefore, differences in the fields of view of the left and right cameras can lead to situations where occlusions, shadows, or other abnormal areas in a common field of view cannot find "corresponding point pairs" in the other field of view, and non-common areas cannot be matched and reconstructed, resulting in missing 3D point cloud data and holes.

[0003] Prior art patent CN110044301A discloses a 3D point cloud computing method based on monocular and binocular hybrid measurement. It calibrates the projector using an "inverse camera" method by projecting horizontal and vertical stripes, establishing a matching relationship between corresponding dots on a calibration plate between the image planes of the left and right cameras and the projection plane of the projector. However, the phase error introduced by the projector's gamma distortion in this calibration method leads to insufficient accuracy in the generated point cloud. Prior art patent CN107421465A discloses a 3D measurement method combining monocular and binocular vision systems. In its system calibration, it equates the projector to a camera, resulting in insufficient accuracy in monocular point cloud reconstruction. Prior art patent CN111649694A discloses a binocular measurement missing point cloud interpolation method based on implicit phase-parallax mapping. It calculates missing unknown point clouds based on the mapping relationship between pixels. However, the two mapping relationships are not corrected, resulting in low accuracy in the point cloud interpolation. The prior art patent application publication number CN110617781A discloses a binocular structured light imaging system based on stripe projection, which fuses the point clouds reconstructed by two monocular structured light systems. The monocular system reconstructs the areas not covered by the binocular system, which increases the measurement time and also causes data overlap in these areas.

[0004] In summary, there is a need to propose a method and apparatus for 3D data fusion of monocular and binocular structured light that can improve the accuracy of reconstructed point clouds and point cloud fusion, accurately determine the fusion region required by monocular structured light, and avoid point cloud data overlap. Summary of the Invention

[0005] This invention aims to solve the problem of missing point cloud data and holes caused by the inability to properly reconstruct common or non-common areas in binocular structured light due to the limited field of view in the prior art. At the same time, it can ensure the accuracy of reconstructed point cloud and point cloud fusion, and avoid point cloud data overlap. A method and device for binocular and monocular structured light three-dimensional data fusion are proposed.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for fusion of monocular and binocular structured light 3D data includes the following steps:

[0008] S1. Perform individual calibrations on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system.

[0009] S2. Perform joint calibration on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system, which have been individually calibrated.

[0010] S3. Based on the views captured by the left and right cameras, the view area is divided into region B and region M; the divided regions are reconstructed in three dimensions based on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system to obtain 3D point cloud data;

[0011] S4. Perform rigid body transformation on the three-dimensional point cloud data and merge them to obtain fused monocular and binocular three-dimensional point cloud data.

[0012] Preferably, S2 includes:

[0013] S21. Perform joint calibration on the binocular structured light 3D reconstruction system and the left monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TL;

[0014] S22. Perform joint calibration on the binocular structured light 3D reconstruction system and the right monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TR.

[0015] Preferably, S21 includes:

[0016] Based on the binocular structured light 3D reconstruction system, the corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud dataset corner_B;

[0017] Based on the left monocular structured light 3D reconstruction system, the corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud corner_ML;

[0018] Based on the three-dimensional point cloud dataset corner_B and the three-dimensional point cloud corner_ML, the rigid body transformation matrix TL1 is obtained;

[0019] The target object is reconstructed based on the binocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_B of the target object located in region B is obtained.

[0020] The target object is reconstructed based on the left monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_ML of the target object located in region B is obtained.

[0021] The rigid body transformation matrix TL1 is used to perform matrix transformation on the three-dimensional point cloud data pointcloud_ML to obtain the three-dimensional point cloud data pointcloud_T1_ML;

[0022] The three-dimensional point cloud data pointcloud_B and the three-dimensional point cloud data pointcloud_T1_ML are finely registered to obtain the rigid body transformation matrix TL2.

[0023] Multiply the rigid body transformation matrix TL1 and the rigid body transformation matrix TL2 to obtain the rigid body transformation matrix TL.

[0024] Preferably, S22 includes:

[0025] Based on the right monocular structured light 3D reconstruction system, the corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud corner_MR;

[0026] Based on the three-dimensional point cloud dataset corner_B and the three-dimensional point cloud corner_MR, the rigid body transformation matrix TR1 is obtained;

[0027] The target object is reconstructed based on the binocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_B of the target object located in region B is obtained.

[0028] The target object is reconstructed based on the right monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_MR of the target object located in region B is obtained.

[0029] The rigid body transformation matrix TR1 is used to perform matrix transformation on the three-dimensional point cloud data pointcloud_MR to obtain the three-dimensional point cloud data pointcloud_T1_MR;

[0030] The three-dimensional point cloud data pointcloud_B and the three-dimensional point cloud data pointcloud_T1_MR are finely registered to obtain the rigid body transformation matrix TR2.

[0031] Multiplying the rigid body transformation matrix TR1 and the rigid body transformation matrix TR2 yields the rigid body transformation matrix TR.

[0032] Preferably, the method for dividing the region includes:

[0033] Based on the left monocular structured light 3D reconstruction system, the target object is dephased to obtain the left absolute phase map Image_phaL;

[0034] Based on the right monocular structured light 3D reconstruction system, the target object is dephased to obtain the right absolute phase map Image_phaR;

[0035] Based on the binocular structured light 3D reconstruction system, mapping relationships MapL and MapR are obtained. Based on the mapping relationships MapL and MapR, the left absolute phase image Image_phaL and the right absolute phase image Image_phaR are respectively subjected to limit correction to obtain the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR.

[0036] Phase matching is performed on the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR. After phase matching, corresponding point pairs are obtained. The region of the corresponding point pairs in the corrected left absolute phase image Image_C_phaL is designated as the common normal region BC_L, and the remaining region is designated as the common abnormal region and the non-common region MC_L.

[0037] The corresponding point pairs are designated as the common normal region BC_R in the corrected right absolute phase map Image_C_phaR, and the remaining region is designated as the common abnormal region and the non-common region MC_R.

[0038] Based on the mapping relationship MapL, the common anomaly region and the non-common region MC_L are inversely mapped to obtain the region M of the left monocular structured light 3D reconstruction system view area. L ;

[0039] Based on the mapping relationship MapR, the common anomaly region and the non-common region MC_R are inversely mapped to obtain the region M of the right monocular structured light 3D reconstruction system view area. R ;

[0040] The region B is the sum of the common normal region BC_L and the common normal region BC_R;

[0041] The region M is region M L and region M R The sum of.

[0042] Preferably, S3 includes:

[0043] The coordinates of the corresponding point pairs in region B are used to perform three-dimensional reconstruction with the parameters obtained by the calibration of the binocular structured light three-dimensional reconstruction system to obtain three-dimensional point cloud data cloud_B.

[0044] Based on region M L All point coordinates and the left absolute phase map Image_phaL are used to reconstruct the 3D point cloud data cloud_ML based on the absolute phase corresponding to the coordinates; based on the region M R Based on the coordinates and the absolute phase of the right absolute phase map Image_phaR, a 3D reconstruction is performed on the corresponding absolute phase to obtain the 3D point cloud data cloud_MR.

[0045] Preferably, the TransformPointCloud function is used to perform rigid body transformation on the three-dimensional point cloud data cloud_ML and the three-dimensional point cloud data cloud_MR based on the rigid body transformation matrix TL and the rigid body transformation matrix TR, respectively, to obtain three-dimensional point cloud data cloud_T_ML and three-dimensional point cloud data cloud_T_MR.

[0046] The three-dimensional point cloud data cloud_B, cloud_T_ML, and cloud_T_MR are merged to obtain the fused monocular and binocular three-dimensional point cloud data.

[0047] The present invention also provides a binocular and monocular structured light three-dimensional data fusion device, including a binocular structured light three-dimensional reconstruction system, a left monocular structured light three-dimensional reconstruction system, a right monocular structured light three-dimensional reconstruction system, and a calibration system;

[0048] The binocular structured light 3D reconstruction system includes: a left camera, a projector, and a right camera;

[0049] The left monocular structured light 3D reconstruction system includes: a left camera and a projector;

[0050] The right monocular structured light 3D reconstruction system includes: a right camera and a projector;

[0051] The calibration system includes a checkerboard calibration board and a target object.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] This invention discloses a method and apparatus for fusion of monocular and binocular structured light 3D data. It utilizes calibration methods to calibrate a binocular structured light 3D reconstruction system, a left monocular structured light 3D reconstruction system, and a right monocular structured light 3D reconstruction system. A checkerboard calibration board and an object with rich 3D information are used for joint calibration of the binocular and left monocular structured light 3D reconstruction systems, and also for joint calibration of the binocular and right monocular structured light 3D reconstruction systems. In the left and right camera views, common normal areas, common abnormal areas, and non-common areas are identified. The binocular structured light system is used to perform 3D reconstruction of the common normal areas. The left and right monocular structured light systems perform 3D reconstruction of common and non-common anomaly areas. The 3D point cloud data obtained from the binocular structured light system and the left and right monocular structured light systems are merged to complete the fusion of monocular and binocular 3D data. This invention uses a checkerboard calibration board and an object with rich 3D information to jointly calibrate the monocular and binocular systems. The calibration results are accurate. The calibration error between the systems is solved by the ICP algorithm to obtain the optimal rigid body transformation matrix. In this invention, the point cloud reconstruction accuracy of the monocular and binocular structured light systems does not have the final impact on the solution of the rigid body transformation matrix between the monocular and binocular systems, nor is it limited by the joint calibration between the monocular and binocular systems. This invention directly uses the phase matching results, combined with the epipolar correction mapping relationship, to divide the left and right views into common normal regions, common abnormal regions, and non-common regions. Then, the binocular and monocular systems perform 3D reconstruction on different regions respectively, avoiding repeated 3D reconstruction of common normal regions and ensuring that all regions in the left and right camera fields of view can be reconstructed. Finally, the obtained point cloud data are fused, effectively solving the problems of missing and hole-like issues in binocular structured light 3D reconstruction point cloud data caused by differences in the left and right camera fields of view. It also ensures the accuracy of the 3D reconstructed point cloud, has low experimental requirements, and the implementation scheme is simple and clear, making it highly practical. Attached Figure Description

[0054] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of the overall structure of the device used in this invention;

[0056] Figure 2 This is a schematic diagram of the joint calibration of monocular and binocular systems in this invention;

[0057] Figure 3 This is a schematic diagram illustrating the division of the left and right camera view areas in this invention;

[0058] Figure 4 This is a schematic diagram of the fusion of monocular and binocular 3D point cloud data in this invention.

[0059] Explanation of reference numerals in the attached figures:

[0060] 1-Left camera, 2-Projector, 3-Right camera, 4-Target object, 5-Checkerboard calibration plate, 6-Object being measured. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] Example 1

[0064] This embodiment will describe the present invention in detail with reference to the monocular and binocular structured light three-dimensional data fusion device and fusion method.

[0065] like Figure 1 The diagram shown is a schematic representation of the overall structure of the device in this embodiment. The device includes: a binocular structured light 3D reconstruction system, a left monocular structured light 3D reconstruction system, a right monocular structured light 3D reconstruction system, and a calibration system. The binocular structured light 3D reconstruction system includes: a left camera 1, a projector 2, and a right camera 3. The left monocular structured light 3D reconstruction system includes: a left camera 1 and a projector 2. The right monocular structured light 3D reconstruction system includes: a right camera 3 and a projector 2. The calibration system includes: a checkerboard calibration board 5 and a target object 4. In this embodiment, the target object 4 is a plaster cast with relatively rich 3D information.

[0066] The method includes the following steps:

[0067] S1. Perform individual calibrations on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system.

[0068] Specifically, firstly, left camera 1 and right camera 3 are used to photograph the checkerboard calibration board 5. The pose of the checkerboard calibration board 5 is changed to obtain multiple sets of checkerboard calibration board 5 patterns with different poses. The multiple sets of checkerboard calibration board 5 patterns with different poses are processed by the binocular camera calibration program of the open-source computer vision library OPENCV to complete the calibration of the binocular structured light 3D reconstruction system.

[0069] Then, projector 2 projects blank and striped patterns onto checkerboard calibration board 5. Left camera 1 and right camera 3 are used to photograph checkerboard calibration board 5. By adjusting different poses, multiple sets of blank and striped patterns of checkerboard calibration board 5 in different poses are obtained. The monocular camera calibration program of the open-source computer vision library OPENCV is used to process the multiple sets of blank patterns of checkerboard calibration board 5 in different poses to calibrate left camera 1 and right camera 3. The striped patterns are decomposed, and the projector 2 is calibrated using the "eight-parameter calibration method" to complete the calibration of the left monocular structured light 3D reconstruction system and the right monocular structured light 3D reconstruction system.

[0070] S2. Perform joint calibration on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system, which have been individually calibrated.

[0071] like Figure 2 As shown, in this embodiment, a checkerboard calibration board 5 and a target object 4 are used to jointly calibrate the above system. Specifically,

[0072] S21. Perform joint calibration on the binocular structured light 3D reconstruction system and the left monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TL.

[0073] S22. Perform joint calibration on the binocular structured light 3D reconstruction system and the right monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TR.

[0074] Specifically, projector 2 projects blank and striped patterns onto checkerboard calibration board 5, left camera 1 and right camera 3 capture blank and striped patterns on checkerboard calibration board 5, and binocular structured light 3D reconstruction system reconstructs the corner points of checkerboard calibration board 5 to obtain 3D point cloud dataset corner_B.

[0075] The five corner points of the checkerboard calibration board were reconstructed using a left monocular structured light 3D reconstruction system to obtain the 3D point cloud corner_ML; the five corner points of the checkerboard calibration board were also reconstructed using a right monocular structured light 3D reconstruction system to obtain the 3D point cloud corner_MR. Rigid body transformation matrix TL1 was obtained based on the 3D point cloud dataset corner_B and the 3D point cloud corner_ML; rigid body transformation matrix TR1 was obtained based on the 3D point cloud dataset corner_B and the 3D point cloud corner_MR.

[0076] The process of obtaining the rigid body transformation matrix includes:

[0077] Establish the equation:

[0078] P A =R·P B +T

[0079] In the formula, P A P represents the source 3D point cloud dataset. B Let R represent the target 3D point cloud dataset, where R represents the rotation matrix and T represents the translation matrix.

[0080] Rigid body transformation matrix H:

[0081] H = [RT].

[0082] Specifically, first construct matrix M:

[0083]

[0084] In the formula, N represents the size of the source 3D point cloud dataset and the target 3D point cloud dataset, and P i A From the source 3D point cloud dataset P A P i B From the target 3D point cloud dataset P B C A and C B P A P B The average center. Let: C A (x CA ,y CA ,z CA ),C B (x CB ,y CB ,z CB It can be obtained from the following formula:

[0085]

[0086] Then, singular value decomposition (SVD) is performed on matrix M, yielding the following equation:

[0087] [U,S,V]=SVD(M)

[0088] In the formula, U represents the left singular vector matrix, S represents the singular value matrix, and V represents the right singular vector matrix.

[0089] Then the rotation matrix R and the translation matrix T can be obtained as follows:

[0090]

[0091] This leads to the rigid body transformation matrix H. In this embodiment, the rigid body transformation matrices TL1 and TR1 are obtained by setting the source 3D point cloud dataset and the target 3D point cloud dataset using the above method.

[0092] The target object 4 is reconstructed based on a binocular structured light 3D reconstruction system to obtain the 3D point cloud data pointcloud_B of the target object 4 located in region B. Specifically, the projector 2 transmits the stripe pattern onto the target object 4, and the left camera 1 and the right camera 3 capture images of the target object 4 with the transmitted stripe pattern to obtain the 3D point cloud data pointcloud_B.

[0093] The target object 4 is reconstructed based on the left monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_ML of the target object 4 located in region B is obtained; the target object 4 is reconstructed based on the right monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_MR of the target object 4 located in region B is obtained.

[0094] The rigid body transformation matrix TL1 is used to perform a matrix transformation on the three-dimensional point cloud data pointcloud_ML to obtain the three-dimensional point cloud data pointcloud_T1_ML; the rigid body transformation matrix TR1 is used to perform a matrix transformation on the three-dimensional point cloud data pointcloud_MR to obtain the three-dimensional point cloud data pointcloud_T1_MR.

[0095] The IPC algorithm in the PCL point cloud library is used to perform fine registration on the 3D point cloud data pointcloud_B and pointcloud_T1_ML respectively, setting pointcloud_B as the target point cloud dataset and pointcloud_T1_ML as the source point cloud dataset, to obtain the rigid body transformation matrix TL2; fine registration is also performed on the 3D point cloud data pointcloud_B and pointcloud_T1_MR respectively, setting pointcloud_B as the target point cloud dataset and pointcloud_T1_MR as the source point cloud dataset, to obtain the rigid body transformation matrix TR2;

[0096] Multiplying the rigid body transformation matrices TL1 and TL2 yields the rigid body transformation matrix TL, which transforms the world coordinate system of the left monocular structured light 3D reconstruction system to the world coordinate system of the binocular structured light 3D reconstruction system. Multiplying the rigid body transformation matrices TR1 and TR2 yields the rigid body transformation matrix TR, which transforms the world coordinate system of the right monocular structured light 3D reconstruction system to the world coordinate system of the binocular structured light 3D reconstruction system.

[0097] S3. Based on the views captured by the left and right cameras, the view area is divided into region B and region M; the divided regions are reconstructed in three dimensions using a binocular structured light 3D reconstruction system, a left monocular structured light 3D reconstruction system, and a right monocular structured light 3D reconstruction system to obtain 3D point cloud data.

[0098] Methods for dividing regions include:

[0099] The left absolute phase image Image_phaL is obtained by deconstructing the test object 6 using the left monocular structured light 3D reconstruction system; the right absolute phase image Image_phaR is obtained by deconstructing the test object 6 using the right monocular structured light 3D reconstruction system.

[0100] Specifically, the projector 2 transmits the striped pattern onto the test object 6, and the left camera 1 and the right camera 3 capture images of the test object 6 with the transmitted striped pattern. The image of the test object 6 with the transmitted striped pattern captured by the left camera 1 is dephased to obtain the left absolute phase image Image_phaL; the image of the test object 6 with the transmitted striped pattern captured by the right camera 3 is dephased to obtain the right absolute phase image Image_phaR.

[0101] The calibration method based on the binocular structured light 3D reconstruction system obtains the mapping relationship MapL and the mapping relationship MapR. Based on the mapping relationship MapL and the mapping relationship MapR, the left absolute phase image Image_phaL and the right absolute phase image Image_phaR are respectively subjected to limit correction to obtain the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR.

[0102] Phase matching is performed on the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR. After phase matching, corresponding point pairs are obtained. The area of ​​the corresponding point pair in the corrected left absolute phase image Image_C_phaL is designated as the common normal region BC_L, and the remaining area is designated as the common abnormal region and the non-common region MC_L. Similarly, the area of ​​the corresponding point pair in the corrected right absolute phase image Image_C_phaR is designated as the common normal region BC_L, and the remaining area is designated as the common abnormal region and the non-common region MC_R. In this embodiment, the common normal region BC_L and the common normal region BC_R are the areas of the common normal region B in the left and right camera views of the image taken by the test object 6.

[0103] Based on the mapping relationship MapL, the common anomaly region and the non-common region MC_L are inversely mapped to obtain the region M of the left monocular structured light 3D reconstruction system view area. L Based on the mapping relationship MapR, the common anomaly region and the non-common region MC_R are inversely mapped to obtain the region M of the right monocular structured light 3D reconstruction system view area. R Region M is region M L and region M R The sum. For example, Figure 3 The diagram shown is a schematic representation of the area division.

[0104] The coordinates of the same point pairs in region B are used to perform three-dimensional reconstruction with the parameters obtained from the calibration of the binocular structured light 3D reconstruction system to obtain the 3D point cloud data cloud_B.

[0105] Based on region M L All point coordinates and the left absolute phase map Image_phaL are used to reconstruct the 3D point cloud data cloud_ML based on the absolute phase corresponding to the coordinates; based on region M R Based on the coordinates and the absolute phase of the right absolute phase map Image_phaR, a 3D reconstruction is performed on the corresponding absolute phase to obtain the 3D point cloud data cloud_MR.

[0106] S4. Perform rigid body transformation on the 3D point cloud data and merge them to obtain fused monocular and binocular 3D point cloud data.

[0107] like Figure 4 As shown, specifically, the TransformPointCloud function in the PCL point cloud library is used, with the rigid body transformation matrix TL and the rigid body transformation matrix TR as input parameters, to perform rigid body transformations on the 3D point cloud data cloud_ML and cloud_MR respectively, to obtain the 3D point cloud data cloud_T_ML and cloud_T_MR. The 3D point cloud data cloud_B, cloud_T_ML, and cloud_T_MR are then merged using relevant algorithms from the PCL point cloud library to obtain fused monocular and binocular 3D point cloud data.

[0108] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for fusion of monocular and binocular structured light three-dimensional data, characterized in that, Includes the following steps: S1. Perform individual calibrations on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system. S2. Perform joint calibration on the binocular structured light 3D reconstruction system, the left monocular structured light 3D reconstruction system, and the right monocular structured light 3D reconstruction system, which have been individually calibrated. S3. Based on the views captured by the left and right cameras, divide the view area into region B and region M; The three-dimensional reconstruction of the divided region is performed based on the binocular structured light three-dimensional reconstruction system, the left monocular structured light three-dimensional reconstruction system, and the right monocular structured light three-dimensional reconstruction system to obtain three-dimensional point cloud data; S4. Perform rigid body transformation on the three-dimensional point cloud data and merge them to obtain fused monocular and binocular three-dimensional point cloud data; S2 includes: S21. Perform joint calibration on the binocular structured light 3D reconstruction system and the left monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TL; S22. Perform joint calibration on the binocular structured light 3D reconstruction system and the right monocular structured light 3D reconstruction system to obtain the rigid body transformation matrix TR; S21 includes: Based on the binocular structured light 3D reconstruction system, the 5 corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud dataset corner_B; Based on the left monocular structured light 3D reconstruction system, the 5 corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud corner_ML; Based on the three-dimensional point cloud dataset corner_B and the three-dimensional point cloud corner_ML, the rigid body transformation matrix TL1 is obtained; The target object is reconstructed based on the binocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_B of the target object located in region B is obtained. The target object is reconstructed based on the left monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_ML of the target object located in region B is obtained. The rigid body transformation matrix TL1 is used to perform matrix transformation on the three-dimensional point cloud data pointcloud_ML to obtain the three-dimensional point cloud data pointcloud_T1_ML; The three-dimensional point cloud data pointcloud_B and the three-dimensional point cloud data pointcloud_T1_ML are finely registered to obtain the rigid body transformation matrix TL2. Multiply the rigid body transformation matrix TL1 and the rigid body transformation matrix TL2 to obtain the rigid body transformation matrix TL; S22 includes: Based on the right monocular structured light 3D reconstruction system, the 5 corner points of the chessboard calibration board are reconstructed to obtain the 3D point cloud corner_MR; Based on the three-dimensional point cloud dataset corner_B and the three-dimensional point cloud corner_MR, the rigid body transformation matrix TR1 is obtained; The target object is reconstructed based on the binocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_B of the target object located in region B is obtained. The target object is reconstructed based on the right monocular structured light 3D reconstruction system, and the 3D point cloud data pointcloud_MR of the target object located in region B is obtained. The rigid body transformation matrix TR1 is used to perform matrix transformation on the three-dimensional point cloud data pointcloud_MR to obtain the three-dimensional point cloud data pointcloud_T1_MR; The three-dimensional point cloud data pointcloud_B and the three-dimensional point cloud data pointcloud_T1_MR are finely registered to obtain the rigid body transformation matrix TR2. Multiplying the rigid body transformation matrix TR1 and the rigid body transformation matrix TR2 yields the rigid body transformation matrix TR.

2. The method for fusion of monocular and binocular structured light three-dimensional data according to claim 1, characterized in that, The method for dividing the region includes: Based on the left monocular structured light 3D reconstruction system, the phase of the object under test is deconstructed to obtain the left absolute phase map Image_phaL; Based on the right monocular structured light 3D reconstruction system, the phase of the object under test is deconstructed to obtain the right absolute phase map Image_phaR; Based on the binocular structured light 3D reconstruction system, mapping relationships MapL and MapR are obtained. Based on the mapping relationships MapL and MapR, the left absolute phase image Image_phaL and the right absolute phase image Image_phaR are respectively subjected to limit correction to obtain the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR. Phase matching is performed on the corrected left absolute phase image Image_C_phaL and the corrected right absolute phase image Image_C_phaR. After phase matching, corresponding point pairs are obtained. The region of the corresponding point pairs in the corrected left absolute phase image Image_C_phaL is designated as the common normal region BC_L, and the remaining region is designated as the common abnormal region and the non-common region MC_L. The corresponding point pairs are designated as the common normal region BC_R in the corrected right absolute phase map Image_C_phaR, and the remaining region is designated as the common abnormal region and the non-common region MC_R. Based on the mapping relationship MapL, the common anomaly region and the non-common region MC_L are reverse mapped to obtain the region ML of the left monocular structured light 3D reconstruction system view area; Based on the mapping relationship MapR, the common anomaly region and the non-common region MC_R are reverse mapped to obtain the region MR of the right monocular structured light 3D reconstruction system view area; The region B is the sum of the common normal region BC_L and the common normal region BC_R; The region M is the sum of region ML and region MR.

3. The method for fusion of monocular and binocular structured light 3D data according to claim 2, characterized in that, S3 includes: The coordinates of the corresponding point pairs in region B are used to perform three-dimensional reconstruction with the parameters obtained by the calibration of the binocular structured light three-dimensional reconstruction system to obtain three-dimensional point cloud data cloud_B. Three-dimensional point cloud data cloud_ML is obtained by reconstructing the three-dimensional point cloud based on the coordinates of all points in region ML and the absolute phase of the left absolute phase map Image_phaL corresponding to the coordinates; three-dimensional point cloud data cloud_MR is obtained by reconstructing the three-dimensional point cloud based on the coordinates of region MR and the absolute phase of the right absolute phase map Image_phaR corresponding to the coordinates.

4. The method for fusion of monocular and binocular structured light three-dimensional data according to claim 3, characterized in that, The TransformPointCloud function is used to perform rigid body transformation on the three-dimensional point cloud data cloud_ML and cloud_MR based on the rigid body transformation matrix TL and the rigid body transformation matrix TR, respectively, to obtain three-dimensional point cloud data cloud_T_ML and cloud_T_MR. The three-dimensional point cloud data cloud_B, cloud_T_ML, and cloud_T_MR are merged to obtain the fused monocular and binocular three-dimensional point cloud data.

Images