Point cloud splicing method based on optical mark point three-dimensional centroid matching

Abstract

The invention discloses a point cloud splicing method based on three-dimensional centroid matching of optical mark points, which comprises the following steps of: reconstructing the three-dimensional shape of an object through a monocular system, randomly generating three-dimensional mark points in a standard geometrical shape in a target flat area, reversely resolving the three-dimensional mark points to a two-dimensional projector to generate a mark point pattern, and projecting the mark point pattern to the object; constraint of a public area under multiple view angles is realized; the method comprises the following steps: reconstructing a three-dimensional point cloud and calculating a geometric centroid of a cursor in a public area as a multi-view feature matching point by using data collected by a binocular system at different view angles, constructing feature point pairs based on spatial topology invariance, solving a rigid transformation matrix RT through SVD, and realizing high-precision point cloud splicing. Mark points do not need to be manually pasted, and the splicing efficiency is greatly improved; through three-dimensional centroid constraint, the problem that weak texture splicing errors are large is solved, robustness is high, and high-precision multi-view three-dimensional splicing is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and particularly relates to a point cloud stitching method based on three-dimensional centroid matching of optical fiducial points. Background Art

[0002] Three-dimensional stitching technology has important application values in the fields of industrial inspection, virtual reality, reverse engineering, and digital protection of cultural relics and heritage. Especially in the integration of multi-view three-dimensional shape data of large and complex structure objects, high-precision three-dimensional stitching is an important step in point cloud data processing and also a key technology to ensure the integrity and consistency of subsequent models. Fringe Projection Profilometry (FPP) has the advantages of high precision, high efficiency, non-contact measurement, and convenient data processing, and is a typical three-dimensional reconstruction technology of structured light method for surface. By projecting sinusoidal encoded fringes onto an object, the sinusoidal fringes modulated by the surface topography of the object are synchronously captured by a camera to obtain a deformed fringe pattern containing height information. Subsequently, the phase information of each pixel point on the image is solved by a temporal phase unwrapping algorithm, and this information can establish the corresponding relationship between the camera imaging plane and the pixel coordinate system of the projector, thereby realizing sub-millimeter three-dimensional topography measurement of the object surface under a single view.

[0003] However, in the process of three-dimensional digital reconstruction, affected by the self-occlusion of the object itself and the resolution limitation of the measurement system, it is usually difficult to obtain complete point cloud data by single-view scanning, and the collected three-dimensional data only contains partial information of the object surface. Therefore, to obtain the complete three-dimensional topography of the object, a multi-view data acquisition strategy needs to be established: first, adjust the device pose to obtain a point cloud data set covering all directions of the object, and then use a registration algorithm to unify the discrete point clouds into the same world coordinate system.

[0004] Traditional multi-view stitching methods are generally divided into two stages: initial registration and fine registration. Initial registration aims to roughly align and reduce the rotational and translational errors between point clouds. Common methods include the calibration point method (manually pasting high-contrast landmark points on the object surface for registration), the object self-feature extraction method, and the projected landmark point method. However, the above methods have significant limitations. The calibration point method is complex, requires manual intervention, and has poor environmental adaptability. The feature extraction method is easily affected by point cloud sparsity, noise, and low-texture areas, resulting in insufficient matching robustness. In addition, most landmark point registration methods rely on the extraction and matching of two-dimensional projection features. Such methods project two-dimensional encoded patterns (such as circular landmark points) on the object surface through a projector, collect images from different perspectives using a camera, and extract the center coordinates of the circle as matching feature points, and then solve the rotation and translation matrix (R|T) to complete point cloud stitching. However, when there is a viewing angle difference between the projector and the camera, there is an angle between the physical plane of the spatial circle and the camera imaging plane. At this time, the projected image is an ellipse, and there is a deviation between the geometric center of the ellipse and the true projection point of the three-dimensional circle center. Under the condition of oblique projection, the offset Δp of the ellipse center is positively correlated with the projection angle, resulting in an error in the extraction of the two-dimensional circle center. Therefore, there will be a deviation in 2-D during the stitching process. To address the above problems, existing improved methods, such as the iterative virtual calibration method, the concentric circle constraint method, etc., can compensate for some eccentricity errors, but still rely on complex optical correction models and multi-view data fusion, with high computational complexity.

[0005] In the fine registration stage, the Iterative Closest Point (ICP) algorithm is simple and intuitive and is a classic fine registration algorithm. However, it highly depends on the initial registration accuracy, is prone to falling into local optimal solutions, and is sensitive to changes in the point cloud topology structure and noise. To overcome the defects of ICP, researchers have proposed improved algorithms such as Normal Distribution Transformation (NDT), Corresponding-Contrast Points (CCP), and Coherent Point Drift (CPD), etc. However, these methods still face problems such as high computational complexity and poor adaptability to uneven point cloud density. Summary of the Invention

[0006] The purpose of the present invention is to provide a point cloud stitching method based on three-dimensional centroid matching of optical landmark points. By reconstructing the three-dimensional morphology of the target surface through a monocular structured light system, standard geometric shapes are randomly generated in the flat area of the target as three-dimensional landmark areas, and their geometric centroids are extracted as cross-view matching points, eliminating the 2-D deviation introduced by two-dimensional projection deformation. Starting from three-dimensional attributes, this method breaks through the problem of three-dimensional point cloud stitching errors caused by deformation errors of traditional two-dimensional cursor due to the angle between the projector, camera optical axis and the target plane, solves the problem of large stitching errors in weak texture, effectively improves the stitching accuracy, and realizes high-precision multi-view three-dimensional stitching.

[0007] The technical solution to achieve the purpose of the present invention is: a point cloud stitching method based on three-dimensional centroid matching of optical landmark points, including the following steps:

[0008] Step 1: Set up a monocular structured light system and a binocular stereo vision system. Make the projector project a set of sinusoidal fringes onto the object to be measured, and have the camera synchronously capture the deformed fringes after being modulated by the surface of the object to be measured. Use the Zhang Zhengyou calibration method to calibrate the internal and external parameters of the camera and the projector.

[0009] Step 2: Based on the deformed fringe images captured by the monocular system, obtain the three-dimensional points of the object through phase unwrapping and three-dimensional reconstruction algorithms. Randomly generate three-dimensional fiducial points with standard geometric shapes in the target flat area, and record the three-dimensional coordinates.

[0010] Step 3: Reverse-project the three-dimensional fiducial points into the pixel coordinate system of the projector to generate a two-dimensional encoded pattern with feature points, and project it onto the object surface through the projector.

[0011] Step 4: Move the binocular stereo vision system to Viewpoint 1, and collect the fringe pattern and the fiducial point encoded image calculated in Step 3. Then move the binocular system to Viewpoint 2, repeat the above data collection to ensure there is a common area between the two viewpoints.

[0012] Step 5: Through the images collected by the binocular system, reconstruct the targets captured at Position 1 and Position 2 respectively, and extract the three-dimensional point clouds of each fiducial area. Use the point cloud density weighted algorithm to calculate the geometric centroid of each three-dimensional fiducial point, and construct the centroid feature point set under the two viewpoints.

[0013] Step 6: Match the centroid feature points of the two viewpoints through the spatial distribution topological relationship, and use the singular value decomposition algorithm to solve the rotation and translation matrix between the two viewpoints.

[0014] Step 7: Transform the three-dimensional point cloud of Viewpoint 2 to the global coordinate system of Viewpoint 1 through the rotation and translation matrix obtained in Step 6 to achieve cross-view three-dimensional point cloud stitching.

[0015] In the above Step 1, the projector projects the phase-encoded fringe pattern and the fiducial point pattern onto the object surface, and the industrial camera synchronously captures the deformed fringes and the fiducial point pattern, and completes the system calibration through a circular calibration board.

[0016] Further, in the above Step 2, at random positions in the flat area of the target point cloud, divide standard geometric shapes of different sizes as three-dimensional feature regions, and the shapes and topological structures of the three-dimensional regions reconstructed under different viewpoints remain unchanged.

[0017] Further, in the above Step 3, the method of reverse projection is as follows: According to the internal and external parameters of the monocular system, reverse-solve the point cloud coordinates in the three-dimensional feature region to the two-dimensional pixel plane of the projector, and the formula is as follows; project the fiducial point pattern in the common area to construct the geometric constraint of the common area under multiple viewpoints.

[0018] ;

[0019] ;

[0020] in, is the homography matrix, 、 are the internal and external parameters of the projector, is the three-dimensional coordinate of the point, are the calculated projector pixel coordinates.

[0021] Furthermore, in step 4, when the binocular system captures the image, the marker point coding map is used as a mask to perform three-dimensional reconstruction only on the point cloud within the marker area, ensuring that the reconstruction range is consistent with the initially divided three-dimensional feature area.

[0022] Furthermore, in step 5, the geometric centroid of each 3D marker point is calculated using a point cloud density weighted algorithm. , construct the centroid feature point set under two perspectives;

[0023] ;

[0024] ;

[0025] in, is the number of point clouds in the feature area, is the three-dimensional coordinate of the point cloud, is the set of centroids, represents the centroid coordinates of the j-th 3D feature region in view 1, represents the centroid coordinates of the j-th 3D feature area in view 2, and Represent the total number of 3D feature regions extracted under view 1 and view 2, respectively.

[0026] Furthermore, in step 6, the specific centroid matching method is: encoding and sorting the centroid feature points of the two perspectives, establishing a one-to-one correspondence based on the encoding, and forming a centroid point pair set;

[0027]

[0028] in, is the total number of centroid matches, is the centroid coordinate of the first viewing angle fitting in the same three-dimensional region, is the centroid coordinate corresponding to the second viewing angle of the same feature area.

[0029] Furthermore, we first construct the covariance matrix using the centroid matching point pairs , use SVD singular value decomposition method to solve the rigid transformation matrix ; Obtain the rigid change matrix, transfer the multi-view point cloud to the global coordinate system, and finally realize point cloud stitching;

[0030]

[0031]

[0032]

[0033] in Represents the centroid coordinate set central location, is the left singular vector matrix, providing the main direction information of the target point, Is the right singular vector matrix, providing the main direction of the source point cloud, diagonal matrix To measure the data distribution scale of the point cloud in each main direction, is the rotation matrix, is the translation matrix.

[0034] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0035] A computer-readable storage medium has program codes corresponding to the executable instruction set, and the method is implemented when the program is executed by a processor.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) Based on the three-dimensional marker reverse mapping method, the present invention reversely decodes the three-dimensional feature area to the two-dimensional projector plane to generate a marker point pattern, and uses the projector to project the marker point pattern onto the object, effectively solving the problem of large splicing errors caused by missing features of weak-textured objects; compared with the traditional manual marker point pasting method, this method does not require the use of physical markers, which significantly improves the ease of operation; at the same time, the projector can change the position and size of the marker points in real time according to the three-dimensional morphology of the object itself, expanding the application scenarios and significantly improving the environmental adaptability of this method.

[0038] (2) Based on the spatial invariance of the 3D region centroid, a mapping relationship between multiple perspectives is established, which solves the pose calculation error caused by the angle between the projector and camera optical axis and the target plane in traditional 2D cursor feature matching. Compared with traditional 2D cursor feature matching methods (such as SIFT and ORB), this method can improve the registration accuracy to sub-millimeter level by using 3D geometric centroid constraints, and does not require the use of ICP fine registration algorithm, which significantly improves computational efficiency.

[0039] (3) The centroid coordinates are calculated based on the weighted density of the regional point cloud, with strong anti-noise performance and insensitivity to perspective changes, significantly improving the registration robustness. Description of the Drawings

[0040] Figure 1 It is a diagram of the circular cursor projection deviation model.

[0041] Figure 2 It is a schematic diagram of the point cloud stitching technology based on the three-dimensional centroid matching of optical fiducial points proposed by the present invention. Among them, the upper figure shows the inverse solution of the three-dimensional feature region to the two-dimensional projector, and the lower figure shows the schematic diagram of the stitching process.

[0042] Figure 3 It is a schematic diagram of the flow of the method of the present invention.

[0043] Figure 4 It is a schematic diagram of the experimental device.

[0044] Figure 5 It is the result of the inverse solution of the three-dimensional feature region to the two-dimensional projector; among them, the left figure shows randomly generating standard geometric three-dimensional graphic fiducial points (such as circles, triangles or rectangles) in the flat area of the workpiece, and the right figure shows the inverse projection of the three-dimensional region fiducial points to the two-dimensional figure of the projector.

[0045] Figure 6 It is the fiducial point diagrams taken at two perspectives respectively; among them, the left figure is the fiducial point coding diagram taken at the first perspective, and the right figure is the fiducial point coding diagram taken at the second perspective.

[0046] Figure 7 It is the point cloud of the three-dimensional feature region reconstructed at the first perspective and the geometric centroid of each region.

[0047] Figure 8 It is the point cloud of the three-dimensional feature region reconstructed at the second perspective and the geometric centroid of each region.

[0048] Figure 9 It is the stitching result of the method of the present invention; among them, the left figure is the three-dimensional point cloud of the object at the first perspective, the middle figure is the three-dimensional point cloud of the object at the second perspective, and the right figure is the final point cloud stitching effect.

[0049] Figure 10 It is the three-dimensional stitching results of the traditional two-dimensional feature-based stitching and the method of the present invention for the wall; among them, the left figure is the effect of stitching the wall by the traditional two-dimensional cursor center feature stitching method, and the right figure is the effect of stitching the wall by the present invention.

[0050] Figure 11The result of three-dimensional stitching of the traditional two-dimensional feature stitching and the method of the present invention for the standard double-sphere workpiece of 1000.4777; among them, the left figure is the result of stitching the standard double-sphere workpiece by the traditional two-dimensional cursor center feature stitching method, and the right figure is the result of stitching the standard double-sphere workpiece by the present invention. Detailed implementation mode

[0051] Aiming at the problems of strong dependence on artificial markers, insufficient registration robustness, and high computational complexity in multi-view three-dimensional stitching, the present invention proposes a point cloud stitching technology based on three-dimensional centroid matching of optical fiducial points, abandoning the traditional two-dimensional feature dependence, and directly constructing registration constraints from the geometric attributes of the three-dimensional point cloud. By extracting the geometric centroid of the local three-dimensional region in the multi-view point cloud as the feature matching point, cross-view point cloud stitching is realized. This method breaks through the problem of three-dimensional point cloud stitching error caused by the deformation error of the traditional two-dimensional cursor due to the angle between the projector, camera optical axis and the target plane, and significantly reduces the pose solution error. In addition, this method can achieve sub-millimeter stitching accuracy and reduce the computational complexity without the need for fine registration algorithms such as ICP. Experimental results show that the proposed method can accurately realize cross-view point cloud stitching in low-texture, large-scale and complex curved surface scenes.

[0052] A point cloud stitching method based on three-dimensional centroid matching of optical fiducial points of the present invention includes the following steps:

[0053] Step 1: Build a monocular structured light system and a binocular stereo vision system. Any one of the cameras in the binoculars is used as the camera of the monocular system. The projector projects a set of sinusoidal fringes onto the object to be measured, and the camera synchronously captures the deformed fringes modulated by the surface of the object to be measured. The Zhang-Zhengyou calibration method is used to calibrate the internal and external parameters of the camera and the projector.

[0054] Step 2: Based on the deformed fringe image captured by the monocular system, the three-dimensional points of the object are obtained through the phase unwrapping and three-dimensional reconstruction algorithms. Three-dimensional fiducial points of standard geometric shapes are randomly generated in the target flat area, and the three-dimensional coordinates are recorded.

[0055] Step 3: Reverse-project the three-dimensional fiducial points into the projector pixel coordinate system to generate a two-dimensional encoded pattern with feature points, and project it onto the object surface through the projector.

[0056] Step 4: Move the binocular stereo vision system to view angle one, and collect the fringe image and the fiducial point encoded image calculated in step 3; then move the binocular system to view angle two, repeat the above data collection to ensure that there is a common area between the two views.

[0057] Step 5: For the images captured by the binocular system, reconstruct the objects captured at Position 1 and Position 2 respectively, and extract the three-dimensional point clouds of each marked area. Use the point cloud density weighted algorithm to calculate the geometric centroids of each three-dimensional marked point, and construct the centroid feature point sets under the two perspectives.

[0058] Step 6: Match the centroid feature points of the two perspectives through the spatial distribution topological relationship, and use the singular value decomposition (SVD) algorithm to solve the rotation and translation matrix between the two perspectives.

[0059] Step 7: Transform the three-dimensional point cloud of Perspective 2 to the global coordinate system of Perspective 1 through the rotation and translation matrix obtained in Step 6 to achieve cross-perspective three-dimensional point cloud stitching.

[0060] Further, the system in Step 1 consists of a projector, an industrial camera, and a circular calibration plate. The projector projects the phase-encoded stripe pattern and the marked point pattern onto the object surface, and the industrial camera synchronously captures the deformed stripes and the marked point pattern, and completes the system calibration through the circular calibration plate.

[0061] Further, no matter at what angle the three-dimensional measurement system takes pictures, randomly divide the three-dimensional feature areas of standard geometric figures with different sizes in the flat area of the object of the monocular reconstructed point cloud. The reconstructed three-dimensional areas have shape consistency, and their topological structures remain unchanged from the original divided figures.

[0062] Further, the reverse projection mapping technology can be adopted to back-project the divided three-dimensional feature areas onto the two-dimensional pixel plane of the projector. Project the marked point coding map at the common area of the two perspectives, so that the system captures the feature area images and stripe images with unique coding identifiers at each perspective, and constructs the geometric constraints of the common area under multiple perspectives.

[0063] Further, use the calibrated binocular system to reconstruct the workpieces captured at different perspectives, and use the feature marked map captured in the previous step as a mask to perform three-dimensional reconstruction only on the point cloud within the marked area to ensure that the reconstruction range is consistent with the initially divided three-dimensional feature area.

[0064] Further, extract the geometric centroids of each three-dimensional marked area under the two perspectives and encode them, establish the centroid correspondence according to the encoding, and construct the centroid point pair coordinate set.

[0065] Further, based on the set of matched centroid point pairs, use the singular value decomposition algorithm to solve the optimal rotation matrix R and translation vector T, and finally transform the point cloud of Perspective 2 to the global coordinate system, so as to achieve the stitching of the point cloud of multiple perspectives.

[0066] The method of the present invention is based on the invariance of the geometric centroid of a three-dimensional region. By using a monocular system to inversely map three-dimensional landmark features onto the projection plane, feature landmark points are generated. The constraints between multiple perspectives are directly constructed using the invariance of the three-dimensional region centroid, breaking through the problem of three-dimensional point cloud stitching errors caused by deformation errors of traditional two-dimensional cursors due to the angles between the projector, camera optical axis, and the target plane. High-precision reconstruction of the three-dimensional point cloud of weakly textured objects can be achieved. This method uses the DBSCAN clustering algorithm to segment the feature three-dimensional regions and extract the centroid of each region as the point cloud stitching feature points, without relying on manually pasted landmark points and the ICP iterative algorithm, significantly reducing the computational complexity. At the same time, it improves the registration robustness in complex scenarios such as low texture and large scale, effectively expanding the application scenarios of traditional feature matching algorithms. Experimental results show that this method can achieve seamless stitching of high-precision cross-perspective three-dimensional point clouds.

[0067] The technical solution of the present invention will be described in detail below in conjunction with the embodiments.

[0068] Embodiment

[0069] The present invention will be further clarified below in conjunction with the accompanying drawings and specific embodiments.

[0070] A. Influence of Two-Dimensional Projection Deviation on Three-Dimensional Stitching Accuracy

[0071] For the cursor projection stitching process, two-dimensional projection deviation is an important factor affecting three-dimensional stitching accuracy. When there is an angular tilt between the camera or projector and the target object, the projector projects a standard circular cursor onto the object. At this time, the projection of the standard circle on the imaging plane becomes an ellipse, and the center of the ellipse is not the projection image point of the actual circle center. If the corresponding three-dimensional point is directly calculated using the ellipse center as the stitching feature point, it will lead to a deviation in the calculation of the three-dimensional rigid body transformation matrix Calculation errors will occur, ultimately resulting in stratification and misalignment in the stitching area, seriously affecting the accuracy and effect of three-dimensional stitching.

[0072] The error stems from the non-orthogonality of perspective projection. Taking the oblique cone projection model as an example, as Figure 1 shown: The projection of the circle in the camera coordinate system can be regarded as the cross-section of an oblique cone, and there is a deviation between the center of the ellipse and the projection of the true circle center. The deviation can be expressed by the following formula:

[0073]

[0074] Where respectively represent the deviations of the ellipse center and the actual circle center on the axis axis, is the object distance, is the circle radius, is the angle between the camera optical axis and the imaging plane, is the spatial circle center coordinate 。

[0075] The above deviation results in a rigid transformation matrix There is an error in the calculation of the matrix, which in turn affects the three-dimensional stitching error. If in a binocular system, both the left and right cameras use the ellipse center as the feature point, the error of three-dimensional point reconstruction is as follows:

[0076]

[0077] Therefore, based on the traditional two-dimensional cursor center feature matching method, this kind of stitching will accumulate step by step in multi-view stitching, resulting in stratification or distortion of the three-dimensional model. The present invention is based on the three-dimensional centroid of the three-dimensional region as the feature point, directly starting from the three-dimensional perspective to effectively avoid the two-dimensional projection deviation problem, with higher accuracy.

[0078] B. Mathematical Model and Calibration of the Measurement System

[0079] As Figure 2 、 Figure 3 shown, an example of a point cloud stitching method based on three-dimensional centroid matching of optical fiducial points invented includes the following steps:

[0080] Step 1: Use a projector to project a set of phase-shifted sine fringes, and use a camera to synchronously collect. Further, the fringe expression collected by the camera is:

[0081]

[0082] where represents the number of phase steps, is the fringe image collected by the camera, represents the pixel coordinates of the camera, and respectively represent the background light intensity and the sine fringe modulation degree, represents the phase information to be solved, is the fringe phase shift amount.

[0083]

[0084] The wrapped phase of is obtained by solving through the above arctangent function, and its range is . Use the time phase unwrapping algorithm to solve the phase discontinuity problem, and obtain the required absolute phase through the following method:

[0085]

[0086] where is the fringe order.

[0087] Calibration of the measurement system: The traditional camera model is the pinhole imaging model. The object surface reaches the camera imaging surface through four coordinate systems: the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system. Its world coordinate and pixel coordinate are related as shown in the following formula:

[0088]

[0089] where is the normalization coefficient, is the component of the focal length in the direction, is the unit pixel size, is the pixel coordinate of the optical center, represents the rotation and translation matrix, is the internal parameter matrix, is the external parameter matrix.

[0090] The projector model is the same as the camera model. The projector projects structured light stripes in the horizontal and vertical directions on the calibration board respectively. Phase recovery and phase unwrapping are performed on the two sets of acquired stripe images, and then the absolute phase distribution maps in the two directions are obtained, which are respectively expressed as and , that is, the inverse process of camera shooting. However, different from the camera that can directly obtain pixel coordinates, it is necessary to calculate the projector pixel coordinates by means of phase information. Let the stripe frequencies of the projector in the horizontal and vertical directions be and respectively, and the resolutions of the projector in the two directions are and . The corresponding position coordinates in the projector image pixel coordinate system can be calculated by the following formula: can be obtained from the following formula:

[0091]

[0092] where is the camera pixel coordinate, is the calculated projector pixel coordinate.

[0093] The camera is calibrated based on the Zhang Zhengyou calibration method. Circular checkerboards are placed at different positions in the field of view. The center coordinates of the checkerboards at different positions are obtained through the center extraction method. According to the center coordinates of the checkerboards and the world coordinates, the homography matrix is obtained, where is called the internal parameter matrix, is the external parameter matrix.

[0094]

[0095] In stereo vision, when the target points in the three-dimensional space are synchronously collected by the device, the two-dimensional projection coordinates formed by the target points on different imaging planes are and Combined with the device calibration parameters, we can obtain the 3D data of the point and complete the 3D reconstruction at that perspective. We can list the equation as follows:

[0096]

[0097]

[0098] in , are the normalization coefficients of the camera and projector respectively, are the camera’s intrinsic and extrinsic parameters, are the internal and external parameters of the projector.

[0099] Further combining the equations, we can obtain:

[0100]

[0101] The above equation contains four equations and three unknowns , so the least squares method can be used to solve the system of equations to obtain the optimal solution.

[0102] C. Point cloud stitching based on 3D centroid matching of optical landmarks

[0103] Step 2: Analyze the 3D shape of the object reconstructed by the monocular system, and randomly divide the 3D regions of different sizes and geometric standard shapes at different positions in the flat area of the object point cloud, such as Figure 5 As shown in the left figure in . Based on the 3D region invariance, that is, the 3D region reconstructed at different angles remains unchanged, constraints between multiple perspectives are constructed.

[0104] Step 3: If Figure 5 As shown in the right figure, through the inverse operation of three-dimensional measurement, the formula is as follows. The divided three-dimensional area can be inversely solved to the two-dimensional projector, and the landmark point coding map can be projected in the common area of the two perspectives to realize the constraint of the common area under multiple perspectives and provide feature constraints for the stitching of the two perspectives.

[0105]

[0106] Step 4: Project the image of the marked point features containing coding information obtained in Step 3, and use the binocular three-dimensional measurement system to take pictures from different perspectives. Among them, Perspective 1 refers to the measurement position when the projector and the camera are in the initial measurement position; Perspective 2 represents the new measurement position formed after the system moves. At this time, the camera and the projector have moved to the new positions, and it is ensured that there is a certain common area between the two measurements to meet the feature matching requirements for subsequent stitching. Obtain the fringe pattern and the marked point feature map from different perspectives, as Figure 6 shown.

[0107] Step 5: Perform three-dimensional reconstruction based on the fringe patterns taken from different perspectives. At the same time, use Figure 6 as a mask to ensure that the three-dimensional reconstruction is only carried out within the circular cursor area, and calculate the three-dimensional marked areas captured from different perspectives. Perform region segmentation on the reconstructed three-dimensional feature area through the density clustering algorithm (DBSCAN), divide the point cloud into multiple independent sub-regions, and calculate the centroid coordinates of each region of the three-dimensional feature area through the following formula ( , , ), construct the centroid coordinate set , and perform visual display on the image.

[0108]

[0109]

[0110] Among them, is the number of point clouds in the feature area, is the three-dimensional coordinate of the point cloud, is the set composed of centroids, represents the centroid coordinate of the j-th three-dimensional feature area in Perspective 1, represents the centroid coordinate of the j-th three-dimensional feature area in Perspective 2, and respectively represent the total number of three-dimensional feature areas extracted in Perspective 1 and Perspective 2.

[0111] Step 6: According to Figure 7 , Figure 8 , code these three-dimensional areas according to a certain rule. The feature areas of the two perspectives can be matched according to the code, and a one-to-one correspondence relationship is established according to the code to form a set of centroid point pairs.

[0112]

[0113] Among them, is the total number of centroid matches, is the centroid coordinate of the first viewing angle fitting in the same three-dimensional region, is the centroid coordinate corresponding to the second viewing angle of the same feature area.

[0114] We can further calculate the transformation relationship between the two perspectives based on the centroid point pair obtained above. Here we only consider rigid transformations, that is, the shape of the data under the two perspectives remains unchanged, there is no scaling transformation, and only rotation and translation transformations are considered.

[0115]

[0116] in, For around The angle of axis rotation, For around The angle of axis rotation, For around The angle of axis rotation, is the translation matrix The components on each coordinate axis, is the rotation matrix, ( ) is the coordinate value of the point cloud after rotation and translation.

[0117] Based on the matched centroid point pair set, the error function is constructed and calculated. The matrix is equivalent to solving the minimum value of the error function.

[0118]

[0119] Use SVD singular value decomposition method to solve, first construct the covariance matrix , use linear algebra knowledge to perform SVD decomposition and then solve .

[0120]

[0121]

[0122]

[0123] in Represents the centroid coordinate set central location, is the left singular vector matrix, providing the main direction information of the target point, Is the right singular vector matrix, providing the main direction of the source point cloud, diagonal matrix It is used to measure the data distribution scale of the point cloud in each main direction.

[0124] Step 7: Based on the result from step 6 a matrix to transform the overall three-dimensional point cloud of Viewpoint Two into the global coordinate system , ultimately realizing the cross-view three-dimensional point cloud stitching. As Figure 9 shown is the final stitching effect.

[0125]

[0126] D. Experiment

[0127] To verify the effect of the present invention, a three-dimensional point cloud stitching measurement system was built, including a three-dimensional measurement device and a fiducial point projection device. The experiment used a fringe projection binocular stereo vision matching system, including a DMD projector (DLP6500 with a resolution of 1920*1200), two CMOS cameras (Basler acA1920-155um, with a resolution of 1920×1200 pixels), and the cameras were equipped with camera lenses with a focal length of 12mm. For the fiducial point projection device, we used an HD301A1-H2 microelectromechanical system (MEMS) projector with a resolution of 800*800 pixels. The fringes used in the experiment were , , to generate an 8-step phase-shifting pattern. The object to be measured was placed at a position about 0.8m from the binocular measurement system. The experimental setup was as Figure 4 shown. First, it was placed at the middle position of the object, and a monocular measurement system constructed by the MEMS projector and one of the cameras in the binocular system was used for reconstruction to generate a three-dimensional feature constraint graph and back-solve it to the two-dimensional MEMS projector. Subsequently, the binocular measurement system was placed at Viewpoint One and Viewpoint Two respectively for data acquisition, including fringe images and fiducial point feature images. Viewpoint One refers to the observation view when the projector and the camera are in the initial measurement position, and Viewpoint Two refers to the observation view formed after the system moves to a new measurement position. It is necessary to ensure that there is a certain common area in the two measurements to meet the feature matching requirements in the subsequent stitching process. Finally, the RT matrix was calculated according to the constraints to achieve three-dimensional point cloud stitching.

[0128] We further tested the performance of the point cloud stitching technology based on the three-dimensional centroid matching of optical fiducial points of the present invention through experiments. The wall surface and a standard double-sphere workpiece were measured, and the error between the fitted center distance of the standard double-sphere workpiece and the actual center distance was used as the accuracy evaluation of the stitching error.

[0129] Figure 10The comparative experimental results of two point cloud stitching methods for stitching a wall surface are shown. The left figure shows the stitching effect of the traditional two-dimensional cursor center feature matching. Although initial position alignment can be achieved, there are obvious layered misalignments and "wavy" errors at the stitching boundary. The right figure presents the point cloud stitching result based on the three-dimensional centroid matching of optical fiducial points. There is no layering phenomenon in the stitching transition area of this method, and the dual-view point clouds are seamlessly fused in terms of spatial continuity and geometric consistency. The experimental results show that the point cloud stitching method based on the three-dimensional centroid matching of optical fiducial points proposed in this study has significant advantages in stitching accuracy and surface continuity.

[0130] Figure 11 The comparative experimental results of the stitching accuracy of a standard double-sphere workpiece (theoretical center distance between spheres is 1000.4777 mm) are shown. The experimental data show that: the left figure shows that the center distance between spheres fitted by using the traditional two-dimensional cursor center feature matching algorithm is 1000.7237 mm, and the absolute error reaches 0.2460 mm. For the point cloud stitching algorithm based on the three-dimensional centroid matching of optical fiducial points proposed in this invention, the fitted center distance between spheres is 1000.6204 mm, as shown in the right figure, and the absolute error is reduced to 0.1427 mm. Through quantitative comparison, it can be seen that the method based on three-dimensional feature matching is significantly superior to the traditional two-dimensional method in registration accuracy, and the error reduction amplitude is 0.1033 mm (a decrease of 41.9%), verifying the technical superiority of this method in cross-view three-dimensional stitching.

[0131] In summary, the point cloud stitching technology based on the three-dimensional centroid matching of optical fiducial points proposed in this study has significant advantages compared with the traditional two-dimensional feature registration method: First, automatic matching is achieved by generating three-dimensional feature constraints in the target flat area, which is more convenient and efficient than manually pasting fiducial points. Second, for workpieces with weak textures, the problem of large stitching errors caused by insufficient self-features can be avoided, and the adaptability is stronger. Third, this technology is based on the three-dimensional level, solving the two-dimensional projection deviation caused by the angles of the camera and the projector, and the problem of three-dimensional stitching errors caused by two-dimensional errors generated when extracting two-dimensional feature points. Through experimental verification, its accuracy is better. In scenarios such as three-dimensional inspection of industrial parts and digital reconstruction of cultural relics that require high-precision multi-view fusion, this technology shows broad application prospects.

[0132] The above specific embodiments only describe the preferred embodiments of the present invention, rather than limiting the protection scope of the present invention. Without departing from the design concept and spirit scope of the present invention, various deformations, substitutions, and improvements made by those of ordinary skill in the art to the technical solutions of the present invention based on the written description and drawings provided by the present invention shall all fall within the protection scope of the present invention.

Claims

1. A point cloud stitching method based on three-dimensional centroid matching of optical fiducial points, characterized in that Including the following steps: Step 1: Build a monocular structured light system and a binocular stereo vision system. Make the projector project a set of sinusoidal fringes onto the object to be measured, and the camera synchronously captures the deformed fringes after being modulated by the surface of the object to be measured. Use the Zhang Zhengyou calibration method to calibrate the internal and external parameters of the camera and the projector; Step 2: Based on the deformed fringe image captured by the monocular system, obtain the three-dimensional points of the object through the phase unwrapping and three-dimensional reconstruction algorithms. Randomly generate three-dimensional fiducial points with standard geometric shapes in the target flat area, and record the three-dimensional coordinates; Step 3: Reverse-project the three-dimensional fiducial points into the pixel coordinate system of the projector to generate a two-dimensional encoded pattern with feature points, and project it onto the object surface through the projector; Step 4: Move the binocular stereo vision system to Viewpoint 1, and collect the fringe image and the fiducial point encoded image calculated in Step 3. Then move the binocular system to Viewpoint 2, repeat the above data collection to ensure that there is a common area between the two viewpoints; Step 5: Through the images collected by the binocular system, reconstruct the targets captured at Position 1 and Position 2 respectively, and extract the three-dimensional point clouds of each fiducial area. Use the point cloud density weighted algorithm to calculate the geometric centroid of each three-dimensional fiducial point, and construct the centroid feature point set under the two viewpoints; Step 6: Match the centroid feature points of the two viewpoints through the spatial distribution topological relationship, and use the singular value decomposition algorithm to solve the rotation and translation matrix between the two viewpoints; Step 7: Transform the three-dimensional point cloud of Viewpoint 2 to the global coordinate system of Viewpoint 1 through the rotation and translation matrix obtained in Step 6 to achieve cross-view three-dimensional point cloud stitching.

2. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 1, wherein In Step 1, the projector projects the phase-encoded fringe pattern and the fiducial point pattern onto the object surface, and the industrial camera synchronously captures the deformed fringes and the fiducial point pattern, and completes the system calibration through a circular calibration board.

3. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 1, wherein In Step 2, standard geometric figures of different sizes are divided at random positions in the flat area of the target point cloud as three-dimensional feature regions, and the shapes and topological structures of the three-dimensional regions reconstructed under different viewpoints remain unchanged.

4. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 1, characterized in that In Step 3, the method of reverse projection is as follows: According to the internal and external parameters of the monocular system, reverse-solve the point cloud coordinates in the three-dimensional feature region to the two-dimensional pixel plane of the projector, and the formula is as follows; project the fiducial point pattern in the common area to construct the geometric constraint of the common area under multiple viewpoints; ； ； Among them, is a homography matrix, , are the internal and external parameters of the projector, is the three-dimensional coordinate of the point, is the calculated pixel coordinate of the projector.

5. The point cloud stitching method based on three-dimensional centroid matching of optical landmark points according to claim 1, characterized in that, In Step 4, when the binocular system captures images, use the fiducial point encoded image as a mask, and only perform three-dimensional reconstruction on the point cloud within the fiducial area to ensure that the reconstruction range is consistent with the initially divided three-dimensional feature region.

6. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 1, characterized in that In step five, the geometric centroid of each three-dimensional landmark point is calculated using the point cloud density weighting algorithm , and the centroid feature point sets under two perspectives are constructed; ； ； Among them, is the number of point clouds in the feature region, is the three-dimensional coordinates of the point cloud, is the set composed of the centroids, represents the centroid coordinates of the j-th three-dimensional feature region in the first view, represents the centroid coordinates of the j-th three-dimensional feature region in the second view, and respectively represent the total number of three-dimensional feature regions extracted under the first view and the second view.

7. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 1, characterized in that In Step 6, the centroid matching method is: encode and sort the centroid feature points of the two viewpoints, establish a one-to-one correspondence according to the encoding, and form a set of centroid point pairs; ； Among them, is the total number of centroid matches, is the centroid coordinates fitted from the first perspective under the same three-dimensional region, is the centroid coordinates corresponding to the second perspective of the same feature region.

8. The point cloud stitching method based on three-dimensional centroid matching of optical fiducial points according to claim 7, wherein Construct a covariance matrix using centroid matching point pairs , and use the SVD singular value decomposition method to solve the rigid transformation matrix ; Obtain the rigid transformation matrix, transfer the point clouds of multiple viewpoints to the global coordinate system, and finally realize point cloud stitching; ； ； ； wherein respectively represent the centroid coordinate sets of the central positions, is the left singular vector matrix, providing the main direction information of the target points, is the right singular vector matrix, providing the main direction of the source point cloud, and the diagonal matrix measures the data distribution scale of the point cloud in each main direction, is the rotation matrix, is the translation matrix.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1-8.

10. A computer-readable storage medium storing an executable instruction set, characterized in that, When the instruction is executed by the processor, it implements the method described in any one of claims 1-8.

Images