Point cloud stitching method based on 3D centroid matching of optical landmarks

Abstract

The present invention discloses a point cloud stitching method based on three-dimensional centroid matching of optical marker points. The method reconstructs the three-dimensional shape of an object through a monocular system, randomly generates three-dimensional marker points of standard geometric shapes in the target flat area, and reversely solves them to a two-dimensional projector to generate a marker point pattern and project it onto the object, thereby realizing the constraint of the common area under multiple perspectives. The method uses data collected by the binocular system under different perspectives to reconstruct the three-dimensional point cloud and calculate the geometric centroid of the cursor in the common area as the multi-perspective feature matching point. Based on the invariance of spatial topology, feature point pairs are constructed, and the rigid transformation matrix RT is solved through SVD to achieve high-precision point cloud stitching. The present invention does not require manual pasting of marker points, greatly improving the stitching efficiency. The three-dimensional centroid constraint solves the problem of large errors in weak texture stitching, and has high robustness, thereby realizing high-precision multi-perspective three-dimensional stitching.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a point cloud stitching method based on three-dimensional centroid matching of optical marker points. Background Art

[0002] 3D stitching technology has important applications in industrial inspection, virtual reality, reverse engineering, and the digital preservation of cultural heritage. High-precision 3D stitching is particularly important for integrating multi-view 3D morphological data of large, complex objects. High-precision 3D stitching is a crucial step in point cloud data processing and a key technology for ensuring model integrity and consistency. Fringe Projection Profilometry (FPP) offers advantages such as high precision, high efficiency, non-contact measurement, and convenient data processing. It is a typical surface structured light 3D reconstruction technique. By projecting sinusoidal coded fringes onto an object, these fringes, modulated by the surface topography, are synchronously captured by a camera to produce a deformation fringe pattern containing height information. A temporal phase unwrapping algorithm is then used to calculate the phase information of each pixel in the image. This information establishes a correspondence between the camera imaging plane and the projector pixel coordinate system, enabling submillimeter 3D morphological measurement of the object surface from a single viewpoint.

[0003] However, during the 3D digital reconstruction process, single-view scanning often struggles to capture complete point cloud data due to object self-occlusion and the resolution limitations of the measurement system. The resulting 3D data only contains partial information about the object's surface. Therefore, to obtain the object's complete 3D shape, a multi-view data acquisition strategy is necessary: ​​first, the device's position is adjusted to acquire a point cloud dataset covering all directions of the object. Then, a registration algorithm is used to unify the discrete point clouds into a common world coordinate system.

[0004] Traditional multi-view stitching methods typically consist of two stages: initial registration and fine registration. Initial registration aims to achieve coarse alignment and minimize rotational and translational errors between point clouds. Common methods include the calibration point method (artificially attaching high-contrast markers to the object surface for registration), object feature extraction, and projected marker methods. However, these methods have significant limitations: the calibration point method is complex, requires manual intervention, and has poor environmental adaptability, while the feature extraction method is susceptible to point cloud sparsity, noise, and low-texture areas, resulting in insufficient robustness. Furthermore, marker point registration methods often rely on the extraction and matching of 2D projected features. These methods use a projector to project a 2D coded pattern (such as a circular marker) onto the object surface. A camera captures images from different viewpoints and extracts the coordinates of the circle's center as matching feature points. The rotation and translation matrix (R|T) is then calculated to complete point cloud stitching. However, when the projector and camera have different viewpoints, the physical plane of the circle in space is at an angle to the camera's imaging plane. Consequently, the projected image is an ellipse, and the geometric center of the ellipse deviates from the true projection point of the 3D circle center. Under oblique projection conditions, the ellipse center offset Δp is positively correlated with the projection angle, leading to errors in 2D center extraction. Consequently, 2D deviations can occur during the stitching process. While existing methods, such as iterative virtual calibration and concentric circle constraints, can partially compensate for this eccentricity, they still rely on complex optical correction models and multi-view data fusion, resulting in high computational complexity.

[0005] During the fine registration phase, the Iterative Closest Point (ICP) algorithm is a classic, simple and intuitive method. However, it is highly dependent on the accuracy of the initial registration, prone to local optima, and sensitive to changes in point cloud topology and noise. To overcome ICP's shortcomings, researchers have proposed improved algorithms such as the Normal Distribution Transform (NDT), Contrasting Projection Points (CCP), and Coherent Point Drift (CPD). However, these methods still suffer from high computational complexity and poor adaptability to uneven point cloud density. Summary of the Invention

[0006] The purpose of this invention is to provide a point cloud stitching method based on 3D centroid matching of optical marker points. This method uses a monocular structured light system to reconstruct the 3D topography of the target surface. Standard geometric shapes are randomly generated within the target's flat area as 3D marker areas, and their geometric centroids are extracted as cross-view matching points, eliminating the 2D deviation introduced by 2D projection deformation. This method, based on 3D properties, overcomes the problem of 3D point cloud stitching errors caused by deformation errors between the projector and camera optical axes and the target plane in traditional 2D cursors. This method solves the significant problem of weak texture stitching errors, effectively improves stitching accuracy, and achieves high-precision, multi-view 3D stitching.

[0007] The technical solution for achieving the purpose of the present invention is: a point cloud stitching method based on three-dimensional centroid matching of optical landmark points, comprising the following steps:

[0008] Step 1: Build a monocular structured light system and a binocular stereo vision system. The projector projects a set of sinusoidal stripes onto the object to be measured, and the camera simultaneously captures the deformation stripes modulated by the surface of the object to be measured. Use the Zhang Zhengyou calibration method to calibrate the camera and projector for internal and external parameters.

[0009] Step 2: Based on the deformed fringe image captured by the monocular system, the 3D points of the object are obtained through phase solution and 3D reconstruction algorithms. 3D markers of standard geometric shapes are randomly generated in the target flat area and their 3D coordinates are recorded.

[0010] Step 3: Reversely project the 3D marker points to the projector pixel coordinate system to generate a 2D coded pattern with feature points, and project it onto the object surface through the projector;

[0011] Step 4: Move the binocular stereo vision system to view angle 1 and collect the fringe pattern and the marker point coded image calculated in step 3. Then move the binocular system to view angle 2 and repeat the above data collection to ensure that there is a common area between the two view angles.

[0012] Step 5: Using the images captured by the binocular system, reconstruct the targets captured at position 1 and position 2 respectively, and extract the 3D point cloud of each landmark area. Use the point cloud density weighting algorithm to calculate the geometric centroid of each 3D landmark point and construct the centroid feature point set under the two perspectives.

[0013] Step 6: Match the centroid feature points of the two views through the spatial distribution topological relationship, and use the singular value decomposition algorithm to solve the rotation and translation matrix between the two views;

[0014] Step 7: Transform the 3D point cloud of view 2 to the global coordinate system of view 1 using the rotation and translation matrix obtained in step 6 to achieve cross-view 3D point cloud stitching.

[0015] In the step 1, the projector projects the phase-encoded fringe pattern and the marker point pattern onto the surface of the object, the industrial camera synchronously captures the deformation fringe and the marker point pattern, and the system calibration is completed through the circular calibration plate.

[0016] Furthermore, in the step 2, standard geometric figures of different sizes are divided at random positions in the flat area of ​​the target point cloud as three-dimensional feature areas, and the reconstructed shape and topological structure of the three-dimensional area remain unchanged under different viewing angles.

[0017] Furthermore, in step 3, the reverse projection method is as follows: based on the intrinsic parameters and extrinsic parameters of the monocular system, the point cloud coordinates in the three-dimensional feature area are inversely resolved to the two-dimensional pixel plane of the projector, as shown in the following formula; a marker point pattern is projected in the common area to construct the geometric constraints of the common area under multiple perspectives;

[0018] ;

[0019] ;

[0020] in, is the homography matrix, 、 are the internal and external parameters of the projector, are the three-dimensional coordinates of the point, are the calculated projector pixel coordinates.

[0021] Furthermore, in step 4, when the binocular system captures the image, the marker point coding map is used as a mask to perform three-dimensional reconstruction only on the point cloud within the marker area, ensuring that the reconstruction range is consistent with the initially divided three-dimensional feature area.

[0022] Furthermore, in step 5, the geometric centroid of each 3D marker point is calculated using a point cloud density weighted algorithm. , construct the centroid feature point set under two perspectives;

[0023] ;

[0024] ;

[0025] in, is the number of point clouds in the feature area, is the three-dimensional coordinate of the point cloud, is the set of centroids, represents the centroid coordinates of the j-th 3D feature region in view 1, represents the centroid coordinates of the j-th 3D feature area in view 2, and Represent the total number of 3D feature regions extracted under view 1 and view 2, respectively.

[0026] Furthermore, in step 6, the specific centroid matching method is: encoding and sorting the centroid feature points of the two perspectives, establishing a one-to-one correspondence based on the encoding, and forming a centroid point pair set;

[0027]

[0028] in, is the total number of centroid matches, is the centroid coordinate of the first viewing angle fitting in the same three-dimensional region, is the centroid coordinate corresponding to the second viewing angle of the same feature area.

[0029] Furthermore, we first construct the covariance matrix using the centroid matching point pairs , use SVD singular value decomposition method to solve the rigid transformation matrix ; Obtain the rigid change matrix, transfer the multi-view point cloud to the global coordinate system, and finally realize point cloud stitching;

[0030]

[0031]

[0032]

[0033] in Represents the centroid coordinate set central location, is the left singular vector matrix, providing the main direction information of the target point, Is the right singular vector matrix, providing the main direction of the source point cloud, diagonal matrix To measure the data distribution scale of the point cloud in each main direction, is the rotation matrix, is the translation matrix.

[0034] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the program.

[0035] A computer-readable storage medium has program codes corresponding to the executable instruction set, and the method is implemented when the program is executed by a processor.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) Based on the three-dimensional marker reverse mapping method, the present invention reversely decodes the three-dimensional feature area to the two-dimensional projector plane to generate a marker point pattern, and uses the projector to project the marker point pattern onto the object, effectively solving the problem of large splicing errors caused by missing features of weak-textured objects; compared with the traditional manual marker point pasting method, this method does not require the use of physical markers, which significantly improves the ease of operation; at the same time, the projector can change the position and size of the marker points in real time according to the three-dimensional morphology of the object itself, expanding the application scenarios and significantly improving the environmental adaptability of this method.

[0038] (2) Based on the spatial invariance of the 3D region centroid, a mapping relationship between multiple perspectives is established, which solves the pose calculation error caused by the angle between the projector and camera optical axis and the target plane in traditional 2D cursor feature matching. Compared with traditional 2D cursor feature matching methods (such as SIFT and ORB), this method can improve the registration accuracy to sub-millimeter level by using 3D geometric centroid constraints, and does not require the use of ICP fine registration algorithm, which significantly improves computational efficiency.

[0039] (3) The centroid coordinates are calculated based on the weighted density of the regional point cloud. They are highly resistant to noise and insensitive to changes in viewing angles, which significantly improves the robustness of the registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a diagram of the circular cursor projection deviation model.

[0041] Figure 2 This is a schematic diagram of the point cloud stitching technology based on three-dimensional centroid matching of optical marker points proposed in the present invention, wherein the upper figure is the inverse solution of the three-dimensional feature area to the two-dimensional projector, and the lower figure is a schematic diagram of the stitching process.

[0042] Figure 3 Schematic diagram of the process of the present invention.

[0043] Figure 4 Schematic diagram of the experimental setup.

[0044] Figure 5 The result of inverse solution of three-dimensional feature area to two-dimensional projector; the left picture shows the random generation of standard geometric three-dimensional graphic marker points (such as circle, triangle or rectangle) in the flat area of ​​the workpiece, and the right picture shows the reverse projection of the three-dimensional area marker points to the projector two-dimensional image.

[0045] Figure 6 These are the marker point images taken from two viewing angles respectively; the left image is the marker point coding image taken from viewing angle one, and the right image is the marker point coding image taken from viewing angle two.

[0046] Figure 7 The point cloud of the three-dimensional feature area and the geometric centroid of each area are reconstructed from the perspective.

[0047] Figure 8 The point cloud of the three-dimensional feature area and the geometric centroid of each area are reconstructed under the second perspective.

[0048] Figure 9 This is the stitching result of the method of the present invention; the left picture is the three-dimensional point cloud of the object at the first viewing angle, the middle picture is the three-dimensional point cloud of the object at the second viewing angle, and the right picture is the final point cloud stitching effect.

[0049] Figure 10 These are the three-dimensional stitching results of the wall using the traditional two-dimensional feature stitching method and the method of the present invention; the left picture shows the effect of the wall stitching using the traditional two-dimensional cursor circle center feature stitching method, and the right picture shows the effect of the wall stitching using the present invention.

[0050] Figure 11These are the three-dimensional stitching results of the 1000.4777 standard double-ball workpiece using the traditional two-dimensional feature stitching method and the method of the present invention; the left picture shows the result of stitching the standard double-ball workpiece using the traditional two-dimensional cursor circle center feature stitching method, and the right picture shows the result of stitching the standard double-ball workpiece using the method of the present invention. DETAILED DESCRIPTION

[0051] In response to the problems of strong dependence on manual marking, insufficient registration robustness and high computational complexity in multi-view 3D stitching, the present invention proposes a point cloud stitching technology based on 3D centroid matching of optical landmarks, abandoning the traditional 2D feature dependence and directly constructing the registration constraints from the geometric properties of the 3D point cloud. By extracting the geometric centroid of the local 3D area in the multi-view point cloud as the feature matching point, cross-view point cloud stitching is achieved. This method breaks through the problem of 3D point cloud stitching error caused by deformation error between the projector, camera optical axis and target plane of the traditional 2D cursor, and significantly reduces the pose solution error. In addition, without the need for fine registration algorithms such as ICP, this method achieves sub-millimeter stitching accuracy and reduces computational complexity. Experimental results show that the proposed method can accurately achieve cross-view point cloud stitching in low-texture, large-scale and complex curved surface scenes.

[0052] The present invention provides a point cloud stitching method based on three-dimensional centroid matching of optical marker points, comprising the following steps:

[0053] Step 1: Build a monocular structured light system and a binocular stereo vision system. Use either camera in the binocular system as the camera of the monocular system. The projector projects a set of sinusoidal stripes onto the object to be measured. The camera simultaneously captures the deformation stripes after being modulated by the surface of the object to be measured. Use Zhang Zhengyou's calibration method to calibrate the camera and projector for internal and external parameters.

[0054] Step 2: Based on the deformed fringe image captured by the monocular system, the 3D points of the object are obtained through phase solution and 3D reconstruction algorithms. 3D marker points of standard geometric shapes are randomly generated in the target flat area and the 3D coordinates are recorded.

[0055] Step 3: Reversely project the three-dimensional marker points to the projector pixel coordinate system to generate a two-dimensional coding pattern with feature points, and project it onto the surface of the object through the projector.

[0056] Step 4: Move the binocular stereo vision system to view one and collect the fringe pattern and the marker point coded image calculated in step 3. Then move the binocular system to view two and repeat the above data collection to ensure that there is a common area between the two viewpoints.

[0057] Step 5: Using the images captured by the binocular system, reconstruct the targets captured at positions 1 and 2, extracting a 3D point cloud for each landmark area. Use a point cloud density weighting algorithm to calculate the geometric centroid of each 3D landmark point and construct a set of centroid feature points from both perspectives.

[0058] Step 6: Match the centroid feature points of the two perspectives through the spatial distribution topological relationship, and use the singular value decomposition (SVD) algorithm to solve the rotation and translation matrix between the two perspectives.

[0059] Step 7: Transform the 3D point cloud of view 2 to the global coordinate system of view 1 using the rotation and translation matrix obtained in step 6 to achieve cross-view 3D point cloud stitching.

[0060] Furthermore, the system in step 1 is composed of a projector, an industrial camera, and a circular calibration plate. The projector projects the phase-encoded fringe pattern and the marker dot pattern onto the surface of the object, and the industrial camera simultaneously captures the deformation fringe and marker dot pattern, and the circular calibration plate is used to complete the system calibration.

[0061] Furthermore, no matter at what angle the 3D measurement system is used to shoot, the flat area of ​​the object in the monocularly reconstructed point cloud is randomly divided into 3D feature areas of standard geometric shapes of different sizes. The reconstructed 3D areas have shape consistency and their topological structure remains unchanged from the original divided shapes.

[0062] Furthermore, reverse projection mapping technology can be used to back-project the divided 3D feature areas onto the projector's 2D pixel plane. A marker point coding map is projected onto the common area between the two viewpoints, allowing the system to capture uniquely coded feature area images and fringe patterns from each viewpoint, thereby constructing geometric constraints for the common area under multiple viewpoints.

[0063] Furthermore, the calibrated binocular system is used to reconstruct the workpiece photographed from different perspectives, and the feature landmark image taken in the previous step is used as a mask to perform three-dimensional reconstruction only on the point cloud within the landmark area, ensuring that the reconstruction range is consistent with the initially divided three-dimensional feature area.

[0064] Furthermore, the geometric centroids of each three-dimensional landmark area under two viewing angles are extracted and encoded, and a centroid correspondence relationship is established based on the encoding to construct a centroid point pair coordinate set.

[0065] Furthermore, based on the set of matching centroid point pairs, the singular value decomposition algorithm is used to solve the optimal rotation matrix R and translation vector T, and finally the two-view point cloud is transformed into the global coordinate system, thereby realizing multi-view point cloud stitching.

[0066] The method of the present invention is based on the invariance of the geometric centroid of the three-dimensional region. It reversely maps the three-dimensional marker features to the projection plane through the monocular system to generate feature marker points. The direct use of the invariance of the three-dimensional region centroid to construct constraints between multiple perspectives breaks through the problem of three-dimensional point cloud splicing errors caused by the deformation error of the traditional two-dimensional cursor due to the angle between the projector, camera optical axis and the target plane, and can achieve high-precision reconstruction of three-dimensional point clouds of weakly textured objects. This method uses the DBSCAN clustering algorithm to segment the feature three-dimensional region and extract the centroid of each region as the point cloud splicing feature point. There is no need to rely on manual pasting of marker points and ICP iterative algorithm, which significantly reduces the computational complexity. At the same time, it improves the robustness of registration in complex scenes such as low texture and large scale, and effectively expands the application scenarios of traditional feature matching algorithms. Experimental results show that this method can achieve high-precision seamless splicing of three-dimensional point clouds across perspectives.

[0067] The technical solution of the present invention is described in detail below with reference to the embodiments.

[0068] Example

[0069] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0070] A. The influence of 2D projection deviation on 3D stitching accuracy

[0071] In the process of cursor projection stitching, the two-dimensional projection deviation is an important factor affecting the accuracy of three-dimensional stitching. When the camera or projector is tilted at an angle to the target object, the projector projects a standard circular cursor on the object. At this time, the projection of the standard circle on the imaging plane becomes an ellipse, and the center of the ellipse is not the projection image point of the actual circle center. If the corresponding three-dimensional point is directly calculated using the center of the ellipse as the stitching feature point, it will cause the three-dimensional rigid body change matrix to be Deviations in calculations ultimately lead to stratification and misalignment in the stitching area, seriously affecting the accuracy and effect of three-dimensional stitching.

[0072] The error comes from the non-orthogonality of perspective projection. Take the oblique cone projection model as an example, Figure 1 As shown in the figure, the projection of a circle in the camera coordinate system can be viewed as a section of an oblique cone. There is a deviation between the center of the ellipse and the projection of the true center of the circle. The deviation can be expressed by the following formula:

[0073]

[0074] in Respectively represent the ellipse center and the actual circle center axis Deviation on the axis, is the object distance, is the circle radius, is the angle between the camera optical axis and the imaging plane, is the coordinate of the center of the space circle .

[0075] The above deviations lead to the rigid transformation matrix There are errors in the calculation of the matrix, which in turn affects the 3D stitching error. If in a binocular system, both the left and right cameras use the ellipse center as the feature point, the error of the 3D point reconstruction is:

[0076]

[0077] Therefore, based on traditional 2D cursor center feature matching methods, this kind of stitching will gradually accumulate in multi-view stitching, resulting in layered or distorted 3D models. This invention uses the 3D region centroid as a feature point, directly starting from a 3D perspective to effectively avoid the 2D projection deviation problem, and achieve higher accuracy.

[0078] B. Measurement system mathematical model and calibration

[0079] like Figure 2 、 Figure 3 As shown, an example of a point cloud stitching method based on three-dimensional centroid matching of optical marker points includes the following steps:

[0080] Step 1: Use a projector to project a set of phase-shifted sinusoidal stripes, and use a camera to synchronously capture them. Furthermore, the stripe expression captured by the camera is:

[0081]

[0082] in represents the number of phase shift steps, is the fringe image captured by the camera, represents the pixel coordinates of the camera, and They are respectively represented by the background light intensity and the sinusoidal fringe modulation degree, Indicates the phase information that needs to be solved, is the fringe phase shift.

[0083]

[0084] Solving the above inverse tangent function yields The wrapped phase of The time phase unwrapping algorithm is used to solve the phase discontinuity problem and obtain the required absolute phase by the following method:

[0085]

[0086] in It is the stripe level.

[0087] Calibration of the measurement system: The traditional camera model is a pinhole imaging model. The object surface reaches the camera imaging surface through four coordinate systems: world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system. Its world coordinate system is With pixel coordinates The relationship between them is shown in the following formula:

[0088]

[0089] in is the normalization coefficient, The focal length is The direction component, is the unit pixel size, is the pixel coordinate of the optical center, represents the rotation and translation matrix, is the internal parameter matrix, is the external parameter matrix.

[0090] The projector model is the same as the camera model. The projector projects structured light stripes in the horizontal and vertical directions on the calibration plate respectively. Phase recovery and phase unwrapping are performed on the two sets of stripe images collected, and then the absolute phase distribution maps in the two directions are obtained, which are expressed as and , which is the reverse process of camera shooting. However, unlike the camera which can directly obtain pixel coordinates, it is necessary to use phase information to calculate the projector pixel coordinates. Suppose the fringe frequencies of the projector in the horizontal and vertical directions are and , the resolution of the projector in two directions is and Its corresponding position coordinates in the projector image pixel coordinate system can be calculated by the following formula:

[0091]

[0092] in is the camera pixel coordinate, are the calculated projector pixel coordinates.

[0093] The camera is calibrated based on Zhang Zhengyou's calibration method. Circular checkerboards are placed at different positions in the field of view. The center coordinates of the checkerboards at different positions are obtained by the circle center extraction method. The center coordinates of the checkerboards and the world coordinates are substituted into the above formula to obtain the homography matrix. ,in It is called the internal parameter matrix. is the external parameter matrix.

[0094]

[0095] In stereo vision, when the target points in the three-dimensional space are synchronously collected by the device, the two-dimensional projection coordinates formed by the target points on different imaging planes are and Combined with the device calibration parameters, we can obtain the 3D data of the point and complete the 3D reconstruction at that perspective. We can list the equation as follows:

[0096]

[0097]

[0098] in , are the normalization coefficients of the camera and projector respectively, are the camera’s intrinsic and extrinsic parameters, are the internal and external parameters of the projector.

[0099] Further combining the equations, we can obtain:

[0100]

[0101] The above equation contains four equations and three unknowns , so the least squares method can be used to solve the system of equations to obtain the optimal solution.

[0102] C. Point cloud stitching based on 3D centroid matching of optical landmarks

[0103] Step 2: Analyze the 3D shape of the object reconstructed by the monocular system, and randomly divide the 3D regions of different sizes and geometric standard shapes at different positions in the flat area of ​​the object point cloud, such as Figure 5 As shown in the left figure in . Based on the 3D region invariance, that is, the 3D region reconstructed at different angles remains unchanged, constraints between multiple perspectives are constructed.

[0104] Step 3: If Figure 5 As shown in the right figure, through the inverse operation of three-dimensional measurement, the formula is as follows. The divided three-dimensional area can be inversely solved to the two-dimensional projector, and the landmark point coding map can be projected in the common area of ​​the two perspectives to realize the constraint of the common area under multiple perspectives and provide feature constraints for the stitching of the two perspectives.

[0105]

[0106] Step 4: Project the image of the marker point features containing the coded information obtained in step 3, and use the binocular 3D measurement system to shoot at different viewing angles. The first viewing angle refers to the measurement position when the projector and camera are in the initial measurement position; the second viewing angle represents the new measurement position formed after the system moves. At this time, the camera and projector have moved to the new position, and ensure that there is a certain common area between the two measurements to meet the feature matching requirements of the subsequent splicing. Obtain the fringe image and the marker point feature image at different viewing angles, such as Figure 6 shown.

[0107] Step 5: Perform 3D reconstruction based on the fringe patterns taken at different viewing angles, and use Figure 6 As a mask, ensure that 3D reconstruction is performed only within the circular cursor area, and solve the 3D mark area captured at different viewing angles. The reconstructed 3D feature area is segmented using the density clustering algorithm (DBSCAN), and the point cloud is divided into multiple independent sub-areas. The centroid coordinates of each area of ​​the 3D feature area are calculated using the following formula ( , , ), construct the centroid coordinate set , and visualized on the image.

[0108]

[0109]

[0110] in, is the number of point clouds in the feature area, is the three-dimensional coordinate of the point cloud, is the set of centroids, represents the centroid coordinates of the j-th 3D feature region in view 1, represents the centroid coordinates of the j-th 3D feature area in view 2, and Represent the total number of 3D feature regions extracted under view 1 and view 2, respectively.

[0111] Step 6: According to Figure 7 、 Figure 8 ,These 3D regions are encoded according to a certain rule, and the feature regions of the two perspectives can be matched according to the encoding, and a one-to-one correspondence is established based on the encoding to form a set of centroid point pairs.

[0112]

[0113] in, is the total number of centroid matches, is the centroid coordinate of the first viewing angle fitting in the same three-dimensional region, is the centroid coordinate corresponding to the second viewing angle of the same feature area.

[0114] We can further calculate the transformation relationship between the two perspectives based on the centroid point pair obtained above. Here we only consider rigid transformations, that is, the shape of the data under the two perspectives remains unchanged, there is no scaling transformation, and only rotation and translation transformations are considered.

[0115]

[0116] in, For around The angle of axis rotation, For around The angle of axis rotation, For around The angle of axis rotation, is the translation matrix The components on each coordinate axis, is the rotation matrix, ( ) is the coordinate value of the point cloud after rotation and translation.

[0117] Based on the matched centroid point pair set, the error function is constructed and calculated. The matrix is ​​equivalent to solving the minimum value of the error function.

[0118]

[0119] Use SVD singular value decomposition method to solve, first construct the covariance matrix , use linear algebra knowledge to perform SVD decomposition and then solve .

[0120]

[0121]

[0122]

[0123] in Represents the centroid coordinate set central location, is the left singular vector matrix, providing the main direction information of the target point, Is the right singular vector matrix, providing the main direction of the source point cloud, diagonal matrix It is used to measure the data distribution scale of the point cloud in each main direction.

[0124] Step 7: Based on the results from step 6 Matrix, the overall three-dimensional point cloud of view 2 Transform to the global coordinate system , and finally realize cross-view 3D point cloud stitching, such as Figure 9 Shown is the final stitching effect.

[0125]

[0126] D. Experiment

[0127] In order to verify the effect of the present invention, a three-dimensional point cloud stitching measurement system was built, which includes a three-dimensional measurement device and a marker point projection device. The experiment uses a stripe projection binocular stereo vision matching system, including a DMD projector (DLP6500 with a resolution of 1920*1200), two CMOS cameras (Basler acA1920-155 um, with a resolution of 1920×1200 pixels), and a camera lens with a focal length of 12mm. For the marker point projection device, we use a HD301A1-H2 micro-electromechanical system (MEMS) projector with a resolution of 800*800 pixels. The stripes used in the experiment are , , generating an 8-step phase shift pattern. The object to be measured is placed at a position of about 0.8m from the binocular measurement system. The experimental setup is as follows Figure 4 As shown. First, it is placed in the middle of the object and reconstructed using a monocular measurement system constructed using a MEMS projector and a camera in the binocular system. A three-dimensional feature constraint graph is generated and inversely translated to a two-dimensional MEMS projector. The binocular measurement system is then placed at viewpoints one and two for data acquisition, including fringe patterns and landmark feature maps. Viewpoint one refers to the observation angle when the projector and camera are in the initial measurement position, and viewpoint two refers to the observation angle formed after the system moves to the new measurement position. The two measurements must ensure that there is a certain common area to meet the feature matching requirements in the subsequent stitching process. Finally, the RT matrix is ​​calculated based on the constraints to achieve three-dimensional point cloud stitching.

[0128] We further tested the performance of our proposed point cloud stitching technique based on 3D centroid matching of optical landmarks through experiments. We measured a wall and a standard double-sphere workpiece, and used the error between the center distance of the standard double-sphere workpiece and the actual center distance as the stitching error accuracy assessment.

[0129] Figure 10The experimental results of two point cloud stitching methods for wall stitching are compared. The left image shows the stitching effect of traditional 2D cursor circle center feature matching. Although it can achieve preliminary position alignment, there are obvious layered misalignment and "wavy" errors at the stitching boundary. The right image shows the point cloud stitching result based on 3D centroid matching of optical marker points. This method does not show any stratification in the stitching transition area, and the dual-view point clouds achieve seamless fusion in terms of spatial continuity and geometric consistency. The experimental results show that the point cloud stitching method based on 3D centroid matching of optical marker points proposed in this study has significant advantages in stitching accuracy and surface continuity.

[0130] Figure 11 The results of a comparative experiment comparing the stitching accuracy of a standard double-sphere workpiece (theoretical sphere center distance of 1000.4777 mm) are presented. The experimental data shows that the sphere center distance fitted using a traditional 2D cursor circle center feature matching algorithm is 1000.7237 mm, with an absolute error of 0.2460 mm. In contrast, the proposed point cloud stitching algorithm based on 3D centroid matching of optical landmarks achieves a sphere center distance of 1000.6204 mm, as shown in the right figure, with an absolute error reduced to 0.1427 mm. A quantitative comparison shows that the 3D feature matching-based method significantly outperforms the traditional 2D method in registration accuracy, reducing the error by 0.1033 mm (a 41.9%), validating the method's technical superiority in cross-view 3D stitching.

[0131] In summary, the point cloud stitching technology based on 3D centroid matching of optical landmarks proposed in this study has significant advantages over traditional 2D feature registration methods: First, it achieves automatic matching by generating 3D feature constraints within the target flat area, making it simpler and more efficient than manually pasting landmarks. Second, for workpieces with weak textures, it avoids large stitching errors caused by insufficient native features, making it more adaptable. Third, based on the 3D layer, this technology addresses the 2D projection deviation caused by the angle between the camera and the projector, as well as the 2D error generated when extracting 2D feature points, which in turn causes 3D stitching errors. Experimental verification shows that it has superior accuracy. This technology shows broad application prospects in scenarios requiring high-precision multi-view fusion, such as 3D inspection of industrial parts and digital reconstruction of cultural relics.

[0132] The above specific embodiments merely describe preferred embodiments of the present invention and do not limit the scope of protection of the present invention. Without departing from the design concept and spirit of the present invention, various modifications, substitutions, and improvements made by those skilled in the art to the technical solution of the present invention based on the text description and drawings provided herein shall fall within the scope of protection of the present invention.

Claims

1. A point cloud stitching method based on three-dimensional centroid matching of optical landmarks, characterized in that: The following steps are involved: Step 1: Build a monocular structured light system and a binocular stereo vision system. The projector projects a set of sinusoidal stripes onto the object to be measured, and the camera simultaneously captures the deformation stripes modulated by the surface of the object to be measured. Use the Zhang Zhengyou calibration method to calibrate the camera and projector for internal and external parameters. Step 2: Based on the deformed fringe image captured by the monocular system, the 3D points of the object are obtained through phase solution and 3D reconstruction algorithms. 3D markers of standard geometric shapes are randomly generated in the target flat area and their 3D coordinates are recorded. Step 3: Reversely project the 3D marker points to the projector pixel coordinate system to generate a 2D coded pattern with feature points, and project it onto the object surface through the projector; Step 4: Move the binocular stereo vision system to view angle 1 and collect the fringe pattern and the marker point coded image calculated in step 3. Then move the binocular system to view angle 2 and repeat the above data collection to ensure that there is a common area between the two view angles. Step 5: Using the images captured by the binocular system, reconstruct the targets captured at position 1 and position 2 respectively, and extract the 3D point cloud of each landmark area. Use the point cloud density weighting algorithm to calculate the geometric centroid of each 3D landmark point and construct the centroid feature point set under the two perspectives. Step 6: Match the centroid feature points of the two views through the spatial distribution topological relationship, and use the singular value decomposition algorithm to solve the rotation and translation matrix between the two views; Step 7: Transform the 3D point cloud of view 2 to the global coordinate system of view 1 using the rotation and translation matrix obtained in step 6 to achieve cross-view 3D point cloud stitching.

2. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 1, the projector projects the phase-encoded fringe pattern and marker point pattern onto the surface of the object, and the industrial camera simultaneously captures the deformed fringe and marker point pattern, completing the system calibration using a circular calibration plate.

3. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 2, standard geometric shapes of different sizes are divided at random positions in the flat area of ​​the target point cloud as three-dimensional feature regions, and the reconstructed shape and topological structure of the three-dimensional region remain unchanged under different viewing angles.

4. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 3, the reverse projection method is as follows: based on the intrinsic and extrinsic parameters of the monocular system, the point cloud coordinates in the 3D feature area are inversely resolved to the 2D pixel plane of the projector. The formula is shown below; a pattern of marker points is projected into the common area to construct the geometric constraints of the common area under multiple views. ； ； in, is the homography matrix, 、 are the internal and external parameters of the projector, is the three-dimensional coordinate of the point, are the calculated projector pixel coordinates.

5. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 4, when the binocular system captures the image, it uses the marker point coding map as a mask to perform three-dimensional reconstruction only on the point cloud within the marker area, ensuring that the reconstruction range is consistent with the initially divided three-dimensional feature area.

6. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 5, the point cloud density weighted algorithm is used to calculate the geometric centroid of each 3D landmark point. , construct the centroid feature point set under two perspectives; ； ； in, is the number of point clouds in the feature area, is the three-dimensional coordinate of the point cloud, is the set of centroids, represents the centroid coordinates of the j-th 3D feature region in view 1, represents the centroid coordinates of the j-th 3D feature area in view 2, and Represent the total number of 3D feature regions extracted under view 1 and view 2, respectively.

7. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 1, characterized in that: In step 6, the centroid matching method is as follows: the centroid feature points of the two perspectives are coded and sorted, and a one-to-one correspondence is established according to the coding to form a centroid point pair set; ； in, is the total number of centroid matches, is the centroid coordinate of the first viewing angle fitting in the same three-dimensional region, is the centroid coordinate corresponding to the second viewing angle of the same feature area.

8. The point cloud stitching method based on three-dimensional centroid matching of optical marker points according to claim 7, characterized in that: Constructing the covariance matrix using centroid matching point pairs , use SVD singular value decomposition method to solve the rigid transformation matrix ; Obtain the rigid change matrix, transfer the multi-view point cloud to the global coordinate system, and finally realize point cloud stitching; ； ； ； in Represents the centroid coordinate set central location, is the left singular vector matrix, providing the main direction information of the target point, Is the right singular vector matrix, providing the main direction of the source point cloud, diagonal matrix To measure the data distribution scale of the point cloud in each main direction, is the rotation matrix, is the translation matrix.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing an executable instruction set, characterized in that: When the instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images