Point cloud splicing method based on digital speckle target plate constraint

Abstract

This invention discloses a point cloud stitching method based on digital speckle target plate constraints, comprising: acquiring modulated speckle images of a target plate from different viewpoints using a binocular or multi-view stereo vision system; performing sub-pixel-level matching on the speckle images of the target plate from multiple viewpoints to establish a two-dimensional pixel correspondence; back-projecting the two-dimensional corresponding points to three-dimensional space to reconstruct a set of three-dimensional corresponding point pairs on the target plate surface; solving the rigid body transformation matrix between viewpoints using the least squares criterion through singular value decomposition; and unifying the workpiece point clouds independently reconstructed by a structured light system from each viewpoint to the global coordinate system according to the obtained transformation matrix to achieve complete and seamless three-dimensional stitching. This invention significantly improves adaptability and robustness in extreme industrial scenarios such as roughness, weak texture, and even no texture by introducing an external stable benchmark; it avoids feature point drift and mismatch caused by surface undulations and uneven reflection, ensuring registration accuracy.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a point cloud stitching method based on digital speckle target constraints. Background Technology

[0002] In fields such as industrial inspection, reverse engineering, and intelligent manufacturing, obtaining complete and accurate three-dimensional topographic data of large and complex workpieces is crucial. Fringe projection profilometry (FPP), as a high-precision, high-efficiency, non-contact surface structured light three-dimensional measurement technique, has been widely used. However, limited by the field of view of a single measurement, it is necessary to obtain complete three-dimensional point cloud data of the workpiece through multi-view scanning and stitching.

[0003] Traditional multi-view point cloud stitching methods typically rely on features extracted from the object's surface for registration, mainly categorized into those based on manual labeling, those based on the object's own features, and those based on projection-encoded features. These methods perform well with smooth, textured objects, but in real-world industrial settings, they often encounter workpieces with extremely rough surfaces, uneven reflections, or no texture at all, such as those from welding, casting, or sandblasting processes. For such workpieces, traditional methods face fundamental challenges: 1) Surface roughness and geometric undulations severely degrade the quality of the 3D reconstructed point cloud in the structured light projection area, resulting in numerous missing features, noise, and uneven distribution, causing surface features calculated from this area to drift and leading to a loss of registration accuracy; 2) The lack of stable, repeatable visual or geometric features leads to feature matching failures. Existing improvements often focus on optimizing feature extraction algorithms or introducing complex iterative optimization processes, but they fail to fundamentally eliminate the dependence on workpiece surface quality, and still suffer from insufficient robustness and difficulty in guaranteeing accuracy under extremely rough surface conditions.

[0004] Furthermore, the Iterative Closest Point (ICP) and its variants, which are widely used in the fine registration stage, are highly dependent on the accuracy of the initial registration and are sensitive to point cloud noise and initial pose. They are prone to getting trapped in local optima when the point cloud quality is poor, and their computational complexity is also high. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies that rely on workpiece surface features for point cloud stitching, and to provide a point cloud stitching method based on digital speckle target plate constraints. This method introduces a fixed external speckle target plate as a stable spatial reference, and transfers the feature matching process on which registration depends from the rough and varied workpiece surface to the stable and reliable target plate surface, thereby achieving automated point cloud stitching that can maintain high accuracy and strong robustness even under surface roughness, weak texture, or even no texture conditions.

[0006] To achieve the above objectives, this invention provides a point cloud stitching method based on digital speckle target constraints, comprising the following steps:

[0007] Step 1: Construct a stereo vision system consisting of at least two cameras, calibrate the stereo vision system to obtain the intrinsic parameters of the cameras and the extrinsic parameters between the cameras; fix a digital speckle target plate with a random speckle pattern in the measurement field of view near the workpiece to be measured.

[0008] Step 2: Place the stereo vision system in the first measurement viewpoint. With the digital speckle target plate placed in the common field of view, simultaneously acquire speckle images of the digital speckle target plate from the left and right cameras in the first measurement viewpoint. Then remove or avoid the digital speckle target plate, and use structured light projection to acquire and reconstruct the local three-dimensional point cloud of the workpiece in the first measurement viewpoint.

[0009] Step 3: Move the stereo vision system to the second measurement viewpoint, repeat step 2, and simultaneously acquire speckle images of the same stationary digital speckle target plate from the left and right cameras under the second measurement viewpoint, and reconstruct the local three-dimensional point cloud of the workpiece under the second measurement viewpoint.

[0010] Step 4: Based on digital image correlation technology, perform sub-pixel level matching on the left camera image of the first measurement viewpoint, the right camera image of the first measurement viewpoint, the left camera image of the second measurement viewpoint, and the right camera image of the second measurement viewpoint to establish a two-dimensional pixel correspondence relationship across viewpoints.

[0011] Step 5: Combining the system calibration parameters obtained in Step 1, the corresponding points of the two-dimensional pixels obtained in Step 4 are back-projected to the three-dimensional space to reconstruct the three-dimensional coordinate point set of the same physical point on the surface of the digital speckle target plate in the first measurement view coordinate system and the three-dimensional coordinate point set in the second measurement view coordinate system, thereby constructing a set of three-dimensional corresponding point pairs.

[0012] Step 6: Based on the set of corresponding three-dimensional points, use the singular value decomposition algorithm to solve for the rotation matrix and translation vector between the two viewpoints;

[0013] Step 7: Using the rotation matrix and translation vector obtained in Step 6, transform the local 3D point cloud of the workpiece under the first measurement viewpoint to the coordinate system of the second measurement viewpoint, and realize the stitching of the point clouds from the two views. For the multi-view case, repeat Steps 2 to 7 to perform pairwise registration and cumulative transformation, and finally obtain the complete 3D point cloud of the workpiece in the global coordinate system.

[0014] Furthermore, the stereo vision system in step one is a binocular stereo vision system.

[0015] Furthermore, in steps two and three, the structured light projection method employs fringe projection profilometry, projecting phase-shifted fringes and calculating the phase, combined with system calibration parameters to achieve three-dimensional reconstruction.

[0016] ,

[0017] ,

[0018] ;

[0019] in, The coordinates of the point to be reconstructed are in the world coordinate system. These are the calculated projector pixel coordinates; , These are the horizontal and vertical components of the pixel coordinates in the camera image, respectively. , These are the horizontal and vertical components of the pixel coordinates in the projector image, respectively. , These are the homogeneous normalized scaling factors for the camera and projector ends, respectively. , These are the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, respectively. , These are the projector intrinsic parameter matrix and the projector extrinsic parameter matrix, respectively. , These are the elements in the i-th row and j-th column of the camera combination projection matrix and the projector combination projection matrix, respectively, where i and j take values ​​from 1 to 4.

[0020] Furthermore, in step four, using the left camera image from the first measurement viewpoint as the reference image, digital image correlation matching is performed with the right camera image from the first measurement viewpoint, the left camera image from the second measurement viewpoint, and the right camera image from the second measurement viewpoint, respectively:

[0021] ,

[0022] ;

[0023] in, is the zero-mean normalized squared difference evaluation function, where P is the reference pixel in the reference image; , Each is a reference sub-region and target candidate sub-region Pixel coordinates within; , These are the grayscale values ​​of the corresponding pixels within the two sub-regions; , These are the average gray levels of the reference sub-region and the target candidate sub-region, respectively. For the reason The relevance evaluation value obtained through conversion.

[0024] Furthermore, in step five, the method for constructing the three-dimensional corresponding point pairs is as follows: stereo matching and reconstruction are performed on the left and right camera images under the first measurement viewpoint to obtain the three-dimensional coordinates of the target point under the first measurement viewpoint. Stereo matching and reconstruction are performed on the left and right camera images from the second measurement perspective to obtain the three-dimensional coordinates of the same target point from the second measurement perspective. The and The correspondence is established by the cross-view image matching results in step four.

[0025] Furthermore, in step six, the set of three-dimensional corresponding points is used. Constructing the covariance matrix The rigid body transformation matrix is ​​solved using singular value decomposition (SVD); the obtained rigid body transformation matrix is ​​then used to transform the point clouds from multiple perspectives to the global coordinate system, ultimately achieving point cloud stitching.

[0026]

[0027]

[0028]

[0029] Where Q is a set consisting of K sets of corresponding three-dimensional point pairs; , These are the matching 3D point sets under the first and second measurement perspectives, respectively. , For the k-th group of three-dimensional points with the same name; μ A μ B They are respectively , The center of mass; The covariance matrix is ​​constructed from the decentralized corresponding points; , , for The matrix obtained by performing singular value decomposition; For rotation matrix, The translation vectors together constitute the rigid body transformation parameters from the first measurement viewpoint to the second measurement viewpoint.

[0030] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0031] A computer-readable storage medium having program code corresponding to the executable instruction set thereon, which, when executed by a processor, implements the above-described method.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] (1) Free from dependence on workpiece surface, with extremely strong robustness: The present invention innovatively introduces an external fixed digital speckle target plate as the registration reference, so that the reliability of feature matching is completely based on the stable and controllable target plate surface, rather than the rough and variable workpiece surface. This enables the method to maintain stable high-precision registration capability when facing extremely rough, weak texture or no texture surface such as welding, casting, sandblasting, etc., and solves the fundamental problem of failure of traditional methods in such scenarios.

[0034] (2) Based on the high-precision digital image correlation matching technology of speckle sub-regions, sub-pixel-level image feature matching can be achieved. Combined with high-precision system calibration, the precise three-dimensional coordinates of feature points on the target plate can be reconstructed. The three-dimensional corresponding point pairs constructed in this way have low noise and accurate correspondence, providing high-quality input for subsequent SVD solution of rigid body transformation, thus ensuring the final stitching accuracy. Experiments show that the stitching error of this method on standard parts can reach the sub-millimeter level.

[0035] (3) The entire process does not require any pretreatment on the workpiece surface (such as pasting marks) or complex feature extraction and screening algorithms. Simply place and remove the target plate according to a fixed procedure during the measurement process, and the subsequent splicing process is fully automated, which significantly improves measurement efficiency and reduces the difficulty of operation and dependence on human experience. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the overall workflow of the point cloud stitching method based on digital speckle target constraints proposed in this invention.

[0037] Figure 2 This is a schematic diagram of the experimental setup.

[0038] Figure 3 This is a graph showing the deviation between the fitting results of the two spheres and the distance between their centers in the splicing experiment.

[0039] Figure 4 The figure shows the accuracy verification results of the splicing experiment of steel plates using the method of the present invention. Detailed Implementation

[0040] This invention proposes a robust point cloud stitching method based on digital speckle target plate constraints. Addressing the problems of low stitching accuracy and insufficient robustness in 3D measurement of workpieces with rough or textureless surfaces, such as those from welding or casting, due to poor point cloud reconstruction quality and difficulty in feature extraction, this invention proposes a multi-view registration scheme that introduces an externally fixed target plate as a stable spatial reference. This method abandons the approach of extracting features from the rough workpiece surface, instead fixing a digital speckle target plate in the measurement field of view as a common reference. The technical solution includes: acquiring modulated speckle images of a target plate from different perspectives using a binocular or multi-view stereo vision system; performing sub-pixel-level matching on the speckle images of the target plate from multiple perspectives based on digital image correlation technology to establish a high-precision two-dimensional pixel correspondence; back-projecting the two-dimensional corresponding points to three-dimensional space using pre-calibrated system parameters to reconstruct a set of three-dimensional corresponding point pairs on the target plate surface; solving the rigid body transformation matrix between perspectives using the least squares criterion through singular value decomposition based on this set of three-dimensional point pairs; and finally, unifying the workpiece point clouds independently reconstructed by the structured light system from each perspective to the global coordinate system according to the obtained transformation matrix to achieve complete and seamless three-dimensional stitching. Compared with splicing methods that rely on the characteristics of the workpiece itself, this invention completely eliminates the dependence on the surface quality and texture of the workpiece by introducing an external stable benchmark, significantly improving adaptability and robustness in extreme industrial scenarios such as roughness, weak texture, and even no texture; it avoids feature point drift and mismatch caused by surface undulations and uneven reflection, ensuring registration accuracy; the entire process does not require any preprocessing or marking of the workpiece, is simple to operate, and has a high degree of automation, providing a reliable technical means for high-precision 3D inspection of large-size, complex surface workpieces across the entire field of view.

[0041] like Figure 1 As shown, a point cloud stitching method based on digital speckle target constraints includes the following steps:

[0042] Step 1: Construct a stereo vision system consisting of at least two cameras, calibrate the system to obtain the intrinsic parameters of the cameras and the extrinsic parameters between the cameras; fix a digital speckle target plate with a random speckle pattern in the measurement field of view near the workpiece to be measured;

[0043] Step 2: Place the stereo vision system in the first measurement viewpoint. With the digital speckle target plate placed in the common field of view, simultaneously acquire speckle images of the target plate from the left and right cameras in the first viewpoint. Then remove or avoid the target plate, and use structured light projection to acquire and reconstruct the local three-dimensional point cloud of the workpiece in the first viewpoint.

[0044] Step 3: Move the stereo vision system to the second measurement viewpoint, repeat step 2, and simultaneously acquire speckle images of the same stationary target plate from the left and right cameras under the second viewpoint, and reconstruct the local three-dimensional point cloud of the workpiece under the second viewpoint.

[0045] Step 4: Based on digital image correlation technology, perform sub-pixel level matching on the first-view left camera image, the first-view right camera image, the second-view left camera image, and the second-view right camera image to establish a cross-view two-dimensional pixel correspondence relationship.

[0046] Step 5: Combining the system calibration parameters obtained in Step 1, backproject the corresponding two-dimensional pixel points obtained in Step 4 to three-dimensional space, and reconstruct the three-dimensional coordinate point set P1 of the same physical point on the surface of the digital speckle target plate in the first view coordinate system and the three-dimensional coordinate point set P2 in the second view coordinate system, thereby constructing a set of three-dimensional corresponding point pairs Q;

[0047] Step 6: Based on the set of corresponding three-dimensional point pairs Q, use the singular value decomposition (SVD) algorithm to solve for the rotation matrix and translation vector between the two viewpoints.

[0048] Step 7: Using the rotation matrix R and translation vector T obtained in Step 6, transform the local 3D point cloud of the workpiece in the first viewpoint to the coordinate system of the second viewpoint to achieve the stitching of the point clouds from the two views. For the multi-view case, repeat Step 2 to Step 7 to perform pairwise registration and cumulative transformation, and finally obtain the complete 3D point cloud of the workpiece in the global coordinate system.

[0049] Furthermore, the system includes a projector for structured light encoding, at least two industrial cameras, and a calibration board for system calibration; the projector is used to project a phase-shifted fringe pattern, and the industrial cameras are used to simultaneously acquire modulated fringe images and digital speckle target images.

[0050] Furthermore, the digital speckle target plate has a random and non-repeating high-contrast speckle pattern and optical stability to achieve reliable identification and matching under different measurement angles and lighting conditions.

[0051] Furthermore, in step four, a digital image correlation matching method based on speckle sub-regions is adopted, and matching errors caused by noise and uneven illumination are suppressed through cost aggregation or global optimization.

[0052] Furthermore, based on the set of corresponding three-dimensional point pairs, the singular value decomposition algorithm is used to solve the rotation matrix R and translation vector T that transform the coordinates of the first measurement viewpoint to the coordinates of the second measurement viewpoint, and the multi-view point cloud is unified to the global coordinate system through pairwise cumulative transformation.

[0053] The technical solution of the present invention will be described in detail below with reference to the embodiments.

[0054] Example

[0055] like Figure 1 As shown, a point cloud stitching method constrained by a digital speckle target includes the following steps:

[0056] Step 1: Project a set of phase-shifted sinusoidal fringes using a projector, and simultaneously acquire them using a camera. The expression for the fringes acquired by the camera is:

[0057]

[0058] in Indicates the number of phase shift steps. Represents the camera's pixel coordinates. and These are the background light intensity and the sinusoidal stripe modulation, respectively. This indicates the phase information that needs to be solved. This represents the fringe phase shift. This represents the nth phase-shift fringe image acquired by camera c at pixel coordinates. The grayscale value at the location; when camera number is not distinguished, the above grayscale value can be recorded as... N is the number of phase shift steps, and n is the phase shift image number. Background light intensity, For the fringe modulation, 2π(n-1) / N is the phase shift at the nth step.

[0059]

[0060] The result is obtained by solving the above arctangent function. The enclosed phase, its range is The phase discontinuity problem is solved using a time-phase expansion algorithm, and the required absolute phase is obtained in the following way:

[0061]

[0062] in It is a stripe grade.

[0063] Measurement system calibration: The traditional camera model is a pinhole imaging model. The object surface reaches the camera imaging surface through four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system. Its world coordinate system... with pixel coordinates The relationship between them is shown in the following formula:

[0064]

[0065] in For camera-side homogeneous normalized scale factor, , For focal length at directional components, Unit pixel size Let R be the pixel coordinates of the optical center, and T be the extrinsic parameters consisting of the rotation matrix and the translation vector. , These are the intrinsic parameter matrix and the extrinsic parameter matrix, respectively, with the superscript 'c' indicating the camera side; As a world coordinate system, These are the corresponding pixel coordinates.

[0066] The projector model is the same as the camera model, which is the reverse process of camera shooting. However, unlike a camera where pixel coordinates can be directly obtained, the projector's pixel coordinates need to be calculated using phase information, which can be obtained by the following formula:

[0067]

[0068] in The horizontal resolution of the projector image. The vertical resolution of the projected image. For camera pixel coordinates, These are the calculated projector pixel coordinates. For camera pixels The absolute phase is obtained by solving the vertical fringes. For camera pixels The absolute phase obtained from the horizontal fringes; , These are the spatial frequencies of the vertical and horizontal stripes, respectively. , These are the horizontal and vertical components of the projector pixel coordinates, respectively.

[0069] Camera calibration is performed using Zhang Zhengyou's calibration method—a circular checkerboard pattern is placed at different positions in the field of view, and the center coordinates of the checkerboard pattern at different positions are obtained through a center extraction method. The world coordinates and image coordinates of the checkerboard pattern centers are then used to solve for the planar homography matrix required for calibration. . , These represent the intrinsic parameter matrix and the extrinsic parameter matrix, respectively.

[0070]

[0071] In stereo vision, when a target point in three-dimensional space is simultaneously acquired by the device, the two-dimensional projected coordinates of the target point on different imaging planes are used to... and By combining this data with the equipment calibration parameters, the 3D data of that point can be obtained, completing the 3D reconstruction from that viewpoint. The equations can be listed as follows:

[0072]

[0073] The above equations contain four equations and three unknowns. Therefore, the least squares method can be used to solve the system of equations and obtain the optimal solution.

[0074] Step 2: Place the stereo vision system in the first measurement viewpoint. Under the condition that the digital speckle target plate is located in the common field of view and remains fixed, synchronously acquire the speckle images of the target plate from the left and right cameras under the first measurement viewpoint. Then, without changing the spatial reference position of the target plate, make the target plate avoid the workpiece measurement area, use a projector to project phase-shifting fringes and acquire them synchronously by the binocular system. Obtain the local three-dimensional point cloud of the workpiece in the coordinate system of the first measurement viewpoint through phase calculation and three-dimensional reconstruction.

[0075] Step 3: Keeping the spatial reference position of the digital speckle target unchanged, move the stereo vision system to the second measurement viewpoint and simultaneously acquire the left and right speckle images of the same target under the second measurement viewpoint; then move the target away from the workpiece measurement area, acquire the phase-shifted fringe image and reconstruct the local three-dimensional point cloud of the workpiece under the second measurement viewpoint.

[0076] Step 4: Based on digital image correlation technology, perform sub-pixel level matching on the four target speckle images obtained in Steps 2 and 3. The matching includes the correspondence between left and right images within the same measurement viewpoint for stereo reconstruction, and the correspondence between different measurement viewpoints for constructing cross-viewpoint constraints. The specific operations are as follows: Binocular 3D measurement is performed at the first measurement viewpoint. Structured light phase-shifting technology is used to project and acquire fringe sequences onto the object under test, reconstructing a high-precision local 3D point cloud of the object from this viewpoint. Subsequently, a high-contrast digital speckle target is placed within the overlapping area of ​​the common field of view to be stitched together, kept physically stationary, and speckle texture images captured by the left and right cameras at this viewpoint are acquired separately. The measurement system moves as a whole to the next observation position, and again synchronously acquires left and right image pairs containing the same speckle target. At this point, the stationary target becomes the only physical bridge connecting the old and new viewpoints. After acquiring speckle images from both viewpoints, the target is moved out of the field of view, leaving only the workpiece under test. The structured light projection and acquisition steps are repeated at the second measurement viewpoint to reconstruct a new local object point cloud.

[0077] Through the above process, the system acquires two sets of object topography point clouds from independent viewpoints, as well as a set of four speckle-coordinated images transmitted by a stationary target plate. If stitching together more viewpoints is required, simply reposition the target plate into the new field of view before each mobile device operation and repeat the above process to obtain a speckle image sequence shared between any adjacent viewpoints.

[0078] To achieve highly robust matching, a speckle sub-region-based matching method is employed. This method centers on a pixel in the reference image. Select a sub-image region The zero-mean normalized cross-correlation (ZNCC) criterion, which is insensitive to illumination changes, is used to calculate the similarity coefficient between the reference sub-region and the candidate region in the target image. The pixel-level correspondence is established by searching for the position that maximizes the similarity. The corresponding point of the reference point P is determined by finding the candidate position that minimizes the cost function within the search region of the target image.

[0079]

[0080]

[0081] Among them, C ZNSSD (P) is the zero-mean normalized squared difference evaluation function calculated at the reference point P; P(x,y) is the reference pixel point with coordinates (x,y) in the reference image; For the reference sub-region pixel coordinates, Ω represents the pixel coordinates within the target candidate sub-region. l Ω r These are the reference sub-region and the target candidate sub-region, respectively; F(x) l y l ), G(x r y r ) are the grayscale values ​​of the corresponding pixels in the reference sub-region and the target candidate sub-region, respectively; , These represent the average gray levels of the reference sub-region and the target candidate sub-region, respectively; σ F σ G C represents the L2 norm of the zero-mean grayscale values ​​of the reference sub-region and the target candidate sub-region, respectively. ZNCC (P) is derived from C ZNSSD (P) The correlation evaluation value obtained by conversion; in this embodiment, the two satisfy C. ZNCC (P) = 1 - 0.5 × C ZNSSD (P), i.e., C ZNSSD The smaller (P) is, the more C ZNCC The larger the value of (P), the higher the degree of matching.

[0082] Step 5: Combining the precise calibration parameters obtained in Step 1, backproject the corresponding two-dimensional pixels established in Step 4 into three-dimensional space to reconstruct the three-dimensional coordinates of the physical feature points on the target surface. Using the left and right speckle image pairs from the first measurement viewpoint and their matching point pairs, combined with the calibrated binocular geometric parameters, calculate the three-dimensional coordinate point set P1 of these feature points in the first measurement viewpoint coordinate system using the three-dimensional reconstruction formula from Step 1. Similarly, obtain the three-dimensional coordinate point set P2 in the second measurement viewpoint coordinate system. Based on the correspondence of corresponding points established in Step 4, construct K sets of three-dimensional corresponding point pairs with clear physical meaning. These point pairs are derived entirely from a stable external target plate, and their accuracy and reliability are unaffected by the surface roughness or texture of the workpiece. Where k is the index of the corresponding point pair, K is the total number of corresponding point pairs, and P1... k and P2 k These are the three-dimensional coordinates of the k-th target plate physical point in the first and second measurement viewpoint coordinate systems, respectively.

[0083] Step Six: Based on the high-quality 3D corresponding point pair set obtained in Step Five Solve for the optimal rigid body transformation that transforms the coordinates of the first measurement viewpoint to the coordinates of the second measurement viewpoint.

[0084] Furthermore, the rigid body transformation does not include scale changes, but only includes rotations described by the rotation matrix R and translations described by the translation vector T.

[0085]

[0086] in, To bypass The angle of rotation of the axis To bypass The angle of rotation of the axis To bypass The angle of rotation of the axis Translation vector Components on each coordinate axis, It is a rotation matrix. Where x, y, and z are the coordinate components of the 3D point in the first measurement viewpoint coordinate system before rigid body transformation in the three coordinate axes; x′, y′, and z′ are the coordinate components of the 3D point in the second measurement viewpoint coordinate system or target coordinate system after rotation and translation.

[0087] Constructing an error function based on the set of matched centroid pairs and calculating the R and T matrices is equivalent to finding the minimum value of this error function.

[0088]

[0089] The solution is obtained using Singular Value Decomposition (SVD). First, the covariance matrix is ​​constructed. We use linear algebra to perform SVD decomposition, and then solve for R and T.

[0090]

[0091]

[0092]

[0093] Where Q is a set consisting of K sets of corresponding three-dimensional point pairs; P A PB These are the matching 3D point sets from the first and second measurement perspectives, respectively; P k A P k B For the k-th group of three-dimensional points with the same name; μ A μ B P respectively A P B The centroid of H is the covariance matrix constructed from the decentralized corresponding points; U, Λ, and V are matrices obtained by performing singular value decomposition on H; R is the rotation matrix, and T is the translation vector. Together, they constitute the rigid body transformation parameters from the first measurement viewpoint to the second measurement viewpoint.

[0094] Step 7: Based on the R and T rigid body transformation relationship obtained in Step 6, transform the workpiece 3D point cloud from the first measurement perspective. Transform to the second measurement viewpoint coordinate system or the selected global coordinate system to finally achieve cross-viewpoint 3D point cloud stitching.

[0095]

[0096] Among them, P A P represents the 3D point cloud of the workpiece to be transformed in the first measurement viewpoint coordinate system. A→B Indicates that P A The resulting 3D point cloud is obtained by transforming it to the second measurement viewpoint coordinate system or the global coordinate system using the rotation matrix R and the translation vector T.

[0097] To verify the effectiveness of this invention, a three-dimensional point cloud stitching measurement system was constructed, consisting of a three-dimensional measurement device and a marker point projection device. The experiment employed a fringe projection binocular stereo vision matching system, including a DMD projector (DLP6500 with a resolution of 1920×1200) and two CMOS cameras (Basler acA1920-155 μm, with a resolution of 1920×1200 pixels), each equipped with a 12 mm lens. The experimentally set number of fringe periods and phase shift steps were as follows: , The object to be measured is placed approximately 0.7 m from the binocular measurement system. The experimental setup is as follows: Figure 2 As shown. The binocular measurement system was placed at positions 1 and 2 respectively to collect data, including fringe patterns and speckle patterns, where position 1 is... Figure 2 The current position can capture the left sphere of a standard dual-sphere system at a distance of 1 meter. Position 2 is the position that, based on the current position, is moved to the right to cover the right sphere of the standard dual-sphere system at a distance of 1 meter. Finally, rigid body transformation parameters are calculated based on the aforementioned constraint relationships to achieve 3D point cloud stitching.

[0098] The performance of the method of the present invention is further tested through experiments. Measurements were taken between the steel plate and a standard double-sphere workpiece, and the error between the fitted center-to-center distance of the standard double-sphere workpiece and the actual center-to-center distance was used as an assessment of the accuracy of the splicing error.

[0099] To verify the measurement accuracy and reliability of the speckle target registration method, a standard double-sphere specimen was first used for the experiment. Images containing the double spheres and a fixed speckle target were simultaneously acquired by left and right cameras. The correspondence between viewpoints was established based on the two-dimensional feature points on the target, and then the three-dimensional point clouds reconstructed from different viewpoints were unified into the same coordinate system to achieve stitching. Figure 3 To demonstrate the stitching effect of the point cloud stitching method based on digital speckle target constraints, experimental data on the stitching accuracy of a standard double-sphere workpiece (theoretical center-to-center distance 1000.4777 mm) show that the fitted center-to-center distance of the point cloud stitching method based on digital speckle target constraints proposed in this invention is 1000.431 mm, with an absolute error of 0.046 mm, verifying the technical superiority of this method in cross-view 3D stitching.

[0100] Figure 4 To demonstrate the results of steel plate splicing, this embodiment tested the performance of a registration method based on a speckle target plate under actual working conditions for a typical industrial workpiece with a rough surface, low reflectivity, and complex welding features. Despite significant surface undulations and uneven reflection, the spliced ​​point cloud completely covered the workpiece surface without large-area data loss or discontinuity. Good geometric continuity was maintained in the microscopic morphology features such as the weld-base transition area, weld bead contour, weld bead undulations, and pits, without significant misalignment or delamination due to surface reflection differences or feature loss.

[0101] In summary, the point cloud stitching method based on digital speckle target constraints proposed in this invention has significant advantages and innovations compared to traditional registration techniques that rely on the workpiece's own features: First, by introducing a fixed and stable external target as a spatial reference, it completely eliminates the dependence on the workpiece's surface quality and texture, maintaining extremely high registration robustness even in extreme industrial scenarios such as roughness, weak texture, or even no texture. Second, it utilizes digital image correlation technology to achieve sub-pixel-level image matching, and combines system calibration with high-precision reconstruction of corresponding 3D point pairs, fundamentally avoiding feature extraction errors and drift caused by surface undulations and uneven reflection in traditional methods, achieving sub-millimeter-level stitching accuracy. Third, the entire process requires no preprocessing of the workpiece (such as pasting marker points) and does not rely on time-consuming iterative fine registration algorithms, significantly improving automation and measurement efficiency. Experimental results show that this method can achieve complete, continuous, and high-precision 3D stitching on standard parts and actual industrial workpieces with welds and oxide scale.

[0102] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Various modifications, substitutions, and improvements made by those skilled in the art to the technical solutions of the present invention based on the provided textual description and drawings, without departing from the design concept and spirit of the present invention, should all fall within the scope of protection of the present invention.

Claims

1. A point cloud stitching method based on digital speckle target constraints, characterized in that, Includes the following steps: Step 1: Construct a stereo vision system consisting of at least two cameras, calibrate the stereo vision system to obtain the intrinsic parameters of the cameras and the extrinsic parameters between the cameras; fix a digital speckle target plate with a random speckle pattern in the measurement field of view near the workpiece to be measured. Step 2: Place the stereo vision system in the first measurement viewpoint. With the digital speckle target plate placed in the common field of view, simultaneously acquire speckle images of the digital speckle target plate from the left and right cameras in the first measurement viewpoint. Subsequently, the digital speckle target plate was removed or avoided, and the structured light projection method was used to acquire and reconstruct the local three-dimensional point cloud of the workpiece from the first measurement perspective. Step 3: Move the stereo vision system to the second measurement viewpoint, repeat step 2, and simultaneously acquire speckle images of the same stationary digital speckle target plate from the left and right cameras under the second measurement viewpoint, and reconstruct the local three-dimensional point cloud of the workpiece under the second measurement viewpoint. Step 4: Based on digital image correlation technology, perform sub-pixel level matching on the left camera image of the first measurement viewpoint, the right camera image of the first measurement viewpoint, the left camera image of the second measurement viewpoint, and the right camera image of the second measurement viewpoint to establish a two-dimensional pixel correspondence relationship across viewpoints. Using the left camera image from the first measurement viewpoint as the reference image, digital image correlation matching is performed with the right camera image from the first measurement viewpoint, the left camera image from the second measurement viewpoint, and the right camera image from the second measurement viewpoint, respectively: ， ； in, is the zero-mean normalized squared difference evaluation function, where P is the reference pixel in the reference image; , Each is a reference sub-region and target candidate sub-region Pixel coordinates within; , These are the grayscale values ​​of the corresponding pixels within the two sub-regions; , These are the average gray levels of the reference sub-region and the target candidate sub-region, respectively. For the reason The relevance evaluation value obtained by conversion; Step 5: Combining the system calibration parameters obtained in Step 1, the corresponding points of the two-dimensional pixels obtained in Step 4 are back-projected to the three-dimensional space to reconstruct the three-dimensional coordinate point set of the same physical point on the surface of the digital speckle target plate in the first measurement view coordinate system and the three-dimensional coordinate point set in the second measurement view coordinate system, thereby constructing a set of three-dimensional corresponding point pairs. Step 6: Utilize the set of corresponding three-dimensional points Constructing the covariance matrix The rigid body transformation matrix is ​​solved using singular value decomposition; the obtained rigid body transformation matrix is ​​then used to transform the point clouds from multiple perspectives to the global coordinate system, ultimately achieving point cloud stitching. ， ， ； Where Q is a set consisting of K sets of corresponding three-dimensional point pairs; , These are the matching 3D point sets under the first and second measurement perspectives, respectively. , For the k-th group of three-dimensional points with the same name; , They are respectively , The center of mass; The covariance matrix is ​​constructed from the decentralized corresponding points; , , for The matrix obtained by performing singular value decomposition; Let be a rotation matrix. As translation vectors, the two together constitute the rigid body transformation parameters from the first measurement viewpoint to the second measurement viewpoint; Step 7: Using the rotation matrix and translation vector obtained in Step 6, transform the local 3D point cloud of the workpiece under the first measurement viewpoint to the coordinate system of the second measurement viewpoint, and realize the stitching of the point clouds from the two views. For the multi-view case, repeat Steps 2 to 7 to perform pairwise registration and cumulative transformation, and finally obtain the complete 3D point cloud of the workpiece in the global coordinate system.

2. The point cloud stitching method based on digital speckle target constraints according to claim 1, characterized in that, The stereo vision system in step one is a binocular stereo vision system.

3. The point cloud stitching method based on digital speckle target constraints according to claim 1, characterized in that, In steps two and three, the structured light projection method employs fringe projection profilometry, projecting phase-shifted fringes and calculating the phase, then combining this with system calibration parameters to achieve 3D reconstruction. ， ， ； in, The coordinates of the point to be reconstructed are in the world coordinate system. These are the calculated projector pixel coordinates; , These are the horizontal and vertical components of the pixel coordinates in the camera image, respectively. , These are the horizontal and vertical components of the pixel coordinates in the projector image, respectively. , These are the homogeneous normalized scaling factors for the camera and projector ends, respectively. , These are the camera intrinsic parameter matrix and the camera extrinsic parameter matrix, respectively. , These are the projector intrinsic parameter matrix and the projector extrinsic parameter matrix, respectively. , These are the elements in the i-th row and j-th column of the camera combination projection matrix and the projector combination projection matrix, respectively, where i and j take values ​​from 1 to 4.

4. The point cloud stitching method based on digital speckle target constraints according to claim 1, characterized in that, In step five, the method for constructing the three-dimensional corresponding point pairs is as follows: stereo matching and reconstruction are performed on the left and right camera images under the first measurement viewpoint to obtain the three-dimensional coordinates of the target point under the first measurement viewpoint. Stereo matching and reconstruction are performed on the left and right camera images from the second measurement perspective to obtain the three-dimensional coordinates of the same target point from the second measurement perspective. ; and The correspondence is established by the cross-view image matching results in step four.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the point cloud stitching method based on digital speckle target plate constraints as described in any one of claims 1 to 4.

6. A computer-readable storage medium having an executable instruction set stored thereon, characterized in that, When the executable instruction set is executed by the processor, it implements the point cloud stitching method based on digital speckle target constraints as described in any one of claims 1 to 4.

Images