A target ball-based point cloud splicing method

Abstract

The application provides a target ball-based point cloud splicing method, which comprises the following steps: segmenting a point cloud to obtain a workpiece point cloud and a spherical point cloud, and fitting a ball center coordinate by using the spherical point cloud; obtaining a rotation and translation relationship between a to-be-spliced image and a reference image based on the corresponding relationship between the ball center coordinates; rotating and translating the segmented spherical point cloud in the to-be-spliced image by using the rotation and translation relationship; storing the spherical point clouds belonging to the same common target ball to form a target ball point cloud set; obtaining an analytical solution of the ball center coordinate by using the target ball point cloud set; calculating the mean distance between the three-dimensional points in the spherical point cloud and the analytical solution of the ball center coordinate; and iteratively obtaining the rotation and translation relationship by taking the difference between the mean distance and the theoretical radius of the common target ball as a constraint. The method uses the constraint in the spherical point cloud to optimize the iterative conversion relationship, obtains a more reasonable conversion matrix, improves the point cloud splicing precision, and reduces the requirement for the number of target ball collections, thereby improving the splicing efficiency.

Description

Technical Field

[0001] This invention relates to the field of point cloud stitching, and more specifically to a point cloud stitching method based on a target sphere. Background Technology

[0002] Point cloud stitching is the process of transforming point clouds from different coordinate systems to the same coordinate system to form a complete 3D model of an object. It is a key step in 3D reconstruction and directly affects the accuracy of the 3D model.

[0003] Point cloud stitching methods are generally divided into featureless registration and feature registration. Featureless registration is usually based on the Iterative Closest Point (ICP) algorithm, but using the ICP algorithm often requires a good initial registration and generally does not meet industrial accuracy requirements. Feature registration uses common feature points in two coordinate systems for registration. Traditional feature registration methods include two types: planar marking method and target ball positioning method.

[0004] The planar marking method involves attaching planar marks to the workpiece. However, planar marks suffer from significant distortion, and the image is greatly affected by ambient light, resulting in unstable stitching accuracy. Furthermore, this method requires overlapping areas in the workpiece point clouds acquired by the sensor at different poses. If only the front and back sides of the workpiece need to be captured, and only two poses need to be scanned, then the target ball positioning method is the only viable option.

[0005] The target sphere localization method involves acquiring spherical point clouds of a target sphere at different poses, fitting the sphere center, and registering the target sphere using the center. However, since the spherical point cloud acquired at a single pose is only a local point cloud of the target sphere (the spherical point cloud used for fitting the sphere center can only cover a maximum of 50% of the target sphere), it suffers from problems such as insufficient point cloud coverage, large distortion at the sphere's edges, and high noise levels. Consequently, the fitted sphere center coordinates contain detection errors, making it difficult to control the sphere center fitting accuracy and resulting in unsatisfactory stitching effects. Furthermore, in existing technologies, this method requires at least three target spheres within the common field of view of the 3D point cloud sensor at different poses. Due to the limited field of view of the 3D point cloud sensor, it is often necessary to increase the shooting pose to ensure a sufficient number of target spheres are acquired. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a point cloud stitching method based on target spheres. This method, based on the transformation relationship derived from the sphere's center coordinates, further utilizes the coordinates of each 3D point in the spherical point cloud to construct constraints, optimizes the iterative transformation relationship, and yields a more reasonable transformation matrix. This effectively improves the accuracy of point cloud stitching. Furthermore, this method only requires at least two target spheres within the common field of view of different poses, reducing the requirement for the number of target spheres acquired and improving stitching efficiency.

[0007] The technical solution is as follows:

[0008] A point cloud stitching method based on target spheres, wherein multiple target spheres are fixed in a detection area; during detection, the workpiece to be tested is placed in the detection station, and a three-dimensional point cloud sensor acquires point cloud images of the workpiece and the target spheres at different poses; one of the poses is taken as the reference pose, and the point cloud image acquired at that pose is recorded as the reference image.

[0009] The following steps are used to stitch point cloud images acquired at other poses to the reference image:

[0010] Step 1: Record the target spheres within the common field of view of the 3D point cloud sensor in the reference pose and other poses as common target spheres, and the number of common target spheres N is greater than or equal to 2; record the point cloud images acquired at other poses as images to be stitched.

[0011] Point cloud segmentation is performed in the reference image and the image to be stitched to obtain N spherical point clouds corresponding to the workpiece point cloud and the common target sphere respectively. Then, the center coordinates of the sphere are fitted using each spherical point cloud.

[0012] Based on the correspondence between multiple pairs of sphere center coordinates, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0013] Step 2: Using rotation and translation relationships, rotate and translate the N spherical point clouds segmented from the image to be stitched to obtain the transformed N spherical point clouds;

[0014] In the N transformed spherical point clouds and the N spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are stored accordingly to form N target sphere point cloud sets.

[0015] A system of linear least squares equations is constructed using each target sphere point cloud set. By solving each system of equations, the analytical solution for the coordinates of the center of each common target sphere is obtained.

[0016] Step 3: Take the average distance between the three-dimensional points in each target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius. Use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint to iteratively update the rotation and translation relationship. Use the rotation and translation relationship with the smallest difference to realize the workpiece point cloud stitching.

[0017] Furthermore, let b be the analytical solution for the coordinates of the center of the i-th common target ball. i b i The expression is as follows:

[0018]

[0019] in, This represents the coordinates of the j-th 3D point in the target point cloud set corresponding to the i-th common target sphere, where j = 1, 2, ..., M. i M i This represents the total number of 3D points in the target point cloud set corresponding to the i-th common target sphere; express The mean, express The mean, express The mean.

[0020] Furthermore, in step three, the rotation and translation relationships are iteratively updated, and the rotation and translation relationships with the smallest difference are found in the following way:

[0021] When there is only one image to be stitched together, there is only one rotation and translation relationship to be solved.

[0022] Construct the following objective function and use optimization methods to iteratively update the Euler angle θ and translation component t in the rotation-translation relationship:

[0023]

[0024] Among them, a k i This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched, where k = 1, 2, ..., S. i S i G(θ,t) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched; k i This indicates that the Euler angle θ and translation component t in the rotation-translation relationship are used to represent the three-dimensional point a. k i Perform rotation and translation; h p i This represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

[0025] Furthermore, when there are multiple images to be stitched together, there are multiple corresponding rotation and translation relationships to be solved.

[0026] Construct the following objective function and use optimization methods to iteratively update the Euler angle θ in the rotation-translation relationship. w Translation component t w :

[0027]

[0028] Among them, a k wi This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched, where k = 1, 2, ..., S. w i S w i G(θ) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched; w ,t w )a k wi This indicates the use of Euler angles θ in the w-th rotation-translation relationship. w Translation component t w For three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, where W represents the total number of images to be stitched; h p i This represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

[0029] Furthermore, in step one, based on the correspondence between the center coordinates of N pairs of common target spheres, the rotation and translation relationship between the image to be stitched and the reference image is derived as follows:

[0030] When the number of common target balls is greater than or equal to 3, the SVD decomposition is performed using the correspondence between N pairs of ball center coordinates to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0031] When there are two common target balls, for both the image to be stitched and the reference image, the center coordinates of one of the common target balls are rotated by a preset angle around the center coordinates of the other common target ball to obtain a pair of virtual center coordinates. Using the correspondence between the two pairs of common target ball center coordinates and the one pair of virtual center coordinates, SVD decomposition is performed to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0032] Furthermore, in step three, the steps for stitching together the workpiece point cloud using the derived rotation and translation relationships are as follows:

[0033] Let RT be the rotation and translation relationship after iterative update;

[0034] When the number of common target balls is greater than or equal to 3, the workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching.

[0035] When there are two common target spheres, RT is used to rotate and translate all point clouds in the image to be stitched together and stitch them into the reference image; then, all point clouds in the image to be stitched together are translated again. When the midpoint between the centers of the two common target spheres coincides with the origin of the sensor coordinate system, the translation matrix is ​​obtained. Where tx, ty, and tz are the translation components in the X, Y, and Z coordinate axes, respectively;

[0036] Then, using the line connecting the centers of the two translated target spheres as the axis of rotation, continue rotating all point clouds in the image to be stitched together. Find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image. Based on the rotation angle and the axis of rotation, derive the rotation matrix R'. Using... Rotate and translate the workpiece point cloud in the image to be stitched to achieve workpiece point cloud stitching.

[0037] Preferably, the radii of the target spheres are all different.

[0038] Preferably, the distance between the centers of the target balls is different from each other.

[0039] Furthermore, the acquisition scenarios of 3D point cloud sensors include the following:

[0040] Scenario 1:

[0041] At the reference pose, the field of view of the 3D point cloud sensor contains all the target spheres; at other poses, the field of view of the 3D point cloud sensor contains at least two target spheres.

[0042] Scenario 2:

[0043] At two adjacent poses, the common field of view of the 3D point cloud sensor contains at least two target spheres. One pose is used as the reference pose, and the point cloud image acquired at the other pose is the image to be stitched together.

[0044] Scenario 3:

[0045] At all poses, the 3D point cloud sensor contains all target spheres within its field of view. One pose is used as the reference pose, and the point cloud images acquired at other poses are the images to be stitched together.

[0046] To reduce the impact of distortion and noise at the edges of the spherical point cloud on the stitching accuracy, preferably, in step one, the spherical point clouds segmented from the reference image and the image to be stitched are processed as follows: the boundary lines of the spherical point clouds are detected, and the point clouds around the boundary lines are removed.

[0047] This method aims to address the low accuracy of existing methods that only use the center coordinates of the sphere to derive the transformation relationship. It optimizes the method by using the distance between the spherical point cloud and the analytical solution of the sphere center to further constrain and iterate the rotation and translation relationship, thereby obtaining a more reasonable transformation matrix and improving the stitching accuracy.

[0048] Meanwhile, this method requires only at least two target spheres within the common field of view of the 3D point cloud sensor at different poses, reducing the constraints on target sphere acquisition and improving stitching efficiency. Detailed Implementation

[0049] The technical solution of the present invention will be described in detail below with reference to the embodiments.

[0050] Example 1

[0051] This embodiment addresses the scenario where there are two common target balls. The specific implementation plan is as follows:

[0052] A point cloud stitching method based on target spheres, wherein multiple target spheres are fixed in a detection area; during detection, the workpiece to be tested is placed in the detection station, and a three-dimensional point cloud sensor acquires point cloud images of the workpiece and the target spheres at different poses; one of the poses is taken as the reference pose, and the point cloud image acquired at that pose is recorded as the reference image.

[0053] The following steps are used to stitch point cloud images acquired at other poses to the reference image:

[0054] Step 1: The target spheres within the common field of view of the 3D point cloud sensor in the reference pose and other poses are recorded as common target spheres. In this embodiment, the number of common target spheres N is equal to 2. The point cloud images acquired at other poses are recorded as images to be stitched.

[0055] In practice, the data acquisition scenarios of 3D point cloud sensors include the following:

[0056] Scenario 1:

[0057] At the reference pose, the field of view of the 3D point cloud sensor contains all the target spheres; at other poses, the field of view of the 3D point cloud sensor contains 2 target spheres.

[0058] Scenario 2:

[0059] At two adjacent poses, the common field of view of the 3D point cloud sensor contains two target spheres. One pose is used as the reference pose, and the point cloud image acquired at the other pose is the image to be stitched together.

[0060] Scenario 3:

[0061] At all poses, the field of view of the 3D point cloud sensor contains all the target spheres (a total of 2 target spheres are placed). One pose is used as the reference pose, and the point cloud images acquired at other poses are the images to be stitched together.

[0062] Point cloud segmentation is performed in the reference image and the image to be stitched to obtain N spherical point clouds corresponding to the workpiece point cloud and the common target sphere respectively. Then, the center coordinates of the sphere are fitted using each spherical point cloud.

[0063] In practice, the point cloud segmentation method is as follows: based on the prior position information of the common target sphere and the workpiece in the image, a ROI selection box is set in advance, and the spherical point cloud and the workpiece point cloud are selected in the reference image / image to be stitched using the ROI selection box.

[0064] For example, target ball number 1 is located in the upper left corner of the point cloud image, and target ball number 2 is located in the upper right corner of the point cloud image.

[0065] More preferably, for ease of differentiation, the radii of the target spheres are different. In application, the target sphere corresponding to each spherical point cloud is distinguished based on the radius; for example, the theoretical radius of the No. 1 common target sphere is 3cm, and the theoretical radius of the No. 2 common target sphere is 5cm. Each spherical point cloud is fitted with a sphere, and the two fitted radii are compared. The spherical point cloud with a fitted radius value close to 3cm is recorded as the spherical point cloud of the No. 1 common target sphere, and the spherical point cloud with a fitted radius value close to 5cm is recorded as the spherical point cloud of the No. 2 common target sphere.

[0066] This solution does not impose specific restrictions on how to distinguish the target sphere corresponding to the spherical point cloud.

[0067] In practice, the center-to-center distance between target spheres can be set to be different, and the common target sphere corresponding to the spherical point cloud can be determined based on the center-to-center distance.

[0068] The corresponding target ball can also be determined based on the position information of the common target ball. For example, if the No. 1 common target ball is located in the upper left corner of the point cloud image, then the spherical point cloud in the upper left corner is recorded as the spherical point cloud of the No. 1 common target ball.

[0069] Based on the correspondence between the center coordinates of multiple pairs of common target spheres, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0070] Specifically, in this embodiment, there are two common target balls. For both the image to be stitched and the reference image, the center coordinates of one of the common target balls are rotated by a preset angle around the center coordinates of the other common target ball to obtain a pair of virtual center coordinates. Using the correspondence between the two pairs of common target ball center coordinates and the one pair of virtual center coordinates, SVD decomposition is performed to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0071] More preferably, in order to reduce distortion and noise at the edge of the spherical point cloud, as a preferred implementation, the spherical point cloud segmented from the reference image and the image to be stitched is processed as follows: the boundary line of the spherical point cloud is detected, and the point cloud around the boundary line is removed, for example, points that are T times the point cloud density at a distance of T from the boundary line are deleted, where T is 1 to 3.

[0072] Step 2: Using rotation and translation relationships, rotate and translate the N spherical point clouds segmented from the image to be stitched to obtain the transformed N spherical point clouds;

[0073] In the N transformed spherical point clouds and the N spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are stored accordingly to form N target sphere point cloud sets.

[0074] Construct a system of linear least squares equations using each target sphere point cloud set, solve each system of equations, and obtain the analytical solution b of the coordinates of the center of each common target sphere. i ;

[0075] Step 3: Take the average distance between the three-dimensional points in each target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius. Use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint to iteratively update the rotation and translation relationship. Use the rotation and translation relationship with the smallest difference to realize the workpiece point cloud stitching.

[0076] Specifically, let RT be the rotation and translation relationship after iterative update;

[0077] In this embodiment, there are two common target spheres. RT is used to rotate and translate all point clouds in the image to be stitched together, stitching them into the reference image. Then, all point clouds in the image to be stitched are translated again. When the midpoint between the centers of the two common target spheres coincides with the origin of the sensor coordinate system, the translation matrix is ​​obtained. Where tx, ty, and tz are the translation components in the X, Y, and Z coordinate axes, respectively;

[0078] Then, using the line connecting the centers of the two translated common target spheres as the axis of rotation, continue rotating all point clouds in the image to be stitched. Find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image (in this embodiment, the workpiece point cloud has local overlap). Based on the rotation angle and the axis of rotation, derive the rotation matrix R' (solving the ICP problem based on the axis-angle method); utilize... Rotate and translate the workpiece point cloud in the image to be stitched to achieve workpiece point cloud stitching.

[0079] For example, let b be the analytical solution for the coordinates of the center of the i-th common target ball. i b i The expression is as follows:

[0080]

[0081] in, This represents the coordinates of the j-th 3D point in the target point cloud set corresponding to the i-th common target sphere, where j = 1, 2, ..., M. i M i This represents the total number of 3D points in the target point cloud set corresponding to the i-th common target sphere; express The mean, express The mean, express The mean of the target balls; i = 1, 2...N, where N represents the total number of common target balls.

[0082] More specifically, in step three, the root mean square distance between the three-dimensional point in each target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates is used as the estimated radius. The squared difference between the estimated radius and the theoretical radius of the common target sphere is used as a constraint to iteratively update the rotation and translation relationship. The workpiece point cloud is stitched together using the rotation and translation relationship with the smallest squared difference.

[0083] In practice, when there is only one image to be stitched together, there is only one rotation and translation relationship to be solved.

[0084] Construct the following objective function, use optimization methods to iteratively update the Euler angle θ and translation component t in the rotation and translation relationship, find the Euler angle θ and translation component t with the smallest squared difference, and when using it, convert the Euler angle θ and translation component t into a rotation and translation relationship, and use the converted rotation and translation relationship to realize the workpiece point cloud stitching.

[0085] The objective function is as follows:

[0086]

[0087] Among them, a k i This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched, where k = 1, 2, ..., S. i S i G(θ,t) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched; k i This indicates that the Euler angle θ and translation component t in the rotation-translation relationship are used to represent the three-dimensional point a. k i Perform rotation and translation; h p iThis represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

[0088] When there are multiple images to be stitched together, there are multiple rotation and translation relationships to be solved.

[0089] Construct the following objective function and use optimization methods to iteratively update the Euler angle θ in the rotation-translation relationship. w Translation component t w :

[0090]

[0091] Among them, a k wi This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched, where k = 1, 2, ..., S. w i S w i G(θ) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched; w ,t w )a k wi This indicates the use of Euler angles θ in the w-th rotation-translation relationship. w Translation component t w For three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, where W represents the total number of images to be stitched; h p i This represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

[0092] Example 2

[0093] This embodiment addresses the scenario where the number of common target balls is three or more. The specific implementation plan is as follows:

[0094] A point cloud stitching method based on target spheres, wherein multiple target spheres are fixed in a detection area; during detection, the workpiece to be tested is placed in the detection station, and a three-dimensional point cloud sensor acquires point cloud images of the workpiece and the target spheres at different poses; one of the poses is taken as the reference pose, and the point cloud image acquired at that pose is recorded as the reference image.

[0095] The following steps are used to stitch point cloud images acquired at other poses to the reference image:

[0096] Step 1: Record the target spheres within the common field of view of the 3D point cloud sensor in the reference pose and other poses as common target spheres, and the number of common target spheres N is greater than or equal to 3; record the point cloud images acquired at other poses as images to be stitched.

[0097] In practice, the data acquisition scenarios of 3D point cloud sensors include the following:

[0098] Scenario 1:

[0099] At the reference pose, the field of view of the 3D point cloud sensor contains all the target spheres; at other poses, the field of view of the 3D point cloud sensor contains at least 3 target spheres.

[0100] Scenario 2:

[0101] At two adjacent poses, the common field of view of the 3D point cloud sensor contains at least 3 target spheres. One pose is used as the reference pose, and the point cloud image acquired at the other pose is the image to be stitched.

[0102] Scenario 3:

[0103] At all poses, the field of view of the 3D point cloud sensor includes all target spheres (e.g., a total of 3 target spheres are installed). One pose is used as the reference pose, and the point cloud images acquired at other poses are the images to be stitched together.

[0104] Point cloud segmentation is performed in the reference image and the image to be stitched to obtain N spherical point clouds corresponding to the workpiece point cloud and the common target sphere respectively. Then, the center coordinates of the sphere are fitted using each spherical point cloud.

[0105] In this embodiment, the point cloud segmentation method and the preferred design of the target sphere radius and center-to-center distance are consistent with those in Embodiment 1; they will not be repeated here.

[0106] Based on the correspondence between the center coordinates of multiple pairs of common target spheres, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0107] In this embodiment, the number of common target balls N is greater than or equal to 3. Using the correspondence between N and the coordinates of the ball centers, SVD decomposition is performed to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0108] Step 2: Using rotation and translation relationships, rotate and translate the N spherical point clouds segmented from the image to be stitched to obtain the transformed N spherical point clouds;

[0109] In the N transformed spherical point clouds and the N spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are stored accordingly to form N target sphere point cloud sets.

[0110] A system of linear least squares equations is constructed using each target sphere point cloud set. By solving each system of equations, the analytical solution for the coordinates of the center of each common target sphere is obtained.

[0111] Step 3: Take the average distance between the three-dimensional points in each target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius. Use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint to iteratively update the rotation and translation relationship. Use the rotation and translation relationship with the smallest difference to realize the workpiece point cloud stitching.

[0112] Specifically, let RT be the rotation and translation relationship after iterative update;

[0113] In this embodiment, the number of common target balls is greater than or equal to 3. The workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching.

[0114] In this embodiment, the analytical solution b for the coordinates of the center of the i-th common target ball is... i The expression is consistent with that in Example 1; in step three, the specific expression for constructing the objective function is consistent with that in Example 1, and will not be repeated here.

[0115] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A point cloud stitching method based on target spheres, wherein multiple target spheres are fixed within a detection area; during detection, the workpiece to be tested is placed in the detection station, and a three-dimensional point cloud sensor acquires point cloud images of the workpiece and the target spheres at different poses; one of the poses is taken as the reference pose, and the point cloud image acquired at that pose is recorded as the reference image. Its features are, The following steps are used to stitch point cloud images acquired at other poses to the reference image: Step 1: Record the target spheres within the common field of view of the 3D point cloud sensor in the reference pose and other poses as common target spheres, and the number of common target spheres N is greater than or equal to 2; record the point cloud images acquired at other poses as images to be stitched. Point cloud segmentation is performed in the reference image and the image to be stitched to obtain N spherical point clouds corresponding to the workpiece point cloud and the common target sphere respectively. Then, the center coordinates of the sphere are fitted using each spherical point cloud. Based on the correspondence between multiple pairs of sphere center coordinates, the rotation and translation relationship between the image to be stitched and the reference image is obtained; Step 2: Using rotation and translation relationships, rotate and translate the N spherical point clouds segmented from the image to be stitched to obtain the transformed N spherical point clouds; In the N transformed spherical point clouds and the N spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are stored accordingly to form N target sphere point cloud sets. A system of linear least squares equations is constructed using each target sphere point cloud set. By solving each system of equations, the analytical solution for the coordinates of the center of each common target sphere is obtained. Step 3: Take the average distance between the three-dimensional points in each target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius. Use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint to iteratively update the rotation and translation relationship. Use the rotation and translation relationship with the smallest difference to realize the workpiece point cloud stitching.

2. The point cloud stitching method as described in claim 1, characterized in that: Let b be the analytical solution for the coordinates of the center of the i-th common target ball. i b i The expression is as follows: in, This represents the coordinates of the j-th 3D point in the target point cloud set corresponding to the i-th common target sphere, where j = 1, 2, ..., M. i M i This represents the total number of 3D points in the target point cloud set corresponding to the i-th common target sphere; express The mean, express The mean, express The mean.

3. The point cloud stitching method as described in claim 2, characterized in that: In step three, the rotation and translation relationships are iteratively updated. The method for finding the rotation and translation relationship with the smallest difference is as follows: When there is only one image to be stitched together, there is only one rotation and translation relationship to be solved. Construct the following objective function and use optimization methods to iteratively update the Euler angle θ and translation component t in the rotation-translation relationship: Among them, a k i This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched, where k = 1, 2, ..., S. i S i G(θ,t) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the images to be stitched; k i This indicates that the Euler angle θ and translation component t in the rotation-translation relationship are used to represent the three-dimensional point a. k i Perform rotation and translation; h p i This represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

4. The point cloud stitching method as described in claim 2, characterized in that: In step three, the rotation and translation relationships are iteratively updated. The method for finding the rotation and translation relationship with the smallest difference is as follows: When there are multiple images to be stitched together, there are multiple rotation and translation relationships to be solved. Construct the following objective function and use optimization methods to iteratively update the Euler angle θ in the rotation-translation relationship. w Translation component t w : Among them, a k wi This represents the k-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched, where k = 1, 2, ..., S. w i S w i G(θ) represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the w-th image to be stitched; w ,t w )a k wi This indicates the use of Euler angles θ in the w-th rotation-translation relationship. w Translation component t w For three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, where W represents the total number of images to be stitched; h p i This represents the p-th 3D point in the spherical point cloud of the i-th common target sphere segmented from the reference image, where p = 1, 2, ..., Q. i Q i r represents the total number of 3D points in the spherical point cloud of the i-th common target sphere segmented from the reference image. i Let represent the theoretical radius of the i-th common target ball, i = 1, 2, ..., N, where N represents the total number of common target balls.

5. The point cloud stitching method as described in claim 1, characterized in that: In step one, based on the correspondence between the center coordinates of N pairs of common target spheres, the rotation and translation relationship between the image to be stitched and the reference image is derived as follows: When the number of common target balls is greater than or equal to 3, the SVD decomposition is performed using the correspondence between N pairs of ball center coordinates to obtain the rotation and translation relationship between the image to be stitched and the reference image. When there are two common target balls, for both the image to be stitched and the reference image, the center coordinates of one of the common target balls are rotated by a preset angle around the center coordinates of the other common target ball to obtain a pair of virtual center coordinates. Using the correspondence between the two pairs of common target ball center coordinates and the one pair of virtual center coordinates, SVD decomposition is performed to obtain the rotation and translation relationship between the image to be stitched and the reference image.

6. The point cloud stitching method as described in claim 1, characterized in that: In step three, the steps for stitching together the workpiece point cloud using the derived rotation and translation relationships are as follows: Let RT be the rotation and translation relationship after iterative update; When the number of common target balls is greater than or equal to 3, the workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching. When there are two common target spheres, RT is used to rotate and translate all point clouds in the image to be stitched together and stitch them into the reference image; then, all point clouds in the image to be stitched together are translated again. When the midpoint between the centers of the two common target spheres coincides with the origin of the sensor coordinate system, the translation matrix is ​​obtained. Where tx, ty, and tz are the translation components in the X, Y, and Z coordinate axes, respectively; Then, using the line connecting the centers of the two translated target spheres as the axis of rotation, continue rotating all point clouds in the image to be stitched together. Find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image. Based on the rotation angle and the axis of rotation, derive the rotation matrix R'. Using... Rotate and translate the workpiece point cloud in the image to be stitched to achieve workpiece point cloud stitching.

7. The point cloud stitching method as described in claim 1, characterized in that: The radii of the target balls are all different.

8. The point cloud stitching method as described in claim 1, characterized in that: The distance between the centers of the target balls is different for each ball.

9. The point cloud stitching method as described in claim 1, characterized in that: The acquisition scenarios of 3D point cloud sensors include the following: Scenario 1: At the reference pose, the field of view of the 3D point cloud sensor contains all the target spheres; at other poses, the field of view of the 3D point cloud sensor contains at least two target spheres. Scenario 2: At two adjacent poses, the common field of view of the 3D point cloud sensor contains at least two target spheres. One pose is used as the reference pose, and the point cloud image acquired at the other pose is the image to be stitched together. Scenario 3: At all poses, the 3D point cloud sensor contains all target spheres within its field of view. One pose is used as the reference pose, and the point cloud images acquired at other poses are the images to be stitched together.

10. The point cloud stitching method as described in claim 1, characterized in that: In step one, the spherical point clouds segmented from the reference image and the image to be stitched are processed as follows: the boundary lines of the spherical point clouds are detected, and the point clouds around the boundary lines are removed.

Images