Point cloud splicing method based on target ball

Abstract

The invention provides a target ball-based point cloud splicing method, which comprises the following steps of: performing point cloud segmentation to obtain a workpiece point cloud and a spherical surface point cloud, and fitting a sphere center coordinate by utilizing the spherical surface point cloud; obtaining a rotation translation relation between the to-be-spliced image and the reference image based on a corresponding relation between the sphere center coordinates; rotating and translating the segmented spherical point clouds in the to-be-spliced image by using the rotating and translating relationship; storing the spherical surface point clouds belonging to the same common target ball to form a target ball point cloud set; obtaining an analytic solution of the sphere center coordinates by using the target sphere point cloud set; calculating a mean value distance between three-dimensional points in the spherical point cloud and a sphere center coordinate analytical solution, and iterating a rotation translation relation by taking a difference value between a distance mean value and a common target sphere theoretical radius as a constraint; according to the method, the constraint is constructed in the spherical point cloud, the iterative conversion relation is optimized, a more reasonable conversion matrix is obtained, the point cloud splicing precision is improved, the requirement for the target ball collection number is reduced, and the splicing efficiency is improved.

Description

Technical Field

[0001] The invention relates to the field of point cloud stitching, and in particular to a point cloud stitching method based on a target sphere. Background Art

[0002] Point cloud stitching is to convert point clouds in different coordinate systems into the same coordinate system to form a complete three-dimensional model of the object. It is a key link in three-dimensional reconstruction and directly affects the accuracy of the three-dimensional model.

[0003] Point cloud stitching methods are generally divided into featureless registration and feature registration. Featureless registration is generally based on the iterative closest point algorithm (ICP), but the use of the ICP algorithm often requires a good initial registration and generally does not meet industrial accuracy requirements. Feature registration is to use the common feature points in two coordinate systems for registration. Traditional feature registration methods include two registration methods: plane marking method and target sphere positioning method:

[0004] The plane marking method refers to pasting a plane marker on the workpiece to be measured. The plane marker has large distortion, and the image is greatly affected by ambient light. Therefore, the stitching accuracy of this method is unstable. At the same time, this method requires that the workpiece point cloud collected by the sensor at different positions has overlapping areas. If the workpiece only requires the front and back sides to be collected and only two positions are scanned, the target ball positioning method can only be selected.

[0005] The target ball positioning method refers to obtaining the spherical point cloud of the target ball at different positions, fitting the center of the ball, and using the center of the ball for alignment. Since the spherical point cloud collected at a single position is only a local point cloud of the target ball (the spherical point cloud of the center of the ball used for fitting can only cover up to 50% of the target ball), there are problems such as small point clouds, large partial distortion of the edge of the spherical surface, and high noise. Therefore, the detection error of the spherical center coordinates obtained by fitting makes it difficult to control the accuracy of the spherical center fitting, and the stitching effect is not satisfactory. In addition, in the prior art, this method must ensure that there are at least 3 target balls within the public field of view of the 3D point cloud sensor at different positions. Due to the limited field of view of the 3D point cloud sensor, it is often necessary to increase the shooting position to ensure that the number of target balls collected is sufficient. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a point cloud stitching method based on a target sphere. Based on the transformation relationship obtained by using the coordinates of the center of the sphere, the method further uses the coordinates of each three-dimensional point in the spherical point cloud to construct constraints, optimizes the iterative transformation relationship, and obtains a more reasonable transformation matrix, which effectively improves the accuracy of point cloud stitching. In addition, the method only requires that there are no less than two target spheres in the public field of view with different postures, which reduces the requirement on the number of target spheres collected and improves the stitching efficiency.

[0007] The technical solution is as follows:

[0008] A point cloud stitching method based on a target ball, wherein a plurality of target balls are provided and fixed in a detection area; during detection, a workpiece to be detected is placed in a detection station, and a three-dimensional point cloud sensor collects point cloud images of the workpiece and the target ball at different positions; one of the positions is taken as a reference position, and the point cloud image collected at the position is recorded as a reference image;

[0009] Use the following steps to stitch the point cloud images collected at other positions into the reference image:

[0010] Step 1: record the target sphere in the common field of view of the three-dimensional point cloud sensor in the reference posture and other postures as a common target sphere, and the number N of the common target spheres is greater than or equal to 2; record the point cloud images collected at other postures as images to be stitched;

[0011] Point cloud segmentation is performed in the reference image and the image to be stitched, respectively, to obtain N pieces of spherical point clouds corresponding to the workpiece point cloud and the public target sphere, and then the coordinates of the sphere center are fitted using each piece of spherical point cloud;

[0012] Based on the correspondence between multiple pairs of spherical center coordinates, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0013] Step 2: Use the rotation and translation relationship to rotate and translate the N spherical point clouds segmented from the stitched image to obtain the converted N spherical point clouds;

[0014] Among the N pieces of converted spherical point clouds and the N pieces of spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are correspondingly stored to form a set of N target sphere point clouds;

[0015] A linear least squares equation system is constructed using each target point cloud set, and each equation system is solved to obtain the analytical solution of the coordinates of the center of each common target ball;

[0016] Step 3: Take the average distance between each three-dimensional point in the target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius, use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint, iteratively update the rotation and translation relationship, and use the rotation and translation relationship with the minimum difference to achieve workpiece point cloud stitching.

[0017] Furthermore, the analytical solution of the coordinates of the center of the i-th public target ball is b i , b i The expression is as follows:

[0018]

[0019] in, Represents the jth 3D point coordinate in the target point cloud set corresponding to the i-th public target, j = 1, 2...M i , M i Represents the total number of three-dimensional points in the target sphere point cloud set corresponding to the i-th public target sphere; express The mean of express The mean of express The mean of .

[0020] Furthermore, in step 3, the rotation-translation relationship is iteratively updated to obtain the rotation-translation relationship with the minimum difference as follows:

[0021] When there is only one image to be stitched, there is only one rotation and translation relationship to be solved;

[0022] Construct the following objective function and use the optimization method to iteratively update the Euler angle θ and translation component t in the rotation and translation relationship:

[0023]

[0024] Among them, a k i represents the kth 3D point in the spherical point cloud of the i-th common target sphere segmented from the image to be stitched, k = 1, 2...S i , S i G(θ,t)a represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the image to be stitched; k i It means that the Euler angle θ and the translation component t in the rotation and translation relationship are used to transform the three-dimensional point a k i Perform rotation and translation; h p i represents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

[0025] Furthermore, when there are multiple images to be stitched, there are multiple corresponding rotation and translation relationships to be solved;

[0026] Construct the following objective function and use the optimization method to iteratively update the Euler angle θ in the rotation and translation relationship: w and the translation component t w :

[0027]

[0028] Among them, a k wi represents the kth 3D point in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched, k = 1, 2...S w i , S w i represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched; G(θ w ,t w ) k wi It means using the Euler angle θ in the wth rotation-translation relationship w and the translation component t w For a three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, W represents the total number of images to be stitched; h p i represents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

[0029] Furthermore, in step 1, based on the correspondence between the coordinates of the centers of N pairs of common target balls, the rotation and translation relationship between the image to be stitched and the reference image is obtained as follows:

[0030] When the number of common target balls is greater than or equal to 3, the corresponding relationship between N pairs of ball center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be stitched and the reference image;

[0031] When the number of public target balls is 2, for both the image to be stitched and the reference image, the coordinates of the center of one of the public target balls are rotated around the coordinates of the center of the other public target ball by a preset angle to obtain a pair of virtual center coordinates. The corresponding relationship between the two pairs of public target ball center coordinates and the one pair of virtual center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0032] Furthermore, in step 3, the steps of using the obtained rotation and translation relationship to realize the workpiece point cloud stitching are as follows:

[0033] The rotation and translation relationship after iterative update is recorded as RT;

[0034] When the number of common target balls is greater than or equal to 3, the workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching;

[0035] When the number of public target balls is 2, use RT to rotate and translate all point clouds in the image to be stitched and stitch them into the reference image; translate all point clouds in the image to be stitched again, and when the midpoint between the centers of the two public target balls coincides with the origin of the sensor coordinate system, obtain the translation matrix Among them, tx, ty, and tz are the translation components in the directions of the three coordinate axes X, Y, and Z respectively;

[0036] Then, take the line between the centers of the two common target balls after translation as the rotation axis, continue to rotate all the point clouds in the image to be stitched, and find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image. Based on the rotation angle and the rotation axis, the rotation matrix R' is obtained. The workpiece point cloud in the image to be stitched is rotated and translated to achieve workpiece point cloud stitching.

[0037] Preferably, the radii of the target spheres are different from each other.

[0038] Preferably, the distances between the target balls' centers are different from each other.

[0039] Furthermore, the acquisition scenarios of 3D point cloud sensors include the following:

[0040] Scenario 1:

[0041] At the reference pose, the field of view of the 3D point cloud sensor contains all target balls. At other poses, the field of view of the 3D point cloud sensor contains at least two target balls.

[0042] Scenario 2:

[0043] At two adjacent positions, the public field of view of the 3D point cloud sensor contains at least two target balls, one of the positions is used as the reference position, and the point cloud image collected at the other position is used as the image to be stitched;

[0044] Scenario 3:

[0045] At all positions, the field of view of the three-dimensional point cloud sensor includes all target balls. One of the positions is taken as the reference position, and the point cloud images collected at other positions are the images to be stitched.

[0046] In order to reduce the influence of the edge distortion and noise of the spherical point cloud on the stitching accuracy, preferably, in step one, the spherical point cloud obtained by segmentation from the reference image and the image to be stitched is processed as follows: the boundary line of the spherical point cloud is detected, and the point cloud around the boundary line is eliminated.

[0047] This method aims to solve the problem of low precision of the transformation relationship obtained by only using the coordinates of the sphere center in the existing method. It is optimized and uses the distance between the spherical point cloud and the analytical solution of the sphere center to further constrain and iterate the rotation and translation relationship to obtain a more reasonable transformation matrix, thereby improving the stitching accuracy.

[0048] At the same time, the three-dimensional point cloud sensor of this method only needs to have at least two target balls within the public field of view at different positions, which reduces the constraints on target ball collection and improves the stitching efficiency. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is described in detail below in conjunction with embodiments.

[0050] Example 1

[0051] This embodiment is aimed at the situation where the number of public target balls is 2, and the specific implementation scheme is as follows:

[0052] A point cloud stitching method based on a target ball, wherein a plurality of target balls are provided and fixed in a detection area; during detection, a workpiece to be detected is placed in a detection station, and a three-dimensional point cloud sensor collects point cloud images of the workpiece and the target ball at different positions; one of the positions is taken as a reference position, and the point cloud image collected at the position is recorded as a reference image;

[0053] Use the following steps to stitch the point cloud images collected at other positions into the reference image:

[0054] Step 1: record the target ball in the common field of view of the three-dimensional point cloud sensor in the reference posture and other postures as a common target ball. In this embodiment, the number N of the common target balls is equal to 2; record the point cloud images collected at other postures as images to be stitched;

[0055] In specific implementation, the acquisition scenarios of 3D point cloud sensors include the following:

[0056] Scenario 1:

[0057] At the reference position, the field of view of the 3D point cloud sensor includes all target balls. At other positions, the field of view of the 3D point cloud sensor includes two target balls.

[0058] Scenario 2:

[0059] At two adjacent positions, the common field of view of the 3D point cloud sensor contains two target balls, one of the positions is used as the reference position, and the point cloud image collected at the other position is the image to be stitched;

[0060] Scenario 3:

[0061] At all positions, the field of view of the 3D point cloud sensor includes all target balls (a total of 2 target balls are placed), one of the positions is taken as the reference position, and the point cloud images collected at other positions are the images to be stitched;

[0062] Point cloud segmentation is performed in the reference image and the image to be stitched, respectively, to obtain N pieces of spherical point clouds corresponding to the workpiece point cloud and the public target sphere, and then the coordinates of the sphere center are fitted using each piece of spherical point cloud;

[0063] In the specific implementation, the point cloud segmentation method is: based on the prior position information of the common target sphere and the workpiece in the image, the ROI selection box is pre-set, and the spherical point cloud and the workpiece point cloud are selected in the reference image / the image to be stitched by the ROI selection box.

[0064] For example, public target ball No. 1 is located at the upper left corner of the point cloud image, and public target ball No. 2 is located at the upper right corner of the point cloud image.

[0065] More preferably, in order to facilitate the distinction, the radius of the target spheres is different. When applied, the target sphere corresponding to each spherical point cloud is distinguished based on the radius; for example, the theoretical radius of public target sphere No. 1 is 3 cm, and the theoretical radius of public target sphere No. 2 is 5 cm. Each spherical point cloud is used to fit the sphere separately, and the two fitted radii are compared. The spherical point cloud with a fitted radius value close to 3 cm is recorded as the spherical point cloud of public target sphere No. 1, and the spherical point cloud with a fitted radius value close to 5 cm is recorded as the spherical point cloud of public target sphere No. 2.

[0066] This solution does not impose any specific restrictions on how to distinguish the target sphere corresponding to the spherical point cloud.

[0067] In specific implementation, the distances between the target spheres can also be set to be different, and the common target sphere corresponding to the spherical point cloud can be determined according to the distances between the spheres.

[0068] The corresponding target ball can also be determined based on the position information of the public target ball. For example, if public target ball No. 1 is located in the upper left corner of the point cloud image, the spherical point cloud in the upper left corner is recorded as the spherical point cloud of public target ball No. 1.

[0069] Based on the corresponding relationship between the coordinates of the centers of multiple pairs of common target balls, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0070] Specifically, in this embodiment, the number of public target balls is 2. For the image to be stitched and the reference image, the center coordinates of one of the public target balls are rotated by a preset angle around the center coordinates of the other public target ball to obtain a pair of virtual center coordinates. The corresponding relationship between the two pairs of public target ball center coordinates and the one pair of virtual center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be stitched and the reference image.

[0071] More preferably, in order to reduce the distortion and noise at the edge of the spherical point cloud, as a preferred implementation, the following processing is performed on the spherical point cloud segmented from the reference image and the image to be stitched: the boundary line of the spherical point cloud is detected, and the point cloud around the boundary line is eliminated, such as deleting points that are T times the point cloud density away from the boundary line, where T takes a value of 1 to 3.

[0072] Step 2: Use the rotation and translation relationship to rotate and translate the N spherical point clouds segmented from the stitched image to obtain the converted N spherical point clouds;

[0073] Among the N pieces of converted spherical point clouds and the N pieces of spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are correspondingly stored to form a set of N target sphere point clouds;

[0074] Use each target ball point cloud set to construct a linear least squares equation system, solve each equation system, and obtain the analytical solution b of the center coordinates of each common target ball. i ;

[0075] Step 3: Take the average distance between each three-dimensional point in the target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius, use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint, iteratively update the rotation and translation relationship, and use the rotation and translation relationship with the minimum difference to achieve workpiece point cloud stitching.

[0076] Specifically, the rotation and translation relationship after iterative update is denoted as RT;

[0077] In this embodiment, the number of public target balls is 2. RT is used to rotate and translate all point clouds in the image to be stitched and stitched into the reference image. All point clouds in the image to be stitched are translated again. When the midpoint between the centers of the two public target balls coincides with the origin of the sensor coordinate system, the translation matrix is ​​obtained. Among them, tx, ty, and tz are the translation components in the directions of the three coordinate axes X, Y, and Z respectively;

[0078] Then, using the line between the centers of the two common target balls after translation as the rotation axis, continue to rotate all the point clouds in the image to be stitched, and find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image (in this embodiment, the workpiece point cloud has partial overlap), and derive the rotation matrix R' based on the rotation angle and the rotation axis (to solve the ICP problem expressed based on the axis-angle method); use The workpiece point cloud in the image to be stitched is rotated and translated to achieve workpiece point cloud stitching.

[0079] For example, the analytical solution of the coordinates of the center of the i-th public target ball is b i , b i The expression is as follows:

[0080]

[0081] in, Represents the jth 3D point coordinate in the target point cloud set corresponding to the i-th public target, j = 1, 2...M i , M i Represents the total number of three-dimensional points in the target sphere point cloud set corresponding to the i-th public target sphere; express The mean of express The mean of express The mean of ; i = 1, 2...N, N represents the total number of public target balls.

[0082] More specifically, in step three, the root mean square of the distance between each three-dimensional point in the target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates is used as the estimated radius, and the square difference between the estimated radius and the theoretical radius of the common target sphere is used as a constraint to iteratively update the rotation and translation relationship, and the workpiece point cloud stitching is achieved using the rotation and translation relationship with the minimum square difference.

[0083] In a specific implementation, when there is only one image to be stitched, there is only one rotation and translation relationship to be solved;

[0084] The following objective function is constructed, and the optimization method is used to iteratively update the Euler angle θ and the translation component t in the rotation-translation relationship to obtain the Euler angle θ and the translation component t with the smallest square error. When used, the Euler angle θ and the translation component t are converted into a rotation-translation relationship, and the converted rotation-translation relationship is used to realize the workpiece point cloud stitching.

[0085] The objective function is as follows:

[0086]

[0087] Among them, a k i represents the kth 3D point in the spherical point cloud of the i-th common target sphere segmented from the image to be stitched, k = 1, 2...S i , S i G(θ,t)a represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the image to be stitched; k i It means that the Euler angle θ and the translation component t in the rotation and translation relationship are used to transform the three-dimensional point a k i Perform rotation and translation; h p irepresents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

[0088] When there are multiple images to be stitched, there are multiple corresponding rotation and translation relationships to be solved;

[0089] Construct the following objective function and use the optimization method to iteratively update the Euler angle θ in the rotation and translation relationship: w and the translation component t w :

[0090]

[0091] Among them, a k wi represents the kth 3D point in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched, k = 1, 2...S w i , S w i represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched; G(θ w ,t w ) k wi It means using the Euler angle θ in the wth rotation-translation relationship w and the translation component t w For a three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, W represents the total number of images to be stitched; h p i represents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

[0092] Example 2

[0093] This embodiment is aimed at the situation where the number of public target balls is greater than or equal to 3, and the specific implementation plan is as follows:

[0094] A point cloud stitching method based on a target ball, wherein a plurality of target balls are provided and fixed in a detection area; during detection, a workpiece to be detected is placed in a detection station, and a three-dimensional point cloud sensor collects point cloud images of the workpiece and the target ball at different positions; one of the positions is taken as a reference position, and the point cloud image collected at the position is recorded as a reference image;

[0095] Use the following steps to stitch the point cloud images collected at other positions into the reference image:

[0096] Step 1: record the target sphere in the common field of view of the three-dimensional point cloud sensor in the reference posture and other postures as a common target sphere, and the number N of the common target spheres is greater than or equal to 3; record the point cloud images collected at other postures as images to be stitched;

[0097] In specific implementation, the acquisition scenarios of 3D point cloud sensors include the following:

[0098] Scenario 1:

[0099] At the reference position, the field of view of the 3D point cloud sensor contains all target balls. At other positions, the field of view of the 3D point cloud sensor contains at least 3 target balls.

[0100] Scenario 2:

[0101] At two adjacent positions, the public field of view of the 3D point cloud sensor contains at least three target balls, one of the positions is taken as the reference position, and the point cloud image collected at the other position is the image to be stitched;

[0102] Scenario 3:

[0103] At all positions, the field of view of the 3D point cloud sensor includes all target balls (e.g., 3 target balls are installed in total), one of the positions is used as the reference position, and the point cloud images collected at other positions are the images to be stitched;

[0104] Point cloud segmentation is performed in the reference image and the image to be stitched, respectively, to obtain N pieces of spherical point clouds corresponding to the workpiece point cloud and the public target sphere, and then the coordinates of the sphere center are fitted using each piece of spherical point cloud;

[0105] In this embodiment, the point cloud segmentation method and the preferred design method of the target sphere radius and the sphere center distance are consistent with those in Embodiment 1, and will not be described in detail.

[0106] Based on the corresponding relationship between the coordinates of the centers of multiple pairs of common target balls, the rotation and translation relationship between the image to be stitched and the reference image is obtained;

[0107] In this embodiment, the number N of common target spheres is greater than or equal to 3, and the corresponding relationship between N pairs of sphere center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be spliced ​​and the reference image.

[0108] Step 2: Use the rotation and translation relationship to rotate and translate the N spherical point clouds segmented from the stitched image to obtain the converted N spherical point clouds;

[0109] Among the N pieces of converted spherical point clouds and the N pieces of spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are correspondingly stored to form a set of N target sphere point clouds;

[0110] A linear least squares equation system is constructed using each target point cloud set, and each equation system is solved to obtain the analytical solution of the coordinates of the center of each common target ball;

[0111] Step 3: Take the average distance between each three-dimensional point in the target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius, use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint, iteratively update the rotation and translation relationship, and use the rotation and translation relationship with the minimum difference to achieve workpiece point cloud stitching.

[0112] Specifically, the rotation and translation relationship after iterative update is denoted as RT;

[0113] In this embodiment, the number of public target spheres is greater than or equal to 3, and the workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching.

[0114] In this embodiment, the analytical solution b of the coordinates of the center of the i-th public target ball is i The expression is consistent with that in Example 1; in step three, the specific expression for constructing the objective function is consistent with that in Example 1 and will not be repeated here.

[0115] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A point cloud stitching method based on a target ball, wherein a plurality of target balls are provided and fixed in a detection area; during detection, a workpiece to be detected is placed in a detection station, and a three-dimensional point cloud sensor collects point cloud images of the workpiece and the target ball at different positions; one of the positions is taken as a reference position, and the point cloud image collected at the position is recorded as a reference image; It is characterized in that Use the following steps to stitch the point cloud images collected at other positions into the reference image: Step 1: record the target sphere in the common field of view of the three-dimensional point cloud sensor in the reference posture and other postures as a common target sphere, and the number N of the common target spheres is greater than or equal to 2; record the point cloud images collected at other postures as images to be stitched; Point cloud segmentation is performed in the reference image and the image to be stitched, respectively, to obtain N pieces of spherical point clouds corresponding to the workpiece point cloud and the public target sphere, and then the coordinates of the sphere center are fitted using each piece of spherical point cloud; Based on the correspondence between multiple pairs of spherical center coordinates, the rotation and translation relationship between the image to be stitched and the reference image is obtained; Step 2: Use the rotation and translation relationship to rotate and translate the N spherical point clouds segmented from the stitched image to obtain the converted N spherical point clouds; Among the N pieces of converted spherical point clouds and the N pieces of spherical point clouds segmented from the reference image, the spherical point clouds belonging to the same common target sphere are correspondingly stored to form N sets of target sphere point clouds; A linear least squares equation system is constructed using each target point cloud set, and each equation system is solved to obtain the analytical solution of the coordinates of the center of each common target ball; Step 3: Take the average distance between each three-dimensional point in the target sphere point cloud set and the corresponding analytical solution of the sphere center coordinates as the estimated radius, use the difference between the estimated radius and the theoretical radius of the common target sphere as a constraint, iteratively update the rotation and translation relationship, and use the rotation and translation relationship with the minimum difference to achieve workpiece point cloud stitching.

2. The point cloud stitching method according to claim 1, characterized in that: The analytical solution of the coordinates of the center of the i-th public target ball is b i , b i The expression is as follows: in, Represents the jth 3D point coordinate in the target point cloud set corresponding to the i-th public target, j = 1, 2...M i , M i Represents the total number of three-dimensional points in the target sphere point cloud set corresponding to the i-th public target sphere; express The mean of express The mean of express The mean of .

3. The point cloud stitching method according to claim 2, characterized in that: In step 3, the rotation-translation relationship is iteratively updated to obtain the rotation-translation relationship with the minimum difference as follows: When there is only one image to be stitched, there is only one rotation and translation relationship to be solved; Construct the following objective function and use the optimization method to iteratively update the Euler angle θ and translation component t in the rotation and translation relationship: Among them, a k i represents the kth 3D point in the spherical point cloud of the i-th common target sphere segmented from the image to be stitched, k = 1, 2...S i , S i G(θ,t)a represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the image to be stitched; k i It means that the Euler angle θ and the translation component t in the rotation and translation relationship are used to transform the three-dimensional point a k i Perform rotation and translation; h p i represents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

4. The point cloud stitching method according to claim 2, characterized in that: In step 3, the rotation-translation relationship is iteratively updated to obtain the rotation-translation relationship with the minimum difference as follows: When there are multiple images to be stitched, there are multiple corresponding rotation and translation relationships to be solved; Construct the following objective function and use the optimization method to iteratively update the Euler angle θ in the rotation and translation relationship: w and the translation component t w : Among them, a k wi represents the kth 3D point in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched, k = 1, 2...S w i , S w i represents the total number of three-dimensional points in the spherical point cloud of the i-th public target sphere segmented from the w-th image to be stitched; G(θ w ,t w ) k wi It means using the Euler angle θ in the wth rotation-translation relationship w and the translation component t w For a three-dimensional point a k wi Perform rotation and translation; w = 1, 2...W, W represents the total number of images to be stitched; h p i represents the pth 3D point in the spherical point cloud of the i-th public target sphere segmented from the reference image, p = 1, 2...Q i ,Q i represents the total number of 3D points in the spherical point cloud of the i-th public target sphere segmented from the reference image, r i represents the theoretical radius of the i-th public target sphere, i=1,2...N, N represents the total number of public target spheres.

5. The point cloud stitching method according to claim 1, characterized in that: In step 1, based on the correspondence between the coordinates of the centers of N pairs of common target balls, the rotation and translation relationship between the image to be stitched and the reference image is obtained as follows: When the number of common target balls is greater than or equal to 3, the corresponding relationship between N pairs of ball center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be stitched and the reference image; When the number of public target balls is 2, for both the image to be stitched and the reference image, the coordinates of the center of one of the public target balls are rotated around the coordinates of the center of the other public target ball by a preset angle to obtain a pair of virtual center coordinates. The corresponding relationship between the two pairs of public target ball center coordinates and the one pair of virtual center coordinates is used to perform SVD decomposition to obtain the rotation and translation relationship between the image to be stitched and the reference image.

6. The point cloud stitching method according to claim 1, characterized in that: In step 3, the steps to realize workpiece point cloud stitching using the obtained rotation and translation relationship are as follows: The rotation and translation relationship after iterative update is recorded as RT; When the number of common target balls is greater than or equal to 3, the workpiece point cloud in the image to be stitched is rotated and translated using RT and stitched into the reference image to achieve workpiece point cloud stitching; When the number of public target balls is 2, use RT to rotate and translate all point clouds in the image to be stitched and stitch them into the reference image; translate all point clouds in the image to be stitched again, and when the midpoint between the centers of the two public target balls coincides with the origin of the sensor coordinate system, obtain the translation matrix Among them, tx, ty, and tz are the translation components in the directions of the three coordinate axes X, Y, and Z respectively; Then, take the line between the centers of the two common target balls after translation as the rotation axis, continue to rotate all the point clouds in the image to be stitched, and find the rotation angle that maximizes the overlap between the rotated point cloud and the point cloud in the reference image. Based on the rotation angle and the rotation axis, the rotation matrix R' is obtained. The workpiece point cloud in the image to be stitched is rotated and translated to achieve workpiece point cloud stitching.

7. The point cloud stitching method according to claim 1, characterized in that: The radii of the target spheres are different from each other.

8. The point cloud stitching method according to claim 1, characterized in that: The distances between the centers of the target balls are different.

9. The point cloud stitching method according to claim 1, characterized in that: The acquisition scenarios of 3D point cloud sensors include the following: Scenario 1: At the reference pose, the field of view of the 3D point cloud sensor contains all target balls. At other poses, the field of view of the 3D point cloud sensor contains at least two target balls. Scenario 2: At two adjacent positions, the public field of view of the 3D point cloud sensor contains at least two target balls, one of the positions is used as the reference position, and the point cloud image collected at the other position is used as the image to be stitched; Scenario 3: At all positions, the field of view of the three-dimensional point cloud sensor includes all target balls. One of the positions is taken as the reference position, and the point cloud images collected at other positions are the images to be stitched.

10. The point cloud stitching method according to claim 1, characterized in that: In step 1, the following processing is performed on the spherical point clouds segmented from the reference image and the image to be stitched: the boundary line of the spherical point cloud is detected, and the point cloud around the boundary line is eliminated.

Images