Global 3D point cloud reconstruction method for high-precision aerospace parts based on multi-line laser and positioning device

Abstract

The present invention discloses a method for global three-dimensional point cloud reconstruction of high-precision aerospace parts based on multi-line laser and positioning device, using a fixed binocular camera C1 and a mobile line laser binocular camera C2 for image acquisition, wherein the line laser binocular camera C2 is placed in a positioning device, on which there are several marker balls, and each marker ball has several marker circles on its surface. In this way, the method based on multi-line laser and positioning device marking points is adopted to perform three-dimensional shape measurement, thereby avoiding the problem of not being able to correctly obtain matching points and causing low precision. At the same time, since the size of large-sized objects is large, and the binocular camera used for tracking or splicing should capture all positioning devices, the marking points on the positioning device occupy a small number of pixels in the overall image, which leads to problems such as failure to extract the marking points or matching errors during the process of extracting and matching the marking points. The present invention uses coarse matching and fine matching to obtain the spliced ​​rotation matrix R for the acquisition of matching points under two adjacent postures. i+1_i and the translation matrix t i+1_i , further improving the reconstruction and stitching accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional shape measurement of large-size objects, and more specifically, relates to a global three-dimensional point cloud reconstruction method for high-precision aerospace parts based on multi-line lasers and positioning devices. Background Art

[0002] With the continuous improvement of scientific and technological levels and industrial production levels, the requirements for detection accuracy in industrial production processes are also increasing. In traditional 3D shape measurement methods, the 3D point cloud reconstruction results for some special measurement objects with large volumes, smooth surfaces, single texture features, and high reflectivity are often poor and low in accuracy, which cannot meet the needs of production and detection.

[0003] Binocular stereo vision reconstruction is a common and mature method in traditional three-dimensional point cloud reconstruction methods. However, reconstruction using only binocular vision is more dependent on the texture information of the object itself, and often requires easily recognizable features on the surface of the object to be measured. For weak-textured objects with a relatively simple surface texture, the traditional binocular stereo vision reconstruction system often cannot correctly and efficiently obtain the landmark points on the surface of the object during three-dimensional point cloud reconstruction, which will lead to reduced reconstruction accuracy or even reconstruction errors. Another widely used three-dimensional point cloud reconstruction method based on surface structured light can project a modulated grating onto the object to be measured through a projection device to assist in the matching of landmark points. However, objects with highly reflective surfaces will reflect large areas of surface structured light, which will affect the extraction of grating fringes and the accuracy of reconstruction.

[0004] At the same time, for the scanning and reconstruction methods of large-sized objects to be measured, traditional measurement methods generally have problems such as insufficient measurement range, complex equipment structure, and high requirements for the working environment. In addition, for the three-dimensional point cloud reconstruction of large-sized objects, it is necessary to splice the point clouds from multiple perspectives. At present, when using a tracking binocular system to shoot the positioning device and the line laser binocular reconstruction device, most of them use a coded image device based on photogrammetry for splicing. However, different coding rules and detection methods will affect the measurement accuracy. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a global three-dimensional point cloud reconstruction method for high-precision aerospace parts based on multi-line lasers and positioning devices to improve the reconstruction and splicing accuracy.

[0006] To achieve the above-mentioned purpose, the present invention provides a global three-dimensional point cloud reconstruction method for high-precision aerospace parts based on a multi-line laser and a positioning device, which is characterized by comprising:

[0007] (1) Image acquisition

[0008] The binocular camera C1 is mounted on a fixed device, and the line laser binocular camera C2 is mounted at the center of a spherical positioning device, wherein the spherical positioning device has K marker balls, each marker ball has a plurality of marker circles on its surface for positioning, and the centers of all marker balls are located on the spherical surface of the spherical positioning device;

[0009] Debug binocular camera C1 and line laser binocular camera C2, so that binocular camera C1 can completely and clearly capture the spherical positioning device, and line laser binocular camera C2 can completely and clearly capture the high-precision parts to be tested, and the line laser can be completely and clearly projected onto the surface of the high-precision parts to be tested;

[0010] The line laser binocular camera C2 is a handheld device. After moving to each different posture, the operator manually controls the shooting. The left and right camera images of the high-precision parts to be tested captured at the i-th, i=1, 2, ..., N postures are recorded as Img2 l_i 、Img2 r_i ;

[0011] The line laser binocular camera C2 is used as the main camera and trigger source. It sends a trigger signal to the binocular camera C1 every time it takes a picture. The binocular camera C1 takes pictures immediately after receiving the trigger signal sent by the line laser binocular camera C2. That is, when the binocular camera C1 takes pictures of the spherical positioning device when the line laser binocular camera C2 is in each posture and takes pictures, the obtained left and right camera images are recorded as Img1 l_i 、Img1 r_i ;

[0012] (2) Calculate the three-dimensional coordinate set of the center of the marker ball in each position based on the image obtained by the binocular camera C1

[0013] 2.1) Fix the binocular camera C1, calibrate the binocular camera C1, and obtain the internal reference matrix K1 of the binocular camera C1 l 、K1 r , the external parameter rotation matrix R1 and translation vector t1 of the binocular camera C1;

[0014] 2.2) First, the rectangular area where the spherical positioning device is located is extracted, then binarized, and the canny edge detector is used for edge detection. Finally, the Hough circle detection process is used to extract the left and right camera images Img1 under the pose i. l_i 、Img1 r_i The coordinate set C1 of the center pixel coordinate system of the marker circle l_i :{c1 l_i_1 , c1 l_i_2 , …, c1 l_i_A} and C1 r_i :{c1 r_i_1 , c1r_i_2 , …, c1 r_i_B},in:

[0015] c1 l_i_a =(u l_i_a , v l_i_a ), a=1,2,...,A,is the set C1 l_i The a-th point in the set C1 l_i There are A points in total. In the pixel coordinate system, c1 l_i_a =(u l_i_a , v l_i_a ) takes the upper left corner of the image as the origin, u l_i_a is the pixel offset in the horizontal direction, v l_i_a is the pixel offset in the vertical direction;

[0016] c1 r_i_b =(u r_i_b , v r_i_b ), b = 1, 2, ..., B, is the set C1 r_i The b-th point in the set C1 r_i There are B points in total. In the pixel coordinate system, c1 r_i_b =(u r_i_b , v r_i_b ) takes the upper left corner of the image as the origin, u r_i_b is the pixel offset in the horizontal direction, v r_i_b is the pixel offset in the vertical direction;

[0017] 2.3) Set the coordinate set C1 in the pixel coordinate system l_i :{c1 l_i_1 , c1 l_i_2 , …, c1 l_i_A} and C1 r_i {c1 r_i_1 , c1 r_i_2 , …, c1 r_i_B All coordinates c1 in} l_i_a =(u l_i_a , v l_i_a ), c1 r_i_b =(u r_i_b , v r_i_b ) is converted to normalized pixel coordinates p1 l_i_a =(u l_i_a , v l_i_a ,1),p1 r_i_b =(u r_i_b , v r_i_b , 1);

[0018] 2.4) Match the left and right images Img1 through epipolar constraint l_i 、Img1r_i the coordinate set C1 in l_i and C1 r_i are matched with the center point of the marker circle:

[0019] Traverse the left camera image Img1 l_i the coordinate set C1 of the center points of the marker circles in i_l , and for each center point to be matched, traverse the right camera image Img1 r_i the set C1 of the center points of the marker circles in r_i , and calculate p1 r_i_b ·F1·p1 l_i_a T Compare with the threshold s. If there is p1 r_i_b ·F1·p1 l_i_a T < s, then record {p1 l_i_a , p1 r_i_b} as a pair of matching points 1 ≤ a ≤ A, 1 ≤ b ≤ B, F1 is the fundamental matrix, and F1 = K1 r -T ·t1^·R1·K1 l -1 , where K1 r -T is the inverse matrix of the transpose matrix of the internal parameter matrix K1 r , t1^ is the skew-symmetric matrix of the external parameter translation vector t1, and K1 l -1 is the inverse matrix of the internal parameter matrix K1 l ;

[0020] 2.5), Based on the center points of the marker circles on the left and right camera images Img1 l_i , Img1 r_i , solve the three-dimensional coordinates of the center point of the marker circle through the binocular camera parallax principle: The three-dimensional point coordinates calculated for the jth pair of matching points are p ij =(x ij , y ij , z ij ), where:

[0021]

[0022] Among them, (u0, v0) are the coordinates of the left camera principal point of the binocular camera C1, f is the focal length of the left and right cameras, T x is the baseline distance between the left and right cameras, is the parallax of the matching points, a j , b j are the serial numbers of the center points of the marker circles corresponding to the jth pair of matching points;

[0023] The three-dimensional coordinates p calculated from all matching pointsij Denote the set of three - dimensional coordinates of the fiducial circles in pose \(i\) as \(P\) i : \(\{p\) i1 , \(p\) i2 , \(\cdots\), \(p\) ij , \(\cdots p\) iJ}\), where \(p\) ij =(x ij , y ij , z ij ). Here, \(i\) represents the \(i\) - th pose, \(0\lt j\lt J\) represents the \(j\) - th three - dimensional coordinate, and a total of \(J\) three - dimensional coordinates are calculated according to the matching points in the \(i\) - th pose;

[0024] 2.6), By the DBSCAN clustering method, cluster all the fiducial circle dots belonging to the fiducial sphere \(k\) in the set \(P\) i into a three - dimensional point set \(P\) i_k : where \(p\) i_k_q =(x ikq , y ikq , z ikq ) represents the three - dimensional coordinate of the \(q\) - th fiducial circle dot on the \(k\) - th fiducial sphere in pose \(i\). For the \(k\) - th fiducial sphere, there are \(Q\) k fiducial circle dots;

[0025] For the three - dimensional point set \(P\) i_k of the fiducial circle dots of the \(k\) - th fiducial sphere in pose \(i\), fit the three - dimensional coordinate \(b\) i_k =(x i_k , y i_k , z i_k ) of the fiducial sphere center by the least - squares algorithm, and denote all the three - dimensional coordinates of the fiducial sphere centers as the set \(B\) i : \(\{b\) i_1 , \(b\) i_2 , \(\cdots\), \(b\) i_K}\);

[0026] Repeat steps 2.2) - 2.6) to obtain the sets \(B_1\), \(B_2\), \(\cdots\), \(B\) N ;

[0027] (3), Calculate the point cloud of the high - precision parts to be measured in each pose according to the image obtained by the line - laser binocular camera \(C_2\)

[0028] 3.1), Fix the line - laser binocular camera \(C_2\), calibrate the line - laser binocular camera \(C_2\) to obtain the internal parameter matrix \(K_2\) l , \(K_2\) r , the external parameter rotation matrix \(R_2\) and translation vector \(t_2\) of the line - laser binocular camera \(C_2\);

[0029] 3.2) Perform light plane calibration to obtain the plane parameter set LP of the light plane of the multi-line laser projected by the multi-line laser generator of the line laser binocular camera C2 in the camera coordinate system: {lp1, lp2, ..., lp G}, where lp g =(a g , b g , c g ), g = 1, 2, ..., G, G is the number of light planes, and the equation of the g-th light plane is: a g ·x+b g y+c g z+1=0;

[0030] 3.3) For the left and right camera images Img2 l_i 、Img2 r_i The laser lines in the image are extracted and the grayscale centroid method is used to obtain the left and right camera images Img2 l_i 、Img2 r_i Points on the laser line Among them, p l_i_m =(u l_i_m , v l_i_m ) is the left camera image Img2 l_i The coordinates of the mth point in the pixel coordinate system, a total of M l points, with the upper left corner of the image as the origin, u l_i_m is the pixel offset in the horizontal direction, v l_i_m is the pixel offset in the vertical direction, p r_i_m =(u r_i_m , v r_i_m ) is the right camera image Img2 r_i The coordinates of the mth point in the pixel coordinate system, a total of M r points, with the upper left corner of the image as the origin, u r_i_m is the pixel offset in the horizontal direction, v r_i_m is the pixel offset in the vertical direction;

[0031] The coordinate p in the pixel coordinate system l_i_m =(u l_i_m , v l_i_m ) is normalized to p r_i_m =(u r_i_m , v r_i_m ) is normalized to

[0032] 3.4) Left camera image Img2 under pose i l_i The coordinates of each point on the laser line are Traverse the right camera image Img2 r_i Points on the laser line, where the epipolar constraint is satisfied The point is taken as the matching point, F2=K2 r -T ·t2^·R2·K2 l -1 is the basic matrix, and the coordinates of the points to be matched are The obtained matching points are N m indivual;

[0033] 3.5) The obtained N m The points to be matched are respectively The disparity is calculated by the triangulation principle. m The 3D coordinates of the points to be matched p c_i_h =(x c_i_h ,y c_i_h , z c_i_h ), where c represents the point calculated by the triangulation principle, h is the serial number of the point to be matched, h = 1, 2, ..., N m ;

[0034] 3.6) The left camera image Img2 l_i Points on the laser line Through the internal parameter matrix K2 l Convert to image coordinate system coordinates:

[0035]

[0036] The image coordinate system takes the intersection of the imaging plane and the camera optical axis as its origin, l_i_m and l_i_m are the offsets in the horizontal and vertical directions respectively;

[0037] Taking the origin of the camera coordinate system as the starting point, The image coordinate system coordinate (x l_i_m ,y l_i_m , 1) intersects with H light planes, where the equation of the g-th light plane is: a g ·x+b g y+c g z = 0, (x, y, z) is the coordinate;

[0038] Starting from the optical center of the camera, the image coordinate system coordinate (x l_i_m ,y l_i_m , 1) is a ray, any point in this direction can be recorded as (s·x l_i_m , s.y l_i_m , s). l_i_m , s.yl_i_m , s) into the light plane equation a g ·x+b g y+c g z+1=0, we get: a g ·s·x l_i_m +b g ·s·y l_i_m +c g ·s+1=0, the solution is:

[0039]

[0040] Substitute the solution s into the point on the ray (s·x l_i_m , s.y l_i_m , s), get point p l_i_m The three-dimensional coordinate p on the g-th light plane i_m_g =(x i_m_g ,y i_m_g , z i_m_g ),in:

[0041]

[0042] In this way, the three-dimensional coordinate point set of the points on the laser line solved according to the light plane equation is recorded as {p i_m_1 , p i_m_2 ,…,p i_m_G};

[0043] 3.7) Traverse the set {p i_m_1 , p i_m_2 ,…,p i_m_G}For every point p i_m_g , calculate its difference with the set For each point p c_i_h The Euclidean distance d g_h , so we get the matrix:

[0044]

[0045] In the matrix D i_m Find the element with the smallest distance d g_h ,like Then we consider the set {p i_m_1 , p i_m_2 ,…,p i_m_G}The gth point p i_m_g for The three-dimensional coordinate point of the point cloud PCD under the pose i is added i Among them, is the distance threshold. If it is not met Discard this point;

[0046] Left camera image Img2 at pose i l_i Each point on the laser line All are processed according to steps 3.4) to 3.7, so that the complete point cloud PCD under pose i is obtained i , i=1,2,…,N;

[0047] (4) Calculate the rotation matrix and translation matrix to stitch the point cloud

[0048] 4.1) Roughly match the three-dimensional coordinates of the center of the marker ball under N positions:

[0049] According to the coordinate set B of the center of the marker ball in each pose i : {b i_1 , b i_2 , …, b i_K}, where all the sphere center points b in this position i_k =(x i_k , yi_k , z i_k ), according to x i_k ,y i_k , z i_k The order of descending order is firstly compared, that is, the x coordinates of all the sphere center points in the posture are arranged in descending order, among which the points with the same x coordinates are arranged in descending order according to the y coordinates, among which the points with the same y coordinates are arranged in descending order according to the z coordinates, and the set after descending order is recorded as B′ i : {b′ i_1 , b′ i_2 ,…,b′ i_K}, the sphere center point b′ of the corresponding order k in two adjacent positions i_k , b′ i+1_k As a rough matching point pair, K rough matching point pairs are obtained;

[0050] The corresponding three-dimensional coordinate set of the marker circle on the marker sphere is denoted as P′ i_k :

[0051] 4.2) Perform precise matching based on distance constraints on the three-dimensional coordinate set of the marker circle on the matching marker sphere:

[0052] First, calculate the distance matrix D i_k :

[0053]

[0054] in, The Euclidean distance from the qth marker circle point on the kth marker ball at position i to the remaining marker circle q′, q′≠q points:

[0055]

[0056] Among them, the distance matrix D i_k The qth row in represents the qth marker circle point on the kth marker ball at position i;

[0057] Then, calculate the distance matrix D i_k Each Euclidean distance of the qth row With the distance matrix D i+1_k Each Euclidean distance of the q″th row The distance norm of But q′≠q, q″′=1, 2, ..., Q k But q″′≠q″, so we get (Q k -1)×(Q k -1) distance two norm;

[0058] Statistical distance two norm Small residual distance threshold d * The number h, if the number h is greater than the number threshold h * , then the qth marker circle point on the kth marker ball under posture i and the q″th marker circle point on the kth marker ball under posture i+1 are considered as matching points. If the qth marker circle point on the kth marker ball under posture i and multiple marker circle points on the kth marker ball under posture i+1 are matching points, then the matching is considered to be wrong, and the matching point of the qth marker circle point on the kth marker ball under posture i is abandoned; if there is no matching point of the qth marker circle point on the kth marker ball under posture i, then the matching of the marker circle point is abandoned;

[0059] The three-dimensional coordinate sets of all the marker circles of all the marker balls at position i and position i+1 are precisely matched, so that J is obtained. * A pair of exact matching points;

[0060] 4.3) Stitching point cloud

[0061] First, the K rough matching point pairs under pose i and pose i+1 and J * The precise matching points are used as a set of matching point pairs, and the rotation matrix R from pose i+1 to pose i is calculated. i+1_i and the translation matrix t i+1_i , then according to the rotation matrix R i+1_i and the translation matrix t i+1_i , PCD of the point cloud under pose i+1 i+1 Convert to posture i, and convert them one by one in this way to convert all point clouds to the same coordinate system to complete point cloud stitching.

[0062] The object of the invention of the present invention is achieved in this way:

[0063] The present invention is a method for global three-dimensional point cloud reconstruction of high-precision aerospace parts based on multi-line laser and positioning device, using a fixed binocular camera C1 and a mobile line laser binocular camera C2 for image acquisition, wherein the line laser binocular camera C2 is placed in a positioning device, and there are several marker balls on the positioning device, wherein the centers of all the marker balls are all located on the spherical surface of a large ball. There are several marker circles on the surface of each marker ball, and the centers of the marker circles are all located on the spherical surface of the marker ball. In this way, the method based on multi-line laser and positioning device marking points is adopted to perform three-dimensional morphology measurement, which avoids the problem of low precision caused by the inability to correctly obtain matching points. At the same time, due to the large size of large-sized objects, and the binocular camera used for tracking and splicing should capture all positioning devices, the marker points on the positioning device occupy a small number of pixels in the overall image, which leads to problems such as failure to extract marker points or matching errors during the process of marker point extraction and matching. The present invention uses coarse matching and fine matching to obtain the splicing rotation matrix R for the acquisition of matching points in two adjacent postures. i+1_i and the translation matrix t i+1_i , further improving the reconstruction and stitching accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a specific implementation method of the global three-dimensional point cloud reconstruction method of aerospace high-precision parts based on multi-line laser and positioning device of the present invention;

[0065] Figure 2 It is a schematic diagram of the image acquisition principle of a specific implementation method of the global three-dimensional point cloud reconstruction method for aerospace high-precision parts based on multi-line laser and positioning device of the present invention. DETAILED DESCRIPTION

[0066] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0067] Figure 1 It is a flowchart of a specific implementation method of the global three-dimensional point cloud reconstruction method of aerospace high-precision parts based on multi-line laser and positioning device of the present invention.

[0068] In this embodiment, if Figure 1 As shown, the global three-dimensional point cloud reconstruction method of aerospace high-precision parts based on multi-line laser and positioning device of the present invention includes the following steps:

[0069] Step S1: Image acquisition

[0070] First, install the camera: install the binocular camera C1 on the fixed device, and install the line laser binocular camera C2 at the center of the spherical positioning device. There are K marking balls on the spherical positioning device, and the surface of each marking ball has several marking circles for positioning. The centers of all marking balls are located on the spherical surface of the spherical positioning device.

[0071] Then debug the camera: debug the binocular camera C1 and the line laser binocular camera C2, so that the binocular camera C1 can completely and clearly capture the spherical positioning device, the line laser binocular camera C2 can completely and clearly capture the high-precision parts to be tested, and the line laser can be completely and clearly projected onto the surface of the high-precision parts to be tested.

[0072] The final image capture: The line laser binocular camera C2 is a handheld device. After moving to each different posture, the operator manually controls the shooting. For the i-th, i=1, 2, ..., N-th posture, the left and right camera images of the high-precision parts to be tested are recorded as Img2 l_i 、Img2 r_i .

[0073] The line laser binocular camera C2 is used as the main camera and trigger source. It sends a trigger signal to the binocular camera C1 every time it takes a picture. The binocular camera C1 takes pictures immediately after receiving the trigger signal sent by the line laser binocular camera C2. That is, when the binocular camera C1 takes pictures of the spherical positioning device when the line laser binocular camera C2 is in each posture and takes pictures, the obtained left and right camera images are recorded as Img1 l_i 、Img1 r_i .

[0074] In this embodiment, the line laser projected onto the surface of the high-precision component to be measured is 7 parallel multi-line lasers.

[0075] Step S2: Calculate the three-dimensional coordinate set of the center of the marker ball in each position based on the image obtained by the binocular camera C1

[0076] Step S2.1: Calibrate the binocular camera C1

[0077] Fix the binocular camera C1, calibrate the binocular camera C1, and obtain the binocular camera C1 internal parameter matrix K1 l 、K1 r , the external parameter rotation matrix R1 and translation vector t1 of the binocular camera C1.

[0078] In this embodiment, the calibration plate is adjusted and the binocular camera C1 is calibrated using Zhang Zhengyou's calibration method.

[0079] Step S2.2: Obtain the coordinate set of the center pixel of the marker circle in the coordinate system

[0080] First, the rectangular area where the spherical positioning device is located, that is, the region of interest (ROI), is extracted. Because the marker ball and the marker point are located on the positioning device, it can be ensured that the ROI includes the marker point. After binarization, the canny edge detector is used for edge detection, and finally the Hough circle detection process is used to extract the left and right camera images Img1 under the pose i. l_i 、Img1 r_i The coordinate set C1 of the center pixel coordinate system of the marker circle 1_i :{c1 l_i_1 , c1 l_i_2 , …, c1 l_i_A} and C1 r_i :{c1 r_i_1 , c1 r_i_2 , …, c1 r_i_B},in:

[0081] c1 l_i_a =(u l_i_a , v l_i_a ), a=1,2,...,A,is the set C1 l_i The a-th point in the set C1 l_i There are A points in total. In the pixel coordinate system, c1 l_i_a =(u l_i_a , v l_i_a ) takes the upper left corner of the image as the origin, u l_i_a is the pixel offset in the horizontal direction, v l_i_a is the pixel offset in the vertical direction;

[0082] c1 r_i_b =(u r_i_b , v r_i_b ), b = 1, 2, ..., B, is the set C1 r_i The b-th point in the set C1 r_i There are B points in total. In the pixel coordinate system, c1 r_i_b =(u r_i_b , v r_i_b ) takes the upper left corner of the image as the origin, u r_i_b is the pixel offset in the horizontal direction, v r_i_b The pixel offset in the vertical direction.

[0083] Step S2.3: Normalized pixel coordinates

[0084] The coordinate set C1 in the pixel coordinate system l_i {c1 l_i_1 , c1 l_i_2 , …, c1 l_i_A} and C1 r_i {c1r_i_1 , c1 r_i_2 , …, c1 r_i_B} of all coordinates c1 l_i_a = (u l_i_a , v l_i_a ), c1 r_i_b = (u r_i_b , v r_i_b ) are converted into normalized pixel coordinates p1 l_i_a = (u l_i_a , v l_i_a , 1), p1 r_i_b = (u r_i_b , v r_i_b , 1).

[0085] Step S2.4: Conduct epipolar constraint matching for the center points of the marked circles

[0086] Conduct epipolar constraint matching for the coordinate sets C1 l_i and C1 r_i in the left and right images Img1 l_i and C1 r_i to match the center points of the marked circles:

[0087] Traverse the coordinate set C1 l_i of the center points of the marked circles in the left camera image Img1 i_l , and for each center to be matched, traverse the set C1 r_i of the center points of the marked circles in the right camera image Img1 r_i , and calculate p1 r_i_b · F1 · p1 l_i_a T and compare it with the threshold s. If there is p1 r_i_b · F1 · p1 l_i_a T < s, then record {p1 l_i_a , p1 r_i_b} as a pair of matching points 1 ≤ a ≤ A, 1 ≤ b ≤ B, F1 is the fundamental matrix, and F1 = K1 r -T · t1^ · R1 · K1 l -1 , where K1 r -T is the inverse matrix of the transpose matrix of the intrinsic matrix K1 r , t1^ is the skew-symmetric matrix of the extrinsic translation vector t1, and K1 l -1 is the inverse matrix of the intrinsic matrix K1 l .

[0088] Step S2.5: Solve the three-dimensional coordinates of the center points of the marked circles

[0089] Based on the left and right camera images Img1 after matching l_i 、Img1 r_i and the center point of the marker circle on Img1, solve the three-dimensional coordinates of the center point of the marker circle through the binocular camera parallax principle: The three-dimensional point coordinates calculated for the j-th pair of matching points are p ij =(x ij , y ij , z ij ), where:

[0090]

[0091] Among them, (u0, v0) is the coordinates of the principal point of the left camera of the binocular camera C1, f is the focal length of the left and right cameras, T x is the baseline distance between the left and right cameras, is the parallax of the matching point, a j , b j is the serial number of the center of the marker circle corresponding to the j-th pair of matching points.

[0092] Record the three-dimensional coordinates p ij calculated from all matching points as the set P of the three-dimensional coordinates of the marker circle in pose i i :{p i1 , p i2 , …, p ij , …p iJ}, where p ij =(x ij , y ij , z ij ) and i represents the i-th pose, 0 < j < J represents the j-th three-dimensional coordinate, and a total of J three-dimensional coordinates are calculated according to the matching points in the i-th pose.

[0093] Step S2.6: Obtain the set of three-dimensional coordinates of the center of the marker ball through clustering and least squares algorithm

[0094] Since the three-dimensional coordinate points of the marker circles in the set P i each belong to a certain marker ball on the positioning device, and we need to calculate the center of the marker ball, so we need to cluster the set P i to group the three-dimensional coordinates of the centers of the marker circles belonging to each marker ball together.

[0095] Through the DBSCAN clustering method, cluster all the marker circle points belonging to the marker ball k in the set P i into a three-dimensional point set P i_k : Among them, p i_k_q =(x ikq , y ikq , z ikq) represents the three-dimensional coordinates of the qth marker circle point on the kth marker ball under position i. For the kth marker ball, there are Q k Logo circle dots.

[0096] For the three-dimensional point set P of the marker circle of the kth marker ball under position i i_k , the three-dimensional coordinates b of the center of the marker ball are fitted by the least squares algorithm i_k =(x i_k ,y i_k , z i_k ), mark the three-dimensional coordinates of the centers of all the marker balls as set B i : {b i_1 , b i_2 , …, b i_K}. Among them, the least squares algorithm is used to fit the three-dimensional coordinates of the center of the marker ball, which is an existing technology and will not be described in detail here.

[0097] Repeat steps S2.2 to S2.6 to obtain the three-dimensional coordinate set B1, B2, ..., B of the center of the marker ball in N positions. N .

[0098] Step S3: Calculate the point cloud of the high-precision parts to be tested in each position according to the image obtained by the line laser binocular camera C2

[0099] Step S3.1: Calibrate the line laser binocular camera C2

[0100] Fix the line laser binocular camera C2, calibrate the line laser binocular camera C2, and obtain the internal parameter matrix K2 of the line laser binocular camera C2 l , K2 r , the external parameter rotation matrix R2 and translation vector t2 of the line laser binocular camera C2.

[0101] In this embodiment, the calibration plate is adjusted and the line laser binocular camera C2 is calibrated using Zhang Zhengyou's calibration method.

[0102] Step S3.2: Get the plane parameter set of the light plane

[0103] Perform light plane calibration to obtain the plane parameter set LP of the light plane of the multi-line laser projected by the multi-line laser generator of the line laser binocular camera C2 in the camera coordinate system: {lp1, lp2, ..., lp G}, where lp g =(a g , b g , c g ), g = 1, 2, ..., G, G is the number of light planes, and the equation of the g-th light plane is: a g ·x+b g y+c g z+1=0.

[0104] Step S3.3: Get the coordinates of the points on the laser line

[0105] For the left and right camera images Img2 l_i 、Img2 r_i The laser lines in the image are extracted and the grayscale centroid method is used to obtain the left and right camera images Img2 l_i 、Img2 r_i Points on the laser line Among them, p l_i_m =(u l_i_m , v l_i_m ) is the left camera image Img2 l_i The coordinates of the mth point in the pixel coordinate system, a total of M l points, with the upper left corner of the image as the origin, u l_i_m is the pixel offset in the horizontal direction, v l_i_m is the pixel offset in the vertical direction, p r_i_m =(u r_i_m , v r_i_m ) is the right camera image Img2 r_i The coordinates of the mth point in the pixel coordinate system, a total of M r points, with the upper left corner of the image as the origin, u r_i_m is the pixel offset in the horizontal direction, v r_i_m is the pixel offset in the vertical direction;

[0106] The coordinate p in the pixel coordinate system l_i_m =(u l_i_m , v l_i_m ) is normalized to p r_i_m =(u r_i_m , v r_i_m ) is normalized to

[0107] Step S3.4: Match the points on the laser line in the left and right camera images using epipolar constraints

[0108] Left camera image Img2 for pose i l_i The coordinates of each point on the laser line are Traverse the right camera image Img2 r_i Points on the laser line, where the epipolar constraint is satisfied The point is taken as the matching point, F2=K2 r -T ·t2^·R2·K2 l-1 is the basic matrix, and the coordinates of the points to be matched are The obtained matching points are N m indivual.

[0109] Since the present invention uses multi-line laser, for each point on the laser line in the left figure There will be multiple corresponding points that satisfy the epipolar constraints on the right figure. Let the total number of corresponding points on the right figure be N m indivual.

[0110] Step S3.5: Calculate N m The 3D coordinates of the points to be matched

[0111] The obtained N m The points to be matched are respectively The disparity is calculated by the triangulation principle. m The 3D coordinates of the points to be matched p c_i_h =(x c_i_h ,y c_i_h , z c_i_h ), where c represents the point calculated by the triangulation principle, h is the serial number of the point to be matched, h = 1, 2, ..., N m .

[0112] Because in fact, N m There is only one correct matching point among the matching points, so N m 3D coordinates Only one of the three dimensional coordinates is correct.

[0113] Step S3.6: Calculate the three-dimensional coordinate point set of the points on the laser line

[0114] The left camera image Img2 l_i Points on the laser line Through the internal parameter matrix K2 l Convert to image coordinate system coordinates:

[0115]

[0116] The image coordinate system takes the intersection of the imaging plane and the camera optical axis as its origin, l_i_m and l_i_m are the offsets in the horizontal and vertical directions respectively. In this embodiment, the unit is millimeter.

[0117] Taking the origin of the camera coordinate system as the starting point, The image coordinate system coordinate (x l_i_m ,y l_i_m, 1) intersects with H light planes, where the equation of the g-th light plane is: a g ·x+b g y+c g z=0, (x, y, z) is the coordinate.

[0118] Starting from the optical center of the camera, the image coordinate system coordinate (x l_i_m ,y l_i_m , 1) is a ray, any point in this direction can be recorded as (s·x l_i_m , s.y l_i_m , s). l_i_m , s.y l_i_m , s) into the light plane equation a g ·x+b g ·y+cg·z+1=0, we get: a g ·s·x l_i_m +b g ·s·y l_i_m +c g ·s+1=0, the solution is:

[0119]

[0120] Substitute the solution s into the point on the ray (s·x l_i_m , s.y l_i_m , s), get point p l_i_m The three-dimensional coordinate p on the g-th light plane i_m_g =(x i_m_g ,y i_m_g , z i_m_g ),in:

[0121]

[0122] In this way, the three-dimensional coordinate point set of the points on the laser line solved according to the light plane equation is recorded as {p i_m_1 , p i_m_2 ,…,p i_m_G}, only one of the three-dimensional coordinate points is correct.

[0123] Step S3.7: Calculate the Euclidean distance and add the point on the laser line with the smallest distance to the point cloud

[0124] Traverse the collection {p i_m_1 , p i_m_2 ,…,p i_m_G}For every point p i_m_g , calculate its difference with the set For each point p c_i_h The Euclidean distance d g_h , so we get the matrix:

[0125]

[0126] In the matrix D i_m Find the element with the smallest distance d g_h ,like Then we consider the set {p i_m_1 , p i_m_2 ,…,p i_m_G}The gth point p i_m_g ,gather The hth p c_i_h is to reconstruct the correct three-dimensional coordinate point, that is The three-dimensional coordinate point of the g-th point p i_m_g Add point cloud PCD under pose i i Among them, is the distance threshold. If it is not satisfied Discard this point.

[0127] Left camera image Img2 at pose i l_i Each point on the laser line All are processed according to steps 3.4) to 3.7, so that the complete point cloud PCD under pose i is obtained i , i=1,2,…,N.

[0128] Step S4: Point cloud stitching

[0129] When performing 3D coordinate matching, since there are many centers of marker circles extracted from each image, it takes more time and resources to directly use constraints such as distance matrix for matching. However, the coordinates of the centers of the marker balls fitted using the marker circle coordinates are relatively few, and when used as 3D points for matching, the distances between the marker balls may not be very different in each position, which affects the accuracy of matching; at the same time, it may cause large errors in the subsequent fitting of rotation and translation matrices.

[0130] Therefore, when performing three-dimensional coordinate matching, the center of each marker ball is first used for rough matching. Since the spatial coordinate position relationship of each marker ball remains basically unchanged when the positioning device and the laser camera C2 are moved in different positions, rough matching is first performed based on the three-dimensional coordinates of the marker ball.

[0131] Step S4.1: Roughly match the three-dimensional coordinates of the center of the marker ball in N positions:

[0132] According to the coordinate set B of the center of the marker ball in each pose i : {b i_1 , b i_2 , …, b i_K}, where all the sphere center points b in this position i_k=(x i_k ,y i_k , z i_k ), according to x i_k ,y i_k , z i_k The order of descending order is firstly compared, that is, the x coordinates of all the sphere center points in the posture are arranged in descending order, among which the points with the same x coordinates are arranged in descending order according to the y coordinates, among which the points with the same y coordinates are arranged in descending order according to the z coordinates, and the set after descending order is recorded as B′ i : {b′ i_1 , b′ i_2 ,…,b′ i_K}, the sphere center point b′ of the corresponding order k in two adjacent positions i_k , b′ i+1_k As a rough matching point pair, K rough matching point pairs are obtained.

[0133] The corresponding three-dimensional coordinate set of the marker circle on the marker sphere is denoted as P′ i_k :

[0134] Step S4.2: Perform precise matching based on distance constraints on the three-dimensional coordinate set of the marker circle on the matching marker sphere:

[0135] First, calculate the distance matrix D i_k :

[0136]

[0137] in, The Euclidean distance from the qth marker circle point on the kth marker ball at position i to the remaining marker circle q′, q′≠q points:

[0138]

[0139] Among them, the distance matrix D i_k The qth row in represents the qth marker circle point on the kth marker ball at position i.

[0140] Then, calculate the distance matrix D i_k Each Euclidean distance of the qth row With the distance matrix D i+1_k Each Euclidean distance of the q″th row The distance norm of But q′≠q, q″′=1, 2, ..., Q k But q″′≠q″, so we get (Q k -1)×(Q k -1) distance two norm.

[0141] Statistical distance two norm Small residual distance threshold d * The number h, if the number h is greater than the number threshold h * , then the qth marker circle point on the kth marker sphere under posture i and the q″th marker circle point on the kth marker sphere under posture i+1 are considered to be matching points; if the qth marker circle point on the kth marker sphere under posture i and multiple marker circle points on the kth marker sphere under posture i+1 are matching points, then the matching is considered to be wrong, and the matching point of the qth marker circle point on the kth marker sphere under posture i is abandoned; if the qth marker circle point on the kth marker sphere under posture i has no matching point, then the matching of this marker circle point is abandoned.

[0142] The three-dimensional coordinate sets of all the marker circles of all the marker balls at position i and position i+1 are precisely matched, so that J is obtained. * A precise matching point pair.

[0143] Step S4.3: Calculate the rotation matrix and translation matrix to stitch the point cloud

[0144] First, the K rough matching point pairs under pose i and pose i+1 and J * The precise matching points are used as a set of matching point pairs, and the rotation matrix R from pose i+1 to pose i is calculated. i+1_i and the translation matrix t i+1_i , then according to the rotation matrix R i+1_i and the translation matrix t i+1_i , PCD of the point cloud under pose i+1 i+1 Convert to posture i, and convert them one by one in this way to convert all point clouds to the same coordinate system to complete point cloud stitching.

[0145] Although the above describes an illustrative specific embodiment of the present invention to facilitate the understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiment. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

Claims

1. A global three-dimensional point cloud reconstruction method for high-precision aerospace parts based on multi-line laser and positioning device, characterized in that: include: (1) Image acquisition The binocular camera C1 is mounted on a fixed device, and the line laser binocular camera C2 is mounted at the center of a spherical positioning device, wherein the spherical positioning device has K marker balls, each marker ball has a plurality of marker circles on its surface for positioning, and the centers of all marker balls are located on the spherical surface of the spherical positioning device; Debug binocular camera C1 and line laser binocular camera C2, so that binocular camera C1 can completely and clearly capture the spherical positioning device, and line laser binocular camera C2 can completely and clearly capture the high-precision parts to be tested, and the line laser can be completely and clearly projected onto the surface of the high-precision parts to be tested; The line laser binocular camera C2 is a handheld device. After moving to each different posture, the operator manually controls the shooting. The left and right camera images of the high-precision parts to be tested captured at the i-th, i=1,2,…,N-th posture are recorded as Img2 l_i 、Img2 r_i ; The line laser binocular camera C2 is used as the main camera and trigger source. It sends a trigger signal to the binocular camera C1 every time it takes a picture. The binocular camera C1 takes pictures immediately after receiving the trigger signal sent by the line laser binocular camera C2. That is, when the binocular camera C1 takes pictures of the spherical positioning device when the line laser binocular camera C2 is in each posture and takes pictures, the obtained left and right camera images are recorded as Img1 l_i 、Img1 r_i ; (2) Calculate the three-dimensional coordinate set of the center of the marker ball in each position based on the image obtained by the binocular camera C1 2.1) Fix the binocular camera C1, calibrate the binocular camera C1, and obtain the internal reference matrix K1 of the binocular camera C1 l 、K1 r , the external parameter rotation matrix R1 and translation vector t1 of the binocular camera C1; 2.2) First, the rectangular area where the spherical positioning device is located is extracted, then binarized, and the canny edge detector is used for edge detection. Finally, the Hough circle detection process is used to extract the left and right camera images Img1 under the pose i. l_i 、Img1 r_i The coordinate set C1 of the center pixel coordinate system of the marker circle l_i :{c1 l_i_1 ,c1 l_i_2 ,…,c1 l_i_A } and C1 r_i :{c1 r_i_1 ,c1 r_i_2 ,…,c1 r_i_B },in: c1 i_i_a =(u l_i_a ,v l_i_a ), a=1,2,…,A, is the set C1 l_i The a-th point in the set C1 l_i There are A points in total. In the pixel coordinate system, c1 l_i_a =(u l_i_a ,v l_i_a ) takes the upper left corner of the image as the origin, u l_i_a is the pixel offset in the horizontal direction, v l_i_a is the pixel offset in the vertical direction; c1 r_i_b =(u r_i_b ,v r_i_b ), b=1,2,…,B, is the set C1 r_i The b-th point in the set C1 r_i There are B points in total. In the pixel coordinate system, c1 r_i_b =(u r_i_b ,v r_i_b ) takes the upper left corner of the image as the origin, u r_i_b is the pixel offset in the horizontal direction, v r_i_b is the pixel offset in the vertical direction; 2.3) Set the coordinate set C1 in the pixel coordinate system l_i :{c1 l_i_1 ,c1 l_i_2 ,…,c1 l_i_A } and C1 r_i {c1 r_i_1 ,c1 r_i_2 ,…,c1 r_i_B All coordinates c1 in} l_i_a =(u l_i_a ,v l_i_a ), c1 r_i_b =(u r_i_b ,v r_i_b ) is converted to normalized pixel coordinates p1 l_i_a =(u l_i_a ,v l_i_a ,1),p1 r_i_b =(u r_i_b ,v r_i_b ,1); 2.4) Match the left and right images Img1 by epipolar constraint l_i 、Img1 r_i The coordinate set C1 in l_i and C1 r_i Match the center point of the circle in the marker: Traverse the left camera image Img1 l_i in the set of coordinates C1 of the centers of the marked circles i_l , and for each center of the circle to be matched, traverse the right camera image Img1 r_i in the set of centers of the marked circles C1 r_i , and calculate p1 r_i_b ·F1·p1 l_i_a T Compare with the threshold s. If there is p1 r_i_b ·F1·p1 l_i_a T < s, then record {p1 l_i_a , p1 r_i_b} as a pair of matching points 1 ≤ a ≤ A, 1 ≤ b ≤ B, F1 is the fundamental matrix, and F1 = K1 r -T ·t1^·R1·K1 l -1 , where K1 r -T is the inverse matrix of the transpose matrix of the intrinsic matrix K1 r , t1^ is the skew-symmetric matrix of the extrinsic translation vector t1, and K1 l -1 is the inverse matrix of the intrinsic matrix K1 l ; 2.5) Based on the matched left and right camera images Img1 l_i 、Img1 r_i The three-dimensional coordinates of the center point of the marker circle on the jth pair of matching points are calculated as p ij =(x ij ,y ij ,z ij ),in: Among them, (u0,v0) is the coordinate of the left camera principal point of the binocular camera C1, f is the focal length of the left and right cameras, T x is the baseline distance between the left and right cameras, is the matching point disparity, a j ,b j is the center number of the marking circle corresponding to the jth pair of matching points; The three-dimensional coordinates p calculated for all matching points ij are denoted as the set P of the three-dimensional coordinates of the fiducial circle in pose i i :{p i1 , p i2 , …, p ij , … p iJ}, where p ij =(x ij , y ij , z ij ). Here, i represents the i-th pose, 0 < j < J represents the j-th three-dimensional coordinate, and a total of J three-dimensional coordinates are calculated based on the matching points in the i-th pose; 2.6) Using the DBSCAN clustering method, the set P is clustered according to the three-dimensional Euclidean distance constraint. i All the landmark circle points belonging to landmark ball k are clustered into a three-dimensional point set Among them, p i_k_q =(x ikq ,y ikq ,z ikq ) represents the three-dimensional coordinates of the qth marker circle point on the kth marker ball under position i. For the kth marker ball, there are Q k Logo circle dot; For the three-dimensional point set P of the marker circle of the kth marker ball under position i i_k , the three-dimensional coordinates b of the center of the marker ball are fitted by the least squares algorithm i_k =(x i_k ,y i_k ,z i_k ), mark the three-dimensional coordinates of the centers of all the marker balls as set B i :{b i_1 ,b i_2 ,…,b i_K }; Repeat steps 2.2) to 2.6) to obtain the three-dimensional coordinate set B1, B2, ..., B of the center of the marker ball in N positions. N ; (3) Calculate the point cloud of the high-precision parts to be tested in various positions based on the images obtained by the line laser binocular camera C2 3.1) Fix the line laser binocular camera C2, calibrate the line laser binocular camera C2, and obtain the internal parameter matrix K2 of the line laser binocular camera C2 l , K2 r , the external parameter rotation matrix R2 and translation vector t2 of the line laser binocular camera C2; 3.2) Perform light plane calibration to obtain the plane parameter set LP of the light plane of the multi-line laser projected by the multi-line laser generator of the line laser binocular camera C2 in the camera coordinate system: {lp1, lp2, …, lp G }, where lp g =(a g ,b g ,c g ),g=1,2,…,G, G is the number of light planes, and the equation of the g-th light plane is: a g ·x+b g y+c g z+1=0; 3.3) For the left and right camera images Img2 l_i 、Img2 r_i The laser lines in the image are extracted and the grayscale centroid method is used to obtain the left and right camera images Img2 l_i 、Img2 r_i Points on the laser line in, p l_i_m =(u l_i_m ,v l_i_m ) is the left camera image Img2 l_i The coordinates of the mth point in the pixel coordinate system, a total of M l points, with the upper left corner of the image as the origin, u l_i_m is the pixel offset in the horizontal direction, v l_i_m is the pixel offset in the vertical direction, p r_i_m =(u r_i_m ,b r_i_m ) is the right camera image Img2 r_i The coordinates of the mth point in the pixel coordinate system, a total of M r points, with the upper left corner of the image as the origin, u r_i_m is the pixel offset in the horizontal direction, v r_i_m is the pixel offset in the vertical direction; The coordinate p in the pixel coordinate system l_i_m =(u l_i_m ,v l_i_m ) is normalized to p r_i_m =(u r_i_m ,v r_i_m ) is normalized to 3.4) Left camera image Img2 under pose i l_i The coordinates of each point on the laser line are Traverse the right camera image Img2 r_i Points on the laser line, where the epipolar constraint is satisfied The point is taken as the matching point, F2=K2 r -T ·t2^·R2·K2 l -1 is the basic matrix, and the coordinates of the points to be matched are The obtained matching points are N m indivual; 3.5) The obtained N m The points to be matched are respectively The disparity is calculated by the triangulation principle. m The three-dimensional coordinates of the points to be matched {p c_i_1 ,p c_i_2 ,…,p c_i_Nm }, p c_i_h =(x c_i_h ,y c_i_h ,Z c_i_h ), where c represents the point calculated by the triangulation principle, h is the serial number of the point to be matched, h = 1, 2, ..., N m ; 3.6) The left camera image Img2 l_i Points on the laser line Through the internal parameter matrix K2 l Convert to image coordinate system coordinates: The image coordinate system takes the intersection of the imaging plane and the camera optical axis as its origin, l_i_m and l_i_m are the offsets in the horizontal and vertical directions respectively; Taking the origin of the camera coordinate system as the starting point, The image coordinate system coordinate (x l_i_m ,y l_i_m ,1) intersects with H light planes, where the equation of the gth light plane is: a g ·x+b g y+c g z = 0, (x, y, z) is the coordinate; Starting from the optical center of the camera, the image coordinate system coordinate (x l_i_m ,y l_i_m ,1) is a ray, any point in this direction can be recorded as (s·x l_i_m ,s·y l_i_m ,s), and the coordinates (s·x l_i_m ,s·y l_i_m ,s) into the light plane equation a g ·x+b g y+c g z+1=0, we get: a g ·s·x l_i_m +b g ·s·y l_i_m +c g ·s+1=0, the solution is: Substitute the solution s into the point on the ray (s·x l_i_m ,s·y l_i_m ,s), get point p l_i_m The three-dimensional coordinate p on the g-th light plane i_m_g =(x i_m_g ,y i_m_g ,z i_m_g ),in: In this way, the three-dimensional coordinate point set of the points on the laser line solved according to the light plane equation is recorded as {p i_m_1 ,p i_m_2 ,…,p i_m_G }; 3.7) Traverse the set {p i_m_1 ,p i_m_2 ,…,p i_m_G }For every point p i_m_g , calculate its difference with the set For each point p c_i_h The Euclidean distance d g_h , so we get the matrix: In the matrix D i_m Find the element with the smallest distance d g_h ,like Then we consider the set {p i_m_1 ,p i_m_2 ,…,p i_m_G }The gth point p i_m_g for The three-dimensional coordinate point of the point cloud PCD under the pose i is added i Among them, is the distance threshold. If it is not satisfied Discard this point; Left camera image Img2 at pose i l_i Each point on the laser line All are processed according to steps 3.4) to 3.7, so that the complete point cloud PCD under pose i is obtained i ,i=1,2,…,N; (4) Point cloud stitching 4.1) Roughly match the three-dimensional coordinates of the center of the marker ball under N positions: According to the coordinate set B of the center of the marker ball in each pose i :{b i_1 ,b i_2 ,…,b i_K }, where all the sphere center points b in this position i_k =(x i_k ,y i_k ,z i_k ), according to x i_k ,y i_k ,z i_k The order of descending order is firstly compared, that is, the x coordinates of all the sphere center points in the posture are arranged in descending order, among which the points with the same x coordinates are arranged in descending order according to the y coordinates, and the points with the same y coordinates are arranged in descending order according to the z coordinates. The set after descending order is recorded as B i ′:{b i ' _1 ,b i ' _2 ,…,b i ' _K }, the sphere center point b of the corresponding order k in two adjacent positions i ' _k ,b i ' +1_k As a rough matching point pair, K rough matching point pairs are obtained; The corresponding three-dimensional coordinate set of the marker circle on the marker sphere is denoted as P i ' _k : 4.2) Perform precise matching based on distance constraints on the three-dimensional coordinate set of the marker circle on the matching marker sphere: First, calculate the distance matrix D i_k : in, The Euclidean distance from the qth marker circle point on the kth marker ball at position i to the remaining marker circle points q′, q′≠q: Among them, the distance matrix D i_k The qth row in represents the qth marker circle point on the kth marker ball at position i; Then, calculate the distance matrix D i_k Each Euclidean distance of the qth row With the distance matrix D i+1_k Each Euclidean distance of the q″th row The distance norm of But q′≠q,q″′=1,2,…,Q k But q″′≠q″, so we get (Q k -1)×(Q k -1) distance two norm; Statistical distance two norm Small residual distance threshold d * The number h, if the number h is greater than the number threshold h * , then the qth marker circle point on the kth marker ball under posture i and the q″th marker circle point on the kth marker ball under posture i+1 are considered as matching points. If the qth marker circle point on the kth marker ball under posture i and multiple marker circle points on the kth marker ball under posture i+1 are matching points, then the matching is considered to be wrong, and the matching point of the qth marker circle point on the kth marker ball under posture i is abandoned; if there is no matching point of the qth marker circle point on the kth marker ball under posture i, then the matching of the marker circle point is abandoned; The three-dimensional coordinate sets of all the marker circles of all the marker balls at position i and position i+1 are precisely matched, so that J is obtained. * A pair of exact matching points; 4.3) Calculate the rotation matrix and translation matrix to stitch the point cloud First, the K rough matching point pairs under pose i and pose i+1 and J * The precise matching points are used as a set of matching point pairs, and the rotation matrix R from pose i+1 to pose i is calculated. i+1_i and the translation matrix t i+1_i , then according to the rotation matrix R i+1_i and the translation matrix t i+1_i , PCD of the point cloud under pose i+1 i+1 Convert to posture i, and convert them one by one in this way to convert all point clouds to the same coordinate system to complete point cloud stitching.

Images