Binocular multi-line laser three-dimensional global measurement method based on coding target

Abstract

The invention discloses a binocular multi-line laser three-dimensional global measurement method based on a coding target, and the method comprises the steps: firstly designing an annular coding mark point, a calibration plate and a target, then building an image collection device, and collecting calibration images and reconstruction images under multiple poses; then, system transformation matrix calibration based on annular coding mark points is completed through calibration images under multiple poses, and multi-line laser light plane calibration is achieved through reconstruction images under multiple poses; and finally, multi-pose point cloud reconstruction and point cloud global registration are realized based on a calibration result, so that the three-dimensional reconstruction precision and the multi-pose point cloud global registration precision under a high unit pose are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction, and more specifically, relates to a binocular multi-line laser three-dimensional global measurement method based on a coded target. Background Art

[0002] Global registration of point clouds has important application value in three-dimensional reconstruction problems such as industrial inspection and cultural heritage protection, where a complete three-dimensional model of an object needs to be obtained through multi-view scanning. In order to obtain a complete point cloud of the object to be measured during three-dimensional reconstruction, data with inconsistent coordinate systems and different modal data must be fused through registration. Laser point cloud data collected in multiple poses need to be spliced after registration. Traditional point cloud splicing algorithms are easily affected by initial values and may fall into local optimality when faced with large-scale or complex-structured point clouds. How to design a robust global registration algorithm to achieve efficient and accurate splicing of point cloud data in multiple poses is another core issue in the overall performance of the system.

[0003] Furthermore, binocular multi-line laser 3D reconstruction technology holds significant application value in fields such as industrial inspection and robotic navigation. With the widespread application of binocular line laser-based 3D reconstruction systems in various fields, complex scenarios (such as dynamic environments, multiple occlusions, and scarce textures) are increasingly demanding reconstruction accuracy and real-time performance. However, binocular multi-line laser systems face various challenges in matching multiple laser lines in practical applications. In binocular systems, after preprocessing the images captured by the left and right cameras, the positions of the laser lines can vary due to factors such as viewing angle, lighting, and camera distortion. Single-line laser matching typically uses epipolar constraints to ensure the matching of corresponding laser lines in the left and right images. However, in multi-line laser systems, due to the presence of multiple laser lines in each image and the potential for partial overlap or interference between them, epipolar constraints alone are insufficient to ensure accurate matching. Utilizing additional geometric information and light plane constraints to segment and match laser lines in this multi-line scenario is crucial for improving 3D reconstruction accuracy. Therefore, calculating and optimizing the accuracy of 3D reconstructed point clouds in a single pose is of urgent theoretical and practical significance for in-depth research on multi-pose point cloud global registration techniques. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a binocular multi-line laser three-dimensional global measurement method based on a coded target. Point cloud splicing can be performed through a marker point that does not need to be pasted, thereby improving the three-dimensional reconstruction accuracy in unit pose and the global registration accuracy of multi-pose point clouds.

[0005] To achieve the above-mentioned object of the invention, the present invention provides a binocular multi-line laser three-dimensional global measurement method based on a coded target, characterized in that it includes the following steps:

[0006] (1) Generation of circular coding markers, calibration plates and targets;

[0007] (2) Build an image acquisition device to collect calibration images and reconstruct images;

[0008] (3) System transformation matrix calibration based on ring-coded landmarks;

[0009] (4) Multi-line laser light plane calibration;

[0010] (5) Multi-pose point cloud reconstruction;

[0011] (6) Global registration of point clouds in multiple poses;

[0012] The object of the invention of the present invention is achieved like this:

[0013] The present invention discloses a binocular multi-line laser three-dimensional global measurement method based on a coded target. The method comprises the following steps: first, designing annular coded markers, a calibration plate and a target; then building an image acquisition device, and acquiring calibration images and reconstructed images in multiple postures; then, using the calibration images in multiple postures, completing the system transformation matrix calibration based on the annular coded markers, and realizing multi-line laser light plane calibration through the reconstructed images in multiple postures; finally, realizing multi-pose point cloud reconstruction and point cloud global registration based on the calibration results, thereby improving the three-dimensional reconstruction accuracy in a single posture and the global registration accuracy of the multi-pose point cloud.

[0014] At the same time, the binocular multi-line laser three-dimensional global measurement method based on coded targets of the present invention also has the following beneficial effects:

[0015] (1) A calibration plate and target based on coded markers were designed. By taking optical calibration images of the target, a point cloud global registration system was constructed that does not require markers to be attached and has no contact with the object to be measured, thus expanding the application scenarios of the multi-pose point cloud global registration method.

[0016] (2) By positioning the binocular camera, the position of the scanning binocular camera and the target is obtained in real time, and the transformation relationship between the local coordinate system and the global coordinate system is established, which realizes the marker-free and error-free splicing of multi-view point clouds and optimizes the traditional global registration algorithm's sensitivity to initial pose and error accumulation problems;

[0017] (3) The laser line separation method based on density clustering and the laser line matching method across views are used to eliminate the matching ambiguity of multi-line lasers during calibration. The point cloud reconstruction under a single pose is performed using the triangulation principle and the light plane method and is screened according to the distance threshold, thereby improving the accuracy of three-dimensional reconstruction under a single pose. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1This is a flow chart of a binocular multi-line laser three-dimensional global measurement method based on a coded target according to the present invention;

[0019] Figure 2 It is a structural diagram of the ring-coded landmark points;

[0020] Figure 3 It is a schematic diagram of the structure of the calibration plate and target;

[0021] Figure 4 It is a schematic diagram of the code value distribution of the circular coding mark points in the calibration plate and target;

[0022] Figure 5 is a schematic diagram of an image acquisition device;

[0023] Figure 6 It is a matching diagram of three-dimensional coordinate points. DETAILED DESCRIPTION

[0024] The following describes the specific embodiments of the present invention in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when detailed descriptions of known functions and designs may dilute the main content of the present invention, such descriptions will be omitted here.

[0025] Example

[0026] In this embodiment, if Figure 1 As shown, the present invention provides a binocular multi-line laser three-dimensional global measurement method based on a coded target, comprising the following steps:

[0027] (1) Generation of circular coding markers, calibration plates and targets;

[0028] In this embodiment, if Figure 2 As shown, the annular coding mark point consists of a central circle and an annular coding belt, wherein the radius of the central circle is R, and the inner radius and outer radius of the annular coding belt are 2R and 3R respectively;

[0029] Annular code belt encoding: Divide the annular code belt into 12 areas according to angle, with the angle of each area being 360° / 12. If an area corresponds to white, it is recorded as binary 0; if an area corresponds to black, it is recorded as binary 1, completing the binary encoding of the 12 areas of the annular code belt.

[0030] like Figure 3 As shown, the structures of the calibration plate and the target are both black and white checkerboards, in which the circular coded markers are placed in the white grids, and the code values of all the circular coded markers in the two black and white checkerboards are unique;

[0031] In this embodiment, the structural diagram of the calibration plate is as follows: Figure 3(a) shows the schematic diagram of the target structure. Figure 3 As shown in (b), in this embodiment, the code value of each annular coding mark point can be determined by the following method: taking any area of the annular coding as the starting position, reading a string of binary codes in a clockwise direction, and reading a total of 12 different binary codes according to the different starting positions of the reading areas, and taking the minimum value of the decimal numbers of these 12 binary codes as the code value of the annular coding mark point.

[0032] In this embodiment, if Figure 4 As shown in the figure, the code values of the circular coding mark points in the calibration plate and the target gradually increase from left to right and from top to bottom, where: Figure 4 (a) is a schematic diagram of the arrangement of the circular coding mark points in the calibration plate. Figure 4 (b) is a schematic diagram of the arrangement of circular coding markers in the target.

[0033] (2) Image acquisition;

[0034] (2.1) Build the image acquisition device;

[0035] like Figure 5 As shown, the positioning binocular camera C1 is installed on the fixed device, the scanning binocular camera C2 and the multi-line laser generator are installed on the binocular plate of the handheld device, wherein the scanning binocular camera C2 is fixed to the target, and the multi-line laser generator can generate N c Laser strips;

[0036] Debug the binocular camera C1 and the scanning binocular camera C2 so that the positioning binocular camera C1 can completely and clearly capture the scanning binocular camera C2 and the target, and the scanning binocular camera C2 can completely and clearly capture the calibration plate, and the line laser can be completely and clearly projected onto the surface of the calibration plate;

[0037] (2.2), collect calibration images;

[0038] Keeping cameras C1 and C2 stationary, the operator moves the calibration plate. After moving the calibration plate to each different posture, the operator manually controls the shooting and captures the calibration plate image at the i-th, i=1,2,…,N-th posture. At the same time, the scanning light binocular camera C2 serves as the main camera and trigger source. It sends a trigger signal to the positioning binocular camera C1 at each shooting. The positioning binocular camera C1 immediately shoots after receiving the trigger signal sent by the scanning binocular camera C2. That is, when the scanning binocular camera C2 is in each posture and shoots, the positioning binocular camera C1 captures the target image fixed to the scanning binocular camera C2.

[0039] When collecting calibration images at each pose, keep the relative positions of the positioning binocular camera C1, the scanning binocular camera C2, and the calibration plate unchanged, and collect calibration plate images three times. Each group collects a total of six images. The six calibration plate images collected at the i-th pose are recorded as:

[0040] (2.3) Acquire and reconstruct images;

[0041] Keep the acquisition device unchanged, replace the calibration plate with the object to be measured, and then keep the position of camera C1 and the object to be measured unchanged. The handheld device scans the binocular camera C2, and the operator manually moves C2 to different postures and controls the shooting, and takes images of the object to be measured in multiple postures; when the positioning binocular camera C1 is in each posture of the scanning binocular camera C2 and shooting, it takes an image of the target fixed to the scanning binocular camera C2, and performs three reconstruction image acquisitions. Each group collects a total of six images. The six calibration plate images collected at the i-th posture are recorded as:

[0042] In this embodiment, we describe in detail the acquisition method of the three calibration plate images or reconstructed images, specifically:

[0043] First, collect the image of the calibration plate without laser lines or reconstruct the image: turn off the multi-line laser generator and use only the binocular camera to shoot the calibration plate or the object to be measured. The image obtained by shooting the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as

[0044] Then take the image of the calibration plate with the left-slanted multi-line laser or the reconstructed image: keep the binocular camera, multi-line laser generator and calibration plate or object in place, turn on the multi-line laser generator, project the left-slanted multi-line laser onto the calibration plate or object and collect the image. The image obtained by taking the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as

[0045] Finally, capture the image of the calibration plate with right-slanted multi-line laser or reconstruct the image: keep the binocular camera, multi-line laser generator and calibration plate or object in the same position, change the multi-line laser generator, project the right-slanted multi-line laser onto the calibration plate or object and capture the image. The image obtained by capturing the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as

[0046] (3) System transformation matrix calibration based on ring-coded landmarks;

[0047] (3.1) Calibration plate image preprocessing: grayscale conversion, Gaussian filtering and binarization are performed on each calibration plate image in sequence to obtain a binary calibration plate image;

[0048] (3.2) Extracting contours: Using an edge detection algorithm to perform edge detection on each binary calibration plate image, obtain the contours in each binary calibration plate image, and number the contours in each binary calibration plate image as k, where k = 1, 2, 3, ...;

[0049] (3.3) Contour screening: traverse each contour. If the number of pixels on a contour is less than 50, discard the contour. Otherwise, go to step (3.4).

[0050] (3.4) Ellipse fitting: Use the least square method to fit the contour to obtain the ellipse parameters, where the ellipse parameters obtained for the kth contour are (x k ,y k ,a k ,b k ), where (x k ,y k ) is the coordinate of the center point of the ellipse, (a k ,b k ) are the semi-major and semi-minor axes of the ellipse;

[0051] (3.5) Determine whether the fitted ellipse is valid;

[0052] (3.5.1) Calculate the area of the ellipse: Area k,1 =π·a k b k ;

[0053] (3.5.2) Use Green's formula to calculate the actual area of the kth contour k,2 ;

[0054] (3.5.3), if Area k,1 and Area k,2 The ratio satisfies And the ratio of the minor axis to the major axis of the ellipse satisfies: Then go to step (3.5.4); otherwise, discard the contour and return to step (3.4);

[0055] (3.5.4), with (x k ,y k ) is the center of the circle, and the minimum circumscribed circle is fitted to the kth contour, and the radius R of the minimum circumscribed circle is recorded. k ;

[0056] (3.5.5) Calculate the area of the minimum circumscribed circle Then judge Area k,2 and Area k,3 Does the ratio satisfy If the conditions are met, the ellipse fitting of the kth contour is determined to be valid, and the ellipse fitting parameters (x k ,y k ,a k ,b k ,θ k ), then go to step (3.5.6); otherwise, discard the contour and return to step (3.4) to perform ellipse fitting of the next contour;

[0057] (3.5.6), crop the target image: (x k ,y k ) is the center point, the minimum circumscribed circle radius R k The target image containing the coded landmarks is cropped from the calibration plate image with a unit length. The formula is:

[0058]

[0059] Among them, O k is the coordinate of the upper left corner pixel of the target image, W k ,H k are the width and height of the target image respectively;

[0060] (3.6) Circular coded landmark decoding;

[0061] In the target image, starting from the 0° area, pixel values are extracted every 2° in the first area. If the 2° area is black, the binary value of the area is set to 1, otherwise it is set to 0, thus obtaining a set of 15 binary strings of 0s and 1s. The number of 1s is then counted. If the number of 1s is greater than 8, the decoded value of the first area is recorded as 1, otherwise it is recorded as 0.

[0062] After the decoding of the first area is completed, continue decoding the subsequent areas, and finally obtain the binary codes of 12 areas;

[0063] Increase the starting position by 30°, and then follow step (3.6) to obtain the binary code of the second decoding position. Then, continue in this way, and read a total of 12 different binary codes according to the different starting positions of the reading area. The minimum decimal value of these 12 binary codes is used as the decoding code value of the circular code mark point;

[0064] (3.7) Use the corner detection algorithm to detect the corners on the calibration plate, and then record the sub-pixel coordinates of each corner in multiple poses in the calibration image, as well as the coordinates in the camera coordinate system {L} and the world coordinate system {W};

[0065] (3.8) Traverse each corner point, find the two adjacent ring-coded marker points for each corner point, and sum the code values of the two ring-coded marker points as the code value of the corner point;

[0066] (3.9) According to the coordinates of the corner points with the same code value in multiple poses in the camera coordinate system {L} and the world coordinate system {W}, fit the transformation matrix R from the camera coordinate system to the world coordinate system L,W ;

[0067] (3.10) Use the corner detection algorithm to detect the corners on the target in multiple poses in the calibration image, and record the coordinates of each corner with the same code value in the target coordinate system {T} and the world coordinate system {W};

[0068] According to the transformation matrix R L,W Convert the coordinates of each corner point on the target in the world coordinate system {W} to the coordinates in the camera coordinate system {L};

[0069] Then, according to the coordinates of the corner points with the same code value in the camera coordinate system {L} and the target coordinate system {T}, the transformation matrix R from the camera coordinate system to the target coordinate system {T} is fitted. L,T ;

[0070] (4) Multi-line laser light plane calibration;

[0071] (4.1) Define the neighborhood parameters (ε, MinPts), where ε represents the density radius of the center point of each laser line, which is used to define the neighborhood of a pixel's two-dimensional coordinate point. MinPts represents the minimum number of center points contained in each cluster of laser line center points after clustering is completed.

[0072] (4.2) The calibration plate images taken at different postures Composition of calibration plate image collection

[0073] (4.3) Randomly select a calibration plate image from the calibration plate image set, and use the laser line center point extraction algorithm to extract the center point p of each laser line on the calibration plate image. il , record the pixel coordinates (x l ,y l ), all the center points p il The set of laser line center points P i =(p i1 ,p i2 ,…,p il ,…,p im ), where p il represents the lth center point in the i-th calibration plate image, and m is the number of center points;

[0074] (4.4) Initialize the core object set Ω to an empty set and the number of clusters k = 0;

[0075] (4.5), at the center point set P of the laser line i In the center point p il As a benchmark, calculate p il With the rest of the center points p ih The distance, h=1,2,…,m and h≠l; if the calculated distance value is less than the set threshold, the center point p ih Add to p il Neighborhood subsample set N ε (p il ), and then determine whether the number of samples in the neighborhood subsample set satisfies: |N ε (p il )|≥MinPts, if satisfied, then p il Added as a core object to the core object sample set: Ω = {p il}; Otherwise, discard p il , and then change the next center point until P i The traversal of the center point in is completed;

[0076] (4.6) Determine whether the core object sample set Ω is empty. If Ω is empty, discard the i-th calibration plate image and return to step (4.3) to process the next calibration plate image; otherwise, proceed to step (4.7);

[0077] (4.7) Randomly select a core object p from the core object set Ω il , p il and its neighborhood subsample set N ε (p il ) are added to the same cluster, and the number of clusters is set to k = k + 1;

[0078] Traverse N ε (p il ) in the center point p ih , if p ih If the number of samples in the neighborhood sample set is less than MinPts, then the center point p ih is the cutoff point, and no cluster expansion is performed; otherwise, p ih All center points in the neighborhood sample set are also added to p ih The cluster where it is located, and then continue to expand p ih All the center points in the neighborhood sample set of , and so on;

[0079] (4.8) Traverse the next core object in the core object sample set Ω until all core objects are traversed, and then count the number of clusters. If the number of clusters is N c , then go to step (4.9); otherwise, discard the i-th calibration plate image and return to step (4.3) to process the next calibration plate image;

[0080] (4.9) Output the center point and pixel coordinates of each cluster according to the cluster number, and denote the g-th center point in the c-th cluster as p ig =(x g ,y g ,c), g is the center point number, c is the cluster number, c=1,2,…,N c ;

[0081] (4.10), calculate the centroid coordinates of each cluster (x c ,y c ): The average of the pixel coordinates of all center points in each cluster is taken as the centroid coordinates of the cluster;

[0082] According to the vertical coordinate value of the centroid coordinate, N c The clusters are sorted in descending order and numbered 1, 2, ..., N c Numbering;

[0083] (4.11) Traverse each calibration plate image in the calibration plate image set and process it according to steps (4.3) to (4.10) to obtain laser line clusters of the calibration plate image at different postures;

[0084] In different positions, all the center points of the laser line clusters with the same number are stored in the same set, and a total of N c A collection;

[0085] According to the left camera intrinsic parameter matrix of the scanning binocular camera C2, the pixel coordinates of the center point in each set are converted into the camera coordinate system coordinates, and then the center point in each set is fitted using the least squares method to obtain the left oblique multi-line laser light plane in the camera coordinate system;

[0086] (4.12) The calibration plate images taken at different postures Composition of calibration plate image collection Execute steps (4.3) to (4.11) to obtain the right-slanted multi-line laser light plane in the camera coordinate system;

[0087] (5) Multi-pose point cloud reconstruction;

[0088] (5.1) Reconstructed images taken at different positions Composition of reconstructed image sets

[0089] (5.2) Select a set of reconstructed images from the reconstructed image set right Perform epipolar correction. After correction, the ordinates of each coordinate point satisfy the epipolar geometric constraints:

[0090]

[0091] Among them, F is the 3×3 basic matrix, (x l1 ,y l1 ) is the left image Corrected pixel coordinate value, (x r1 ,y r1 ) is the right image Corrected pixel coordinate values;

[0092] (5.3) Left image after epipolar correction The center point of each laser line is extracted using the laser line center point extraction algorithm. Record each center point in the left image The pixel coordinates (x l1,s ,y l1,s ), all the center points The set of center points of the laser lines in the left image in, Represents the left image The sth center point in , S is the number of center points;

[0093] Similarly, the center point set of the laser line in the right image is

[0094] (5.4), traverse the collection The center point of As a benchmark, in the collection Select and Construct the center point with the same vertical coordinate The set of candidate matching points;

[0095] The triangulation principle is used to calculate the three-dimensional points corresponding to each center point in the candidate matching point set. The calculation formula is as follows:

[0096]

[0097] Among them, f x 、f y are the focal lengths of the left camera in the X and Y axis directions, ρ is the baseline distance, (c x ,c y ) is the coordinate of the left camera's main focus; x r1,s-τ express The horizontal coordinate of the τth candidate point in the candidate matching point set;

[0098] The light plane method is used to calculate the three-dimensional points corresponding to each center point in the candidate matching point set. The calculation formula is as follows:

[0099]

[0100] Among them, Z c is the distance scalar from the target point to the camera optical center along the optical axis, K is the 3×3 left camera intrinsic parameter matrix, R and t are the 3×3 rotation matrix and 3×1 translation vector of the camera extrinsic parameter; A l,m ,B l,m ,C l,m ,D l,m is the parameter of the mth light plane in the left-slanted multi-line laser light plane;

[0101] Finally, the three-dimensional points calculated by the triangulation principle are combined into a three-dimensional point set P tri , the three-dimensional points calculated by the light plane method are combined into a three-dimensional point set P plane ;

[0102] In this embodiment, if Figure 6 As shown, taking 3 groups of laser lines as an example, point p on the left polar line l The points that satisfy the epipolar geometry constraints on the right are Three, of which obviously is the correct matching point, and is a mismatched point.

[0103] (5.5), with the set P tri As the benchmark, calculate the relationship between each 3D point and the set P plane The distance d between each three-dimensional point in a,b :

[0104]

[0105] in, Represent the set P respectively tri The coordinate value of the a-th three-dimensional point in , Represent the set P respectively plane The coordinate value of the b-th three-dimensional point in ;

[0106] (5.6), select the minimum distance min(d a,b ), judge min(d a,b ) is less than the set threshold ξ, if d a,b <ξ, then the three-dimensional point Add the multi-pose left-slanted 3D point cloud set in the camera coordinate system {L}, otherwise discard it;

[0107] (5.7) Traverse each set of reconstructed images in the reconstructed image set to obtain a set of multi-pose left-oblique three-dimensional point clouds in the camera coordinate system {L};

[0108] (5.8) Reconstructed images taken at different positions Composition of reconstructed image sets Execute steps (5.2) to (5.7) to obtain a multi-pose right-oblique 3D point cloud set in the camera coordinate system {L};

[0109] (6) Global registration of point clouds in multiple poses;

[0110] (6.1) Reconstructed images taken at different positions Composition of reconstructed image sets

[0111] (6.2) Use the corner detection algorithm to detect the corners of each reconstructed image at multiple positions, and then record the coordinates of each corner with the same code value in the target coordinate system {T} and the world coordinate system {W};

[0112] According to the coordinates of the corner points with the same code value under different postures, the transformation matrix R from the target coordinate system {T} to the world coordinate system {W} is fitted T,W ;

[0113] The transformation matrix R T,W With the transformation matrix R L,T Multiply them to get the transformation matrix R from the camera coordinate system {L} to the world coordinate system {W} L,W =R L,T ·R T,W ;

[0114] (6.3) Convert each 3D point in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set in the camera coordinate system {L} from the camera coordinate system {L} to the world coordinate system {W}, that is:

[0115]

[0116] in, They represent the world coordinate system {W} coordinates of the t-th 3D point in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set after transformation, They represent the {L} coordinates of the t-th 3D point in the camera coordinate system in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set respectively;

[0117] Traverse the 3D point clouds in all poses and complete the global registration of point clouds in multiple poses.

[0118] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

Claims

1. A binocular multi-line laser three-dimensional global measurement method based on a coded target, characterized in that: The following steps are involved: (1) Generation of circular coding markers, calibration plates and targets; The annular coding mark point consists of a central circle and an annular coding belt, where the radius of the central circle is R, and the inner and outer radii of the annular coding belt are 2R and 3R respectively; Annular code belt encoding: Divide the annular code belt into 12 areas according to angle, with the angle of each area being 360° / 12. If an area corresponds to white, it is recorded as binary 0; if an area corresponds to black, it is recorded as binary 1, completing the binary encoding of the 12 areas of the annular code belt. The calibration plate and target are both constructed with black and white checkerboards, where circular coded markers are placed within the white grids. The code values of all circular coded markers within the two black and white checkerboards are unique. (2) Image acquisition; (2.1) Build the image acquisition device; The positioning binocular camera C1 is installed on the fixed device, and the scanning binocular camera C2 and the multi-line laser generator are installed on the binocular plate of the handheld device. The scanning binocular camera C2 is fixed to the target, and the multi-line laser generator can generate N c Laser strips; Debug the binocular camera C1 and the scanning binocular camera C2 so that the positioning binocular camera C1 can completely and clearly capture the scanning binocular camera C2 and the target, and the scanning binocular camera C2 can completely and clearly capture the calibration plate, and the line laser can be completely and clearly projected onto the surface of the calibration plate; (2.2), collect calibration images; Keeping cameras C1 and C2 stationary, the operator moves the calibration plate. After moving the calibration plate to each different posture, the operator manually controls the shooting and captures the calibration plate image at the i-th, i=1,2,…,N-th posture. At the same time, the scanning light binocular camera C2 serves as the main camera and trigger source. It sends a trigger signal to the positioning binocular camera C1 at each shooting. The positioning binocular camera C1 immediately shoots after receiving the trigger signal sent by the scanning binocular camera C2. That is, when the scanning binocular camera C2 is in each posture and shoots, the positioning binocular camera C1 captures the target image fixed to the scanning binocular camera C2. When collecting calibration images at each pose, keep the relative positions of the positioning binocular camera C1, the scanning binocular camera C2, and the calibration plate unchanged, and collect calibration plate images three times. Each group collects a total of six images. The six calibration plate images collected at the i-th pose are recorded as: (2.3) Acquire and reconstruct images; Keep the acquisition device unchanged, replace the calibration plate with the object to be measured, and then keep the position of camera C1 and the object to be measured unchanged. The handheld device scans the binocular camera C2, and the operator manually moves C2 to different postures and controls the shooting, and takes images of the object to be measured in multiple postures; when the positioning binocular camera C1 is in each posture of the scanning binocular camera C2 and shooting, it takes an image of the target fixed to the scanning binocular camera C2, and performs three reconstruction image acquisitions. Each group collects a total of six images. The six calibration plate images collected at the i-th posture are recorded as: (3) System transformation matrix calibration based on ring-coded landmarks; (3.1) Calibration plate image preprocessing: grayscale conversion, Gaussian filtering and binarization are performed on each calibration plate image in sequence to obtain a binary calibration plate image; (3.2) Extracting contours: Using an edge detection algorithm to perform edge detection on each binary calibration plate image, obtain the contours in each binary calibration plate image, and number the contours in each binary calibration plate image as k, where k = 1, 2, 3, ...; (3.3) Contour screening: traverse each contour. If the number of pixels on a contour is less than 50, discard the contour. Otherwise, go to step (3.4). (3.4) Ellipse fitting: Use the least square method to fit the contour to obtain the ellipse parameters, where the ellipse parameters obtained for the kth contour are (x k ,y k ,a k ,b k ), where (x k ,y k ) is the coordinate of the center point of the ellipse, (a k ,b k ) are the semi-major and semi-minor axes of the ellipse; (3.5) Determine whether the fitted ellipse is valid; (3.5.1) Calculate the area of the ellipse: Area k,1 =π·a k b k ; (3.5.2) Use Green's formula to calculate the actual area of the kth contour k,2 ; (3.5.3), if Area k,1 and Area k,2 The ratio satisfies And the ratio of the minor axis to the major axis of the ellipse satisfies: Then go to step (3.5.4); otherwise, discard the contour and return to step (3.4); (3.5.4), with (x k ,y k ) is the center of the circle, and the minimum circumscribed circle is fitted to the kth contour, and the radius R of the minimum circumscribed circle is recorded. k ; (3.5.5) Calculate the area of the minimum circumscribed circle Then judge Area k,2 and Area k,3 Does the ratio satisfy If the conditions are met, the ellipse fitting of the kth contour is determined to be valid, and the ellipse fitting parameters (x k ,y k ,a k ,b k ,θ k ), then go to step (3.5.6); otherwise, discard the contour and return to step (3.4) to perform ellipse fitting of the next contour; (3.5.6), crop the target image: (x k ,y k ) is the center point, the minimum circumscribed circle radius R k The target image containing the coded landmarks is cropped from the calibration plate image with a unit length. The formula is: Among them, O k is the coordinate of the upper left corner pixel of the target image, W k ,H k are the width and height of the target image respectively; (3.6) Circular coded landmark decoding; In the target image, starting from the 0° area, pixel values are extracted every 2° in the first area. If the 2° area is black, the binary value of the area is set to 1, otherwise it is set to 0, thus obtaining a set of 15 binary strings of 0s and 1s. The number of 1s is then counted. If the number of 1s is greater than 8, the decoded value of the first area is recorded as 1, otherwise it is recorded as 0. After the decoding of the first area is completed, continue decoding the subsequent areas, and finally obtain the binary codes of 12 areas; Increase the starting position by 30°, and then follow step (3.6) to obtain the binary code of the second decoding position. Then, continue in this way, and read a total of 12 different binary codes according to the different starting positions of the reading area. The minimum decimal value of these 12 binary codes is used as the decoding code value of the circular code mark point; (3.7) Use the corner detection algorithm to detect the corners on the calibration plate, and then record the sub-pixel coordinates of each corner in multiple poses in the calibration image, as well as the coordinates in the camera coordinate system {L} and the world coordinate system {W}; (3.8) Traverse each corner point, find the two adjacent ring-coded marker points for each corner point, and sum the code values of the two ring-coded marker points as the code value of the corner point; (3.9) According to the coordinates of the corner points with the same code value in multiple poses in the camera coordinate system {L} and the world coordinate system {W}, fit the transformation matrix R from the camera coordinate system to the world coordinate system L,W ; (3.10) Use the corner detection algorithm to detect the corners on the target in multiple poses in the calibration image, and record the coordinates of each corner with the same code value in the target coordinate system {T} and the world coordinate system {W}; According to the transformation matrix R L,W Convert the coordinates of each corner point on the target in the world coordinate system {W} to the coordinates in the camera coordinate system {L}; Then, according to the coordinates of the corner points with the same code value in the camera coordinate system {L} and the target coordinate system {T}, the transformation matrix R from the camera coordinate system to the target coordinate system {T} is fitted. L,T ; (4) Multi-line laser light plane calibration; (4.1) Define the neighborhood parameters (ε, MinPts), where ε represents the density radius of the center point of each laser line, which is used to define the neighborhood of a pixel's two-dimensional coordinate point. MinPts represents the minimum number of center points contained in each cluster of laser line center points after clustering is completed. (4.2) The calibration plate images taken at different postures Composition of calibration plate image collection (4.3) Randomly select a calibration plate image from the calibration plate image set, and use the laser line center point extraction algorithm to extract the center point p of each laser line on the calibration plate image. il , record the pixel coordinates (x l ,y l ), all the center points p il The set of laser line center points P i =(p i1 ,p i2 ,…,p il ,…,p im ), where p il represents the lth center point in the i-th calibration plate image, and m is the number of center points; (4.4) Initialize the core object set Ω to an empty set and the number of clusters k = 0; (4.5), at the center point set P of the laser line i In the center point p il As a benchmark, calculate p il With the rest of the center points p ih The distance, h=1,2,…,m and h≠l; if the calculated distance value is less than the set threshold, the center point p ih Add to p il Neighborhood subsample set N ε (p il ), and then determine whether the number of samples in the neighborhood subsample set satisfies: |N ε (p il )|≥MinPts, if satisfied, then p il Added as a core object to the core object sample set: Ω = {p il }; Otherwise, discard p il , and then change the next center point until P i The traversal of the center point in is completed; (4.6) Determine whether the core object sample set Ω is empty. If Ω is empty, discard the i-th calibration plate image and return to step (4.3) to process the next calibration plate image; otherwise, proceed to step (4.7); (4.7) Randomly select a core object p from the core object set Ω il , p il and its neighborhood subsample set N ε (p il ) are added to the same cluster, and the number of clusters is set to k = k + 1; Traverse N ε (p il ) in the center point p ih , if p ih If the number of samples in the neighborhood sample set is less than MinPts, then the center point p ih is the cutoff point, and no cluster expansion is performed; otherwise, p ih All center points in the neighborhood sample set are also added to p ih The cluster where it is located, and then continue to expand p ih All the center points in the neighborhood sample set of , and so on; (4.8) Traverse the next core object in the core object sample set Ω until all core objects are traversed, and then count the number of clusters. If the number of clusters is N c , then go to step (4.9); otherwise, discard the i-th calibration plate image and return to step (4.3) to process the next calibration plate image; (4.9) Output the center point and pixel coordinates of each cluster according to the cluster number, and denote the g-th center point in the c-th cluster as p ig =(x g ,y g ,c), g is the center point number, c is the cluster number, c=1,2,…,N c ; (4.10), calculate the centroid coordinates of each cluster (x c ,y c ): The average of the pixel coordinates of all center points in each cluster is taken as the centroid coordinates of the cluster; According to the vertical coordinate value of the centroid coordinate, N c The clusters are sorted in descending order and numbered 1, 2, ..., N c Numbering; (4.11) Traverse each calibration plate image in the calibration plate image set and process it according to steps (4.3) to (4.10) to obtain laser line clusters of the calibration plate image at different postures; In different positions, all the center points of the laser line clusters with the same number are stored in the same set, and a total of N c A collection; According to the left camera intrinsic parameter matrix of the scanning binocular camera C2, the pixel coordinates of the center point in each set are converted into the camera coordinate system coordinates, and then the center point in each set is fitted using the least squares method to obtain the left oblique multi-line laser light plane in the camera coordinate system; (4.12) The calibration plate images taken at different postures Composition of calibration plate image collection Execute steps (4.3) to (4.11) to obtain the right-slanted multi-line laser light plane in the camera coordinate system; (5) Multi-pose point cloud reconstruction; (5.1) Reconstructed images taken at different positions Composition of reconstructed image sets (5.2) Select a set of reconstructed images from the reconstructed image set right Perform epipolar correction. After correction, the ordinates of each coordinate point satisfy the epipolar geometric constraints: Among them, F is the 3×3 basic matrix, (x l1 ,y l1 ) is the left image Corrected pixel coordinate value, (x r1 ,y r1 ) is the right image Corrected pixel coordinate values; (5.3) Left image after epipolar correction The center point of each laser line is extracted using the laser line center point extraction algorithm. Record each center point in the left image The pixel coordinates (x l1,s ,y l1,s ), all the center points The set of center points of the laser lines in the left image in, Represents the left image The sth center point in , S is the number of center points; Similarly, the center point set of the laser line in the right image is (5.4), traverse the collection The center point of As a benchmark, in the collection Select and Construct the center point with the same vertical coordinate The set of candidate matching points; The triangulation principle is used to calculate the three-dimensional points corresponding to each center point in the candidate matching point set. The calculation formula is as follows: Among them, f x 、f y are the focal lengths of the left camera in the X and Y axis directions, ρ is the baseline distance, (c x ,c y ) is the coordinate of the left camera's main focus; x r1,s-τ express The horizontal coordinate of the τth candidate point in the candidate matching point set; The light plane method is used to calculate the three-dimensional points corresponding to each center point in the candidate matching point set. The calculation formula is as follows: Among them, Z c is the distance scalar from the target point to the camera optical center along the optical axis, K is the 3×3 left camera intrinsic parameter matrix, R and t are the 3×3 rotation matrix and 3×1 translation vector of the camera extrinsic parameter; A l,m ,B l,m ,C l,m ,D l,m is the parameter of the mth light plane in the left-slanted multi-line laser light plane; Finally, the three-dimensional points calculated by the triangulation principle are combined into a three-dimensional point set P tri , the three-dimensional points calculated by the light plane method are combined into a three-dimensional point set P plane ; (5.5), with the set P tri As the benchmark, calculate the relationship between each 3D point and the set P plane The distance d between each three-dimensional point in a,b : in, Represent the set P respectively tri The coordinate value of the a-th three-dimensional point in , Represent the set P respectively plane The coordinate value of the b-th three-dimensional point in ; (5.6), select the minimum distance min(d a,b ), judge min(d a,b ) is less than the set threshold ξ, if d a,b <ξ, then the three-dimensional point Add the multi-pose left-slanted 3D point cloud set in the camera coordinate system {L}, otherwise discard it; (5.7) Traverse each set of reconstructed images in the reconstructed image set to obtain a set of multi-pose left-oblique three-dimensional point clouds in the camera coordinate system {L}; (5.8) Reconstructed images taken at different positions Composition of reconstructed image sets Execute steps (5.2) to (5.7) to obtain a multi-pose right-oblique 3D point cloud set in the camera coordinate system {L}; (6) Global registration of point clouds in multiple poses; (6.1) Reconstructed images taken at different positions Composition of reconstructed image sets (6.2) Use the corner detection algorithm to detect the corners of each reconstructed image at multiple positions, and then record the coordinates of each corner with the same code value in the target coordinate system {T} and the world coordinate system {W}; According to the coordinates of the corner points with the same code value under different postures, the transformation matrix R from the target coordinate system {T} to the world coordinate system {W} is fitted T,W ; The transformation matrix R T,W With the transformation matrix R L,T Multiply them to get the transformation matrix R from the camera coordinate system {L} to the world coordinate system {W} L,W =R L,T ·R T,W ; (6.3) Convert each 3D point in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set in the camera coordinate system {L} from the camera coordinate system {L} to the world coordinate system {W}, that is: in, They represent the world coordinate system {W} coordinates of the t-th 3D point in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set after transformation, They represent the {L} coordinates of the t-th 3D point in the camera coordinate system in the multi-pose left-oblique 3D point cloud set and the multi-pose right-oblique 3D point cloud set respectively; Traverse the 3D point clouds in all poses and complete the global registration of point clouds in multiple poses.

2. The binocular multi-line laser three-dimensional global measurement method based on a coded target according to claim 1, characterized in that: The acquisition method of the three calibration plate images or the reconstructed images is: First, collect the image of the calibration plate without laser lines or reconstruct the image: turn off the multi-line laser generator and use only the binocular camera to shoot the calibration plate or the object to be measured. The image obtained by shooting the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as Then take the image of the calibration plate with the left-slanted multi-line laser or the reconstructed image: keep the binocular camera, multi-line laser generator and calibration plate or object in place, turn on the multi-line laser generator, project the left-slanted multi-line laser onto the calibration plate or object and collect the image. The image obtained by taking the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as Finally, capture the image of the calibration plate with right-slanted multi-line laser or reconstruct the image: keep the binocular camera, multi-line laser generator and calibration plate or object in the same position, change the multi-line laser generator, project the right-slanted multi-line laser onto the calibration plate or object and capture the image. The image obtained by capturing the calibration plate is recorded as The image obtained by shooting the object to be measured is recorded as 3. The binocular multi-line laser three-dimensional global measurement method based on a coded target according to claim 1, characterized in that: The code value of the circular coding mark point is: taking any area of the circular coding as the starting position, a string of binary codes is read in a clockwise direction. Depending on the starting position of the reading area, a total of 12 different binary codes are read, and the minimum value of the decimal numbers of these 12 binary codes is used as the code value of the circular coding mark point.

4. The binocular multi-line laser three-dimensional global measurement method based on a coded target according to claim 1, characterized in that: The code values of the annular coding marking points in the calibration plate and the target gradually increase from left to right and from top to bottom.

Images