Multi-line laser three-dimensional reconstruction method based on optical tracking

Abstract

The invention belongs to the technical field of laser three-dimensional reconstruction, and discloses a multi-line laser three-dimensional reconstruction method based on optical tracking. A binocular tracking system is erected outside a traditional multi-line laser system, and a set of tracking reflective balls is installed on the multi-line laser system. Through the binocular tracking system, accurate three-dimensional reconstruction of the tracking reflective ball can be realized, and unification of a coordinate system is further realized. According to the method, the accuracy of three-dimensional reconstruction is remarkably improved, the application range of the three-dimensional reconstruction technology in the industrial field is greatly expanded due to high efficiency and accuracy, and new possibility is brought to the fields of industrial automation, quality detection and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of laser three-dimensional reconstruction, and particularly relates to a multi-line laser three-dimensional reconstruction method based on optical tracking. BACKGROUND

[0002] In the process of multi-line laser three-dimensional reconstruction, a technical means based on marker point splicing is usually adopted, and the main purpose of this method is to realize accurate extraction of three-dimensional data of the entire object surface. The specific operation steps are to reconstruct three-dimensional data information of the calibration point by using a binocular camera system. Then, by applying a marker point-based splicing algorithm, the three-dimensional data obtained by multi-line laser scanning from different perspectives and poses can be effectively spliced. The key of this process is to ensure that the coordinate systems between different data sets are unified, so as to ensure that the three-dimensional model finally reconstructed has accurate geometric structure and size. In actual engineering implementation, we usually need to paste marker points on the object, but in many industrial scenes, it is impossible to realize the unification of the coordinate system by pasting the calibration method on the object.

[0003] In the process of joint calibration, the transformation relationship [R, T] between multiple coordinate systems needs to be calculated, which mainly involves the rotation matrix R and the translation matrix T between two coordinate systems. Let the point sets in two coordinate systems be P' = {p1p2...p n} and Q' = {q1 q2...q n}. The least squares objective function can be constructed to solve [R, T], and the final purpose is to transform the three-dimensional point set in one coordinate system to another coordinate system, i.e. P = R*Q + T, and the error equation is constructed as:

[0004]

[0005] The purpose of joint calibration is to calibrate the transformation relationship m of the multi-line laser system coordinate system O b to the tracking sphere cage coordinate system O

[0006] However, the above joint calibration method has low precision and is not fast. SUMMARY

[0007] To overcome the shortcomings of existing technologies, this invention provides a multi-line laser 3D reconstruction method based on optical tracking. A binocular tracking system is installed outside the traditional multi-line laser system, and a tracking reflector sphere is mounted on the multi-line laser system. Through the binocular tracking system, accurate 3D reconstruction of the tracking reflector sphere can be achieved, thereby unifying the coordinate system. This method not only significantly improves the accuracy of 3D reconstruction but also greatly expands the application scope of 3D reconstruction technology in the industrial field due to its efficiency and accuracy, bringing new possibilities to fields such as industrial automation and quality inspection.

[0008] The above-mentioned objective of this invention is achieved through the following technical solution: a multi-line laser 3D reconstruction method based on optical tracking, which uses an externally mounted binocular tracking system and a multi-line laser system with a tracking reflective sphere for 3D reconstruction, the steps of which are as follows:

[0009] 1. Collect and reconstruct the marker points into three-dimensional coordinates, and calculate the transformation relationship using the three-dimensional coordinates;

[0010] 2. Change the position of the multi-line laser system, obtain all transformation relationships, form a transformation matrix, and complete the joint calibration;

[0011] 3. Perform 3D reconstruction of the object using the jointly calibrated transformation matrix.

[0012] Furthermore, step 1 is detailed as follows:

[0013] 1.1. Place the crosshair marker within the calibration field of view, acquire images of the marker using a binocular tracking system, and reconstruct all coded marker points to obtain the coordinates of the coded points in the binocular tracking coordinate system O. t The three-dimensional point P below t Then, the coordinate system O of the benchmark is calculated based on the three-dimensional coordinates of the coded points on the benchmark. c To the binocular tracking coordinate system O t Transformation relationship The three-dimensional point representation of the crosshair in the binocular tracking coordinate system is as follows:

[0014] 1.2. Keeping the crosshair stationary, place the multi-line laser system within the field of view, ensuring it can acquire an image of the crosshair. Use the multi-line laser system to reconstruct the coded marker points on the crosshair, obtaining the crosshair's position in the multi-line laser system coordinate system O. m The three-dimensional coordinates P below m Then, based on the coded markers on the marker, the coordinate system O of the marker can be calculated. c To the coordinates O of the multi-line laser system m Transformation relationship Right now

[0015] 1.3. After the completion of step 1.2, keep the multi-line laser system still, use the binocular tracking system to collect the image of the tracking ball cage, calculate the three-dimensional coordinates Q t (x t ,y i ,z i ) of the ball cage in the binocular tracking coordinate system O i , i = 0...n, and calculate the conversion relationship between the ball cage coordinate system O b and the binocular tracking system O t

[0016] 1.4. According to the and calculated in steps 1.2 and 1.3, and the to-be-solved , convert the three-dimensional points on the cross marker rod to the binocular tracking coordinate system, that is,

[0017] Further, the step 2 is specifically:

[0018] According to the calibration steps of step 1, change the position of the multi-line laser system, and at each position, the conversion relationship between the marker rod coordinate and the multi-line laser coordinate and the conversion relationship between the tracking ball cage coordinate system and the binocular tracking coordinate system

[0019]

[0020] According to formula (2), the three-dimensional coordinates P c on the cross marker rod can be converted to P t by the binocular tracking system, and can be converted to P t ′ by the multi-line laser system, since P t and P t ′ are both three-dimensional points on the cross marker rod converted to the same coordinate system, thus P t = P t ′. Combined with formula (2), the following can be obtained:

[0021]

[0022] In formula (3), the to-be-solved parameter is , and multiple and can be obtained by placing the multi-line laser system at multiple positions in the field of view

[0023] ​

[0024] After placing the cross-bar in different positions in the field of view, the conversion matrix in different positions can be obtained, i.e. and Then, according to the three-dimensional points on the cross-bar and the pose conversion matrix in different positions, the conversion matrix from the multi-line laser system coordinate system to the tracking ball coordinate system is calculated by substituting formula (4) Thus, the joint calibration of the tracking system is completed.

[0025] Further, the step 3 is specifically: according to the conversion matrix calculated by joint calibration The three-dimensional points reconstructed by the multi-line laser system are converted to the binocular tracking coordinate system. The binocular tracking system is fixed in a specific region of the measured object, and the multi-line laser system is linked. When the multi-line laser system is used to collect data, the binocular tracking system is used to collect tracking ball images synchronously. According to the three-dimensional coordinates Q b (x b ,y i ,z i ) of the marker points in the tracking ball coordinate system O i , i = 0... n, and the three-dimensional coordinates Q ti (x i ,y i ,z i ) of the tracking ball marker points reconstructed by the binocular tracking system, the conversion matrix

[0026]

[0027] According to the results of joint calibration and the at position i calculated according to formula (4), the three-dimensional data reconstructed by the multi-line laser system can be converted to the binocular tracking coordinate system:

[0028]

[0029] In the process of three-dimensional reconstruction, multi-line structured light technology acquires three-dimensional data by emitting multiple laser lines to the target object and using the projection of these laser lines. In order to reconstruct the complete three-dimensional model of a large object, it is necessary to reconstruct three-dimensional data at different positions multiple times and splice these data. For this purpose, a binocular tracking system is installed in front of the object, which can detect the accurate position and direction of the multi-line laser at each position in the test scene. By applying formula (6), the data reconstructed by the multi-line laser system at different positions can be integrated into a coordinate system, realizing the three-dimensional reconstruction of the entire object.

[0030] The present application has the beneficial effect of providing a multi-line laser three-dimensional reconstruction method based on optical tracking, which significantly improves the accuracy of three-dimensional reconstruction, and due to its high efficiency and accuracy, greatly expands the application range of three-dimensional reconstruction technology in the industrial field, bringing new possibilities for industrial automation and quality detection. BRIEF DESCRIPTION OF DRAWINGS

[0031] The present application will be further described below in conjunction with the drawings and specific embodiments

[0032] Figure 1 is a schematic diagram of the tracking system calibration process in the multi-line laser three-dimensional reconstruction method based on optical tracking of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described below in conjunction with the drawings and specific embodiments

[0034] Example 1

[0035] An innovative multi-line laser three-dimensional reconstruction system based on optical tracking technology. Specifically, the system sets up a binocular tracking system outside the traditional multi-line laser system and installs a tracking reflector ball on the multi-line laser system. Through the binocular tracking system, accurate three-dimensional reconstruction of the tracking reflector ball can be achieved, and the coordinate system can be unified.

[0036] After the binocular tracking system is calibrated, joint calibration of the tracking sphere and the multi-line laser system is necessary to achieve markerless multi-line laser stitching based on this system. The main purpose of joint calibration is to determine the transformation relationship between the two coordinate systems, so as to map the three-dimensional coordinates of the reflective markers in the tracking sphere coordinate system to the multi-line laser system coordinate system. A set of tracking sphere hardware composed of multiple polyhedra is installed on the multi-line laser system equipment, with circular markers with special reflective material affixed to each face. These circular markers can obtain their three-dimensional positions in the tracking sphere cage coordinate system. Through multi-view photogrammetry, the three-dimensional coordinates of all markers in the tracking sphere coordinate system can be obtained and transformed into the tracking sphere cage coordinate system. During the joint calibration of the tracking sphere and the multi-line laser system, the calibrated binocular tracking system and the multi-line laser system should be placed together within the field of view to ensure that the crosshair target can be observed simultaneously. The established coordinate systems are: binocular tracking coordinate system O. t Multi-line laser coordinate system O m Tracking the spherical cage coordinate system O b Crosshair coordinate system O c .

[0037] A calibration method based on a crosshair, such as Figure 1 As shown, crosshairs with multiple coded markers are placed within the designated market area. Each coded marker on the crosshair has a unique number, ensuring that the code value of each marker is unique at any angle and position when the crosshairs are placed. In the crosshair coordinate system O... c The three-dimensional coordinates of all the coding points are P c (x i ,y i ,z i ), i = 0...n. The coordinate system O in the sphere cage is reconstructed beforehand based on multiple views. b The three-dimensional coordinates of all the marked points below are Q. b (x i ,y i ,z i The specific calibration steps for i = 0...n are as follows:

[0038] 1: For example Figure 1 As shown, the crosshair marker is placed within the calibration field of view. A binocular tracking system is used to acquire images of the marker and reconstruct all coded marker points to obtain the coordinates of the coded points in the binocular tracking coordinate system O. t The three-dimensional point P below t Then, the coordinate system O of the benchmark is calculated based on the three-dimensional coordinates of the coded points on the benchmark. c To the binocular tracking coordinate system O t Transformation relationship The three-dimensional point of the crosshair in the binocular tracking coordinate system can be represented as:

[0039] 2. Keeping the crosshair stationary, place the multi-line laser system within the field of view, ensuring it can acquire an image of the crosshair. Use the multi-line laser system to reconstruct the coded marker points on the crosshair, obtaining the crosshair's position in the multi-line laser system's coordinate system O. m The three-dimensional coordinates P below m Then, based on the coded markers on the marker, the coordinate system O of the marker can be calculated. c To the coordinates O of the multi-line laser system m Transformation relationship Right now

[0040] 3: After completing the second step, keep the multi-line laser system stationary and use a binocular tracking system to acquire images of the ball cage. Calculate the position of the ball cage in the binocular tracking coordinate system O based on the parameters of the binocular tracking system. t The three-dimensional coordinates Q t (x i ,y i ,z i ), i = 0...n, and the coordinate system O of the ball cage can be calculated based on the three-dimensional coordinates of all marked points on the ball cage. b To binocular tracking system O t Conversion relationship Right now

[0041] 4: Calculated based on steps 2 and 3 and And what is pending The three-dimensional points on the crosshair can be transformed into the binocular tracking coordinate system using these two transformation matrices.

[0042] Based on the calibration steps above, by changing the position of the multi-line laser system, the transformation relationship from the benchmark coordinates to the multi-line laser coordinates can be obtained at each position. And the transformation relationship from the tracking cage coordinate system to the binocular tracking coordinate system.

[0043]

[0044] According to formula (2), the three-dimensional coordinates P on the crosshair are... c It can be converted to P through a binocular tracking system t It can also be converted to P through a multi-line laser system. t ′, due to P t and P tare all the three-dimensional points on the cross marker converted to the same coordinate system, so there are P t t ′. Combined with equation (2), we can get:

[0045]

[0046] In equation (3), P is the parameter to be solved, and multiple positions of the multi-line laser system in the field of view can be used to solve multiple P and P

[0047]

[0048] After placing the cross marker at different positions in the field of view, the conversion matrix at different positions can be obtained, that is, P and P Then, according to the three-dimensional points on the cross marker and the pose conversion matrix at different positions, equation (4) is used to calculate the conversion matrix of the multi-line laser system coordinate system to the tracking ball coordinate system P Thus, the joint calibration of the tracking system is completed.

[0049] After the joint calibration of the multi-line laser system and the tracking ball is completed, the conversion matrix P is obtained, and the three-dimensional points reconstructed by the multi-line laser system P are converted to the binocular tracking coordinate system. The binocular tracking system is fixed in a specific area of the measured object, and the multi-line laser system is linked. When the multi-line laser system is used to collect data, the binocular tracking system is used to collect tracking ball images synchronously. According to the three-dimensional coordinates of the marker points Q b (x b ,y i ,z i ), i = 0... n in the tracking ball coordinate system O i , and the three-dimensional coordinates of the tracking ball marker points Q ti (x i ,y i ,z i ), i = 0... m reconstructed by the binocular tracking system, the conversion matrix P

[0050]

[0051] According to the joint calibration results P and P ​The three-dimensional data reconstructed by the multi-line laser system can be converted to the binocular tracking coordinate system:

[0052]

[0053] In the three-dimensional reconstruction process, the multi-line structured light technology acquires three-dimensional data by emitting a plurality of laser lines to a target object and using the projection of the laser lines. In order to reconstruct a complete three-dimensional model of a large object, the three-dimensional data must be reconstructed multiple times at different positions and then spliced together. For this purpose, a binocular tracking system is installed in front of the object, which can detect the accurate position and direction of the multi-line laser at each position in the test scene. By applying equation (6), the data reconstructed by the multi-line laser system at different positions can be integrated into a coordinate system, and the three-dimensional reconstruction of the entire object can be achieved.

[0054] The above-described embodiments are merely preferred embodiments of the present application and are not all the embodiments that can be implemented by the present application. Any obvious modifications made by those skilled in the art without departing from the principles and spirit of the present application should be considered to be within the scope of the claims of the present application.

Claims

1. A multi-line laser three-dimensional reconstruction method based on optical tracking, characterized in that, A binocular tracking system and a multi-line laser system with a tracking reflective sphere were externally installed for 3D reconstruction. The steps were as follows: S1. Collect and reconstruct the marker points into three-dimensional coordinates, and calculate the transformation relationship using the three-dimensional coordinates; S2. Change the position of the multi-line laser system, obtain all transformation relationships, form a transformation matrix, and complete the joint calibration; S3. Perform 3D reconstruction of the object using the jointly calibrated transformation matrix.

2. The optical tracking based multi-line laser three-dimensional reconstruction method according to claim 1, wherein, Step S1 specifically involves: S1.

1. Place the crosshair marker within the calibration field of view, use a binocular tracking system to acquire images of the marker, and reconstruct all coded marker points to obtain the coordinates of the coded points in the binocular tracking coordinate system O. t The three-dimensional point P below t Then, the coordinate system O of the benchmark is calculated based on the three-dimensional coordinates of the coded points on the benchmark. c To the binocular tracking coordinate system O t Transformation relation T c t The three-dimensional point representation of the crosshair in the binocular tracking coordinate system is as follows: S1.

2. Keeping the crosshair stationary, place the multi-line laser system within the field of view, ensuring it can acquire an image of the crosshair. Use the multi-line laser system to reconstruct the coded marker points on the crosshair, obtaining the crosshair's coordinates in the multi-line laser system coordinate system O. m The three-dimensional coordinates P below m Then, the coordinate system O of the benchmark can be calculated based on the coded markers on the benchmark. c To the coordinates O of the multi-line laser system m Transformation relationship Right now S1.

3. After completing step S1.2, keep the multi-line laser system stationary, use a binocular tracking system to acquire images of the tracking ball cage, and calculate the position of the ball cage in the binocular tracking coordinate system O based on the parameters of the binocular tracking system. t The three-dimensional coordinates Q t (x i ,y i ,z i ), i = 0...n, and calculate the coordinate system O of the ball cage based on the three-dimensional coordinates of all marked points on the ball cage. b To binocular tracking system O t Conversion relationship Right now S1.

4. Calculated based on steps S1.2 and S1.3 and And what is pending The three-dimensional points on the crosshair are transformed into the binocular tracking coordinate system using these two transformation matrices.

3. The multi-line laser 3D reconstruction method based on optical tracking according to claim 2, characterized in that, Step S2 specifically involves: Following the calibration steps described in step 1, by changing the position of the multi-line laser system, the transformation relationship from the benchmark coordinates to the multi-line laser coordinates can be obtained at each position. And the transformation relationship from the tracking cage coordinate system to the binocular tracking coordinate system. According to formula (2), the three-dimensional coordinates P on the cross staff c are converted to P by the binocular tracking system t , and are converted to P by the multi-line laser system t ′, P t and P t ′ are both three-dimensional points on the cross staff converted to the same coordinate system, so P t =P t ′; combined with formula (2), the following can be obtained: In formula (3) To determine the parameters, the multi-line laser system is placed at multiple positions within the field of view, and multiple [parameters] are calculated. and After placing the crosshair at different positions within the field of view, the transformation matrix at each position is obtained. and Then, based on the three-dimensional points on the crosshair and the pose transformation matrix at different positions, the transformation matrix from the multi-line laser system coordinate system to the tracking spherical coordinate system is calculated using formula (4). The joint calibration of the tracking system has now been completed.

4. The multi-line laser three-dimensional reconstruction method based on optical tracking according to claim 3, characterized in that, Step S3 specifically involves: Based on the jointly calibrated transformation matrix Reconstructing three-dimensional points from a multi-line laser system The object is reconstructed in 3D using a combination of a binocular tracking system and a multi-line laser system, converted to a binocular tracking coordinate system. The binocular tracking system is fixed within a specific area of ​​the object being measured, and simultaneously connected to the multi-line laser system. While the multi-line laser system acquires data, the binocular tracking system synchronously acquires images of the tracking sphere, based on the tracking sphere coordinate system O. b The three-dimensional coordinates of the marker point Q below b (x i ,y i ,z i ), i = 0...n, and the three-dimensional coordinates of the tracking ball marker points reconstructed by the binocular tracking system. Calculate the transformation matrix from the tracking ball to the binocular tracking system. Based on the results of joint calibration And the value at position i calculated according to formula (4) Transform the 3D data reconstructed by the multi-line laser system into the binocular tracking coordinate system: In the process of 3D reconstruction, multi-line structured light technology emits multiple laser lines to the target object and uses the projection of these laser lines to obtain 3D data. By applying formula (6), the data reconstructed by the multi-line laser system at different positions are integrated into a coordinate system to realize the 3D reconstruction of the entire object.

Images