A method for calibration and point cloud generation based on binocular single-line laser

Abstract

The present application belongs to the field of computer three-dimensional vision, and particularly relates to a kind of calibration and point cloud generation method based on binocular single line laser, for the existing calibration method with the aid of plane calibration board, it needs to move ten times or even dozens of times calibration board to complete a calibration, and only one position laser line can be calibrated each time, the step is very cumbersome, it is very troublesome in actual use, the present application proposes the following scheme, which includes calibration and reconstruction, calibration includes the following steps: S1: using conventional calibration algorithm to calibrate binocular camera accurately, all subsequent steps are carried out on the basis of binocular camera calibration; S2: fix the laser sensor, then place the arc-shaped calibration block in front of the laser sensor, and make the arc surface face the laser sensor, the present application simplifies the calibration process, and makes it easier to operate; at the same time, the flexibility of scanning is improved, so that single and binocular cameras can complete the scanning process; the field of view is enlarged, and more point cloud data is reconstructed.

Description

Technical Field

[0001] This invention relates to the field of computer 3D vision technology, and in particular to a method for calibration and point cloud generation based on binocular single-line laser. Background Technology

[0002] Laser-based 3D shape measurement technology, with its advantages of being non-contact, highly accurate, and widely applicable, has become an indispensable technology in the industrial field and has high research value. This technology is widely used in product defect inspection, automated assembly, dimensional measurement, cultural relic reconstruction, and visual navigation, demonstrating significant practical value.

[0003] Conventional calibration methods rely on planar calibration plates, which require moving the calibration plate dozens of times to complete one calibration. Moreover, each calibration can only calibrate the laser line at one position, making the process very cumbersome and troublesome in actual use. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing calibration methods, which rely on planar calibration plates, require moving the calibration plate dozens of times to complete a single calibration, and can only calibrate one laser line position at a time, making the process very cumbersome and inconvenient in practical use. Therefore, this invention proposes a calibration and point cloud generation method based on binocular single-line laser.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for calibration and point cloud generation based on binocular single-line laser, including calibration and reconstruction, wherein the calibration includes the following steps:

[0007] S1: The stereo camera is precisely calibrated using a conventional calibration algorithm. All subsequent steps are based on the stereo camera calibration.

[0008] S2: Fix the laser sensor, then place the arc-shaped calibration block in front of the laser sensor, with the arc surface facing the laser sensor;

[0009] S3: Rotate the laser emitter to move it to the initial emission position on one side;

[0010] S4: The laser sensor sends a trigger signal, and the left and right cameras capture images and transmit them to the PC;

[0011] S5: Extract the center point of the laser line in the image, reconstruct their three-dimensional coordinates using binocular vision, and use these points to fit a laser plane;

[0012] S6: Rotate the laser emitter to the next emission position, and execute S4 and S5 again.

[0013] S7: Repeat S6 continuously until the laser emitter passes through all emission positions, and finally fits the laser plane of all emission positions.

[0014] S8: Move the position of the arc-shaped calibration block backward and execute steps three through seven again. This will eventually result in another set of laser planes. This step is performed to improve the depth of field and accuracy of the calibration. To obtain higher accuracy, the position of the arc-shaped calibration block can be moved multiple times to obtain more sets of laser planes.

[0015] S9: Fit multiple laser planes corresponding to each emission position into one, and finally obtain a set of high-precision laser plane parameters, i.e., laser calibration data.

[0016] Refactoring includes the following steps:

[0017] T1: Rotate the laser emitter to a certain emission position. This emission position may not be the initial emission position. The final reconstructed point cloud only includes the area scanned by the laser.

[0018] T2: The laser sensor sends a trigger signal to one of the cameras, which captures the image and transmits it to the PC. The trigger signal can also be sent to the left and right cameras, causing both cameras to capture the image and transmit it to the PC.

[0019] T3: Extract the center points of the laser line, calculate their coordinates in the image coordinate system, and then transform these coordinates to the camera coordinate system. Starting from the center point of the imaging laser, through the optical center of the camera, fit a ray, calculate the intersection of this ray with the laser plane of the emission position, and this intersection is the corresponding scanned object point. Then repeat this operation for all the remaining center points of the laser line to finally reconstruct a point cloud of a laser line.

[0020] T4: Rotate the laser emitter to the next emission position, and repeat step three;

[0021] T5: Repeat T4 continuously until the laser emitter reaches the target emission position, and finally obtain the point cloud of the laser lines corresponding to all emission positions;

[0022] T6: If only one camera captures the image, the final point cloud is obtained at this point. If both the left and right cameras capture the image, you can choose to keep only the point cloud data from the left or right camera as the final point cloud data, or you can fuse the point cloud data from the left and right cameras to obtain point cloud data with a larger field of view.

[0023] Preferably, in S5, the laser plane refers to the plane on which the laser line projected by the laser emitter at a emission position is located.

[0024] Preferably, in T6, the point cloud data fusion includes the following steps: based on the parameters obtained from the dual-camera calibration, the point cloud coordinates in the right camera coordinate system are transformed to the left camera coordinate system. If a point exists only in one of the cameras, then the point will be retained. If a point exists in both the left and right cameras, then the point in the left camera is retained and the point in the right camera is removed.

[0025] Preferably, in step S2, the laser emitter has a series of fixed emission positions. The position of the emission position is determined by the angle sensor on the laser emitter. Each time it rotates, the laser emitter driven by the motor can only move from one emission position to the next. The position of each emission position relative to the two cameras is strictly fixed and will not change. The relative position between the left and right cameras is also relatively fixed. At any emission position, the laser emitter can send a trigger signal to the left and right cameras. After receiving the trigger signal, the camera will immediately capture the image and then transmit the image to the PC for further processing.

[0026] Preferably, calibration requires the use of a calibration block with one side curved for assistance, and at least two calibration blocks need to be calibrated.

[0027] Preferably, in step S1, after completing the camera calibration, the internal parameter matrix of the left camera can be obtained:

[0028]

[0029] And the internal parameter matrix of the right camera:

[0030]

[0031] in, and These represent the coordinates of the imaging center points of the left and right cameras, respectively; α l and α r β represents the effective focal length of the left and right cameras along the u-axis, respectively; l and β r These represent the effective focal lengths of the left and right cameras along the v-axis, respectively; γ l and γ r These represent the tilt coefficients of the left and right cameras along the U-axis and V-axis, respectively.

[0032] And the rotation matrix R and translation vector t between the left and right cameras.

[0033] Preferably, in step S5, the center points of the laser lines in the image are extracted, and their three-dimensional coordinates are reconstructed using binocular vision. All the points are located on an arc, and a laser plane is fitted using these points.

[0034] Calculate the eigenvalue matrix between the left and right cameras:

[0035] E=t^R

[0036] Take the center point p of the laser line in the left camera. l Using polar constraint equations

[0037] l T Ep l =0

[0038] Among them, l T This represents the set of possible matching points in the right camera, which together form an epipolar line; the intersection of the epipolar line and the laser line in the right camera is the actual matching point.

[0039] Then, by using the principle of similar triangles, the true coordinates of the point in the camera coordinate system can be obtained;

[0040] Using the above steps, the coordinates of all center points along a laser line in the camera coordinate system can be obtained. Since the laser line strikes a curved surface, these reconstructed points are not on a straight line, therefore a plane can be fitted.

[0041] A i x+B i y+C i z+D i =0

[0042] Here, the subscript i represents the i-th transmit bit from left to right.

[0043] Preferably, in T3, the center points of the laser lines are extracted, their coordinates in the image coordinate system are calculated, and then the coordinates are transformed to the camera coordinate system. Starting from the center point of the imaging laser, a ray is fitted through the optical center of the camera. The intersection point of this ray with the laser plane of the emission position is calculated. This intersection point is the corresponding scanned object point. Then, this operation is repeated for all the remaining center points of the laser lines, and finally a point cloud of a laser line can be reconstructed.

[0044] After transforming to the camera coordinate system, the coordinates of the center point can be expressed as (x0, y0, z0). Solving the equations of the straight line and the laser plane at that emission position simultaneously:

[0045]

[0046] The result of this system of equations is the coordinates of the scanned object point in the camera coordinate system.

[0047] The beneficial effects of the method for calibration and point cloud generation based on binocular single-line laser in this invention are as follows:

[0048] This invention combines a binocular camera and a laser sensor to build a visual binocular laser system. Based on the calibration of the binocular camera, the laser plane of each emission position is calibrated. Stable calibration results can be obtained by moving the calibration block at least twice, which greatly simplifies the laser calibration process and makes laser calibration easier to perform, and greatly reduces the difficulty of operation for non-professionals.

[0049] The reconstruction method used in this invention can employ either one or two cameras for scanning. Single-camera scanning is applicable to scenes with a small field of view, while two-camera scanning can fuse their respective point cloud data to obtain a larger field of view. This provides more diverse scanning methods and greater flexibility, meeting different customer needs. By combining different numbers of cameras, products with varying shapes and sizes can be easily designed. Attached Figure Description

[0050] Figure 1 This is a calibration flowchart of a method for calibration and point cloud generation based on binocular single-line laser proposed in this invention.

[0051] Figure 2 This is a reconstruction flowchart of a method for calibration and point cloud generation based on binocular single-line laser proposed in this invention;

[0052] Figure 3 This is a schematic diagram of the calibration block structure of a calibration and point cloud generation method based on binocular single-line laser proposed in this invention.

[0053] Figure 4 This is a schematic diagram of the laser structure used in the method for calibration and point cloud generation based on binocular single-line laser proposed in this invention.

[0054] In the diagram: 1. Laser emitter; 2. Angle sensor; 3. Motor. Detailed Implementation

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0056] Reference Figure 1-4 A method for calibration and point cloud generation based on binocular single-line laser, including calibration and reconstruction, with the following calibration steps:

[0057] S1: The stereo camera is precisely calibrated using a conventional calibration algorithm. All subsequent steps are based on the stereo camera calibration.

[0058] After completing the camera calibration, the intrinsic parameter matrix of the left camera can be obtained:

[0059]

[0060] And the internal parameter matrix of the right camera:

[0061]

[0062] in, and These represent the coordinates of the imaging center points of the left and right cameras, respectively; α l and α r β represents the effective focal length of the left and right cameras along the u-axis, respectively; l and β r These represent the effective focal lengths of the left and right cameras along the v-axis, respectively; γ l and γ r These represent the tilt coefficients of the left and right cameras along the U-axis and V-axis, respectively.

[0063] And the rotation matrix R and translation vector t between the left and right cameras.

[0064] S2: Fix the laser sensor, then place the arc-shaped calibration block in front of the laser sensor, with the arc surface facing the laser sensor.

[0065] S3: Rotate laser emitter 1 to move it to the initial emission position on one side.

[0066] S4: The laser sensor sends a trigger signal, and the left and right cameras capture images and transmit them to the PC.

[0067] S5: Extract the center points of the laser lines in the image and reconstruct their 3D coordinates using a binocular camera. All points lie on an arc, and a laser plane is fitted using these points. Here, the laser plane refers to the plane on which the laser line projected by laser emitter 1 from one emission position lies.

[0068] Calculate the eigenvalue matrix between the left and right cameras:

[0069] E=t^R

[0070] Take the center point p of the laser line in the left camera. l Using polar constraint equations

[0071] l T Ep l =0

[0072] Among them, l T This represents the set of possible matching points in the right camera; these points together form an epipolar line. The intersection of the epipolar line and the laser line in the right camera is the actual matching point.

[0073] Then, by using the principle of similar triangles, the true coordinates of the point in the camera coordinate system can be obtained.

[0074] Using the above steps, the coordinates of all center points along a laser line in the camera coordinate system are obtained. Since the laser line strikes a curved surface, these reconstructed points are not on a straight line; therefore, a plane can be fitted.

[0075] A i x+B i y+C i z+D i =0

[0076] Here, the subscript i represents the i-th transmit bit from left to right.

[0077] S6: Rotate laser emitter 1 to the next emission position. Repeat steps four and five.

[0078] S7: Repeat S6 continuously until laser emitter 1 passes through all emission positions. Finally, the laser plane of all emission positions will be fitted.

[0079] S8: Move the position of the arc-shaped calibration block backward and execute S3 to S7 again. This will ultimately yield another set of laser planes. This step is performed to improve the depth of field and accuracy of the calibration. To obtain even higher accuracy, the position of the arc-shaped calibration block can be moved multiple times to obtain more sets of laser planes.

[0080] S9: Fit multiple laser planes corresponding to each emission position into one, and finally obtain a set of high-precision laser plane parameters, i.e., laser calibration data.

[0081] The laser emitter 11 has a series of fixed emission positions. The position of the emission position is determined by the angle sensor 2 on the laser emitter 1. Each time it rotates, the laser emitter 1 driven by the motor 3 can only move from one emission position to the next. The position of each emission position relative to the two cameras is strictly fixed and will not change. The relative position between the left and right cameras is also relatively fixed. At any emission position, the laser emitter 1 can send a trigger signal to the left and right cameras. After receiving the trigger signal, the camera will immediately capture the image and then transmit the image to the PC for further processing.

[0082] Refactoring steps:

[0083] T1: Rotate laser emitter 1 to a specific emission position. This emission position may not be the initial emission position; the final reconstructed point cloud will only include the area scanned by the laser.

[0084] T2: The laser sensor sends a trigger signal to one of the cameras, which captures the image and transmits it to the PC. The trigger signal can also be sent to both the left and right cameras, causing both cameras to capture images and transmit them to the PC.

[0085] T3: Extract the center points of the laser lines, calculate their coordinates in the image coordinate system, and then transform these coordinates to the camera coordinate system. Starting from the center point of the imaging laser, a ray is fitted through the camera optical center (i.e., the origin of the camera coordinate system). The intersection point of this ray and the laser plane at that emission position is calculated; this intersection point is the corresponding scanned object point. This process is then repeated for all the remaining laser line center points to eventually reconstruct a point cloud of the laser line.

[0086] After transforming to the camera coordinate system, the coordinates of the center point can be expressed as (x0, y0, z0). Solving the equations of the straight line and the laser plane at that emission position simultaneously:

[0087]

[0088] The result of this system of equations is the coordinates of the scanned object point in the camera coordinate system.

[0089] T4: Rotate laser emitter 1 to the next emission position, and execute T3 again.

[0090] T5: Repeat T4 continuously until laser emitter 1 reaches the target emission position, and finally obtain the point cloud of the laser lines corresponding to all emission positions.

[0091] T6: If only one camera captures the image, the final point cloud is obtained at this point. If both the left and right cameras capture the image, you can choose to keep only the point cloud data from the left or right camera as the final point cloud data, or you can fuse the point cloud data from the left and right cameras to obtain point cloud data with a larger field of view.

[0092] The point cloud fusion process is explained below:

[0093] Based on the parameters obtained from the dual-camera calibration, the point cloud coordinates in the right camera coordinate system are transformed to the left camera coordinate system. If a point exists only in one of the cameras, then the point will be retained. If a point exists in both the left and right cameras, then the point in the left camera is retained and the point in the right camera is removed.

[0094] Patents related to this invention include:

[0095] A method for calibration of three-dimensional measurement using line laser scanning (publication number: CN111272102A);

[0096] The difference between this invention and the aforementioned patent is as follows:

[0097] The aforementioned patent uses a monocular camera for calibration, which requires the use of multiple calibration plates. These calibration plates need to be placed in different poses and cannot be located on the same plane.

[0098] The present invention uses a binocular camera for calibration, which only requires the use of an arc-shaped calibration block, and the calibration block only needs to be moved twice at least.

[0099] A method for determining line-structured targets based on chessboard targets, publication number: CN10118528A;

[0100] The difference between this invention and the aforementioned patent is as follows:

[0101] The aforementioned patent uses a monocular camera for calibration, which requires the use of a checkerboard calibration board. The calibration process requires changing the pose of the calibration board multiple times, and only one laser plane can be calibrated at a time.

[0102] The present invention uses a binocular camera for calibration. During calibration, only an arc-shaped calibration block is needed. The calibration block only needs to be moved twice, and the laser plane of all emission positions can be calibrated in one go.

[0103] Calibration plate for line laser position calibration and calibration method for line laser camera measurement system, publication number: CN106056620A;

[0104] The difference between this invention and the aforementioned patent is as follows:

[0105] The aforementioned patent uses a monocular camera for calibration, which requires the use of an irregularly shaped checkerboard calibration plate with protruding areas.

[0106] The present invention uses a binocular camera for calibration, which only requires the aid of an arc-shaped calibration block.

[0107] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for calibration and point cloud generation based on binocular single line laser, comprising calibration and reconstruction, characterized in that, The calibration includes the following steps: S1: using a conventional calibration algorithm to calibrate the binocular camera, and all subsequent steps are based on the calibration of the binocular camera; S2: fix the laser sensor, then place the arc-shaped calibration block in front of the laser sensor, and make the arc surface face the laser sensor; S3: rotate the laser transmitter (1) to move to the initial emission position on one side; S4: the laser sensor sends a trigger signal, and the left and right cameras capture images and transmit them to the PC; S5: extract the center points of the laser lines in the images, reconstruct their three-dimensional coordinates using binoculars, all points are located on an arc line, and a laser plane is fitted using these points; S6: rotate the laser transmitter (1) to reach the next emission position, and execute S4 and S5 again; S7: continuously repeat S6 until the laser transmitter (1) passes through all the emission positions, and finally fit the laser planes of all the emission positions; S8: move the position of the arc-shaped calibration block backward, and execute steps 3 to 7 again, and finally obtain another set of laser plane parameters; S9: fit multiple laser planes corresponding to each emission position into one, and finally obtain a set of high-precision laser plane parameters, i.e. laser calibration data; The reconstruction includes the following steps: T1: rotate the laser transmitter (1) to reach a certain emission position, which can not be the initial emission position, and the finally reconstructed point cloud only includes the scanned region; T2: the laser sensor sends a trigger signal to a certain camera, which captures images and transmits them to the PC, or the trigger signal can be sent to both left and right cameras, so that both cameras capture images and transmit them to the PC; T3: extract the center points of the laser lines, calculate their coordinates in the image coordinate system, and then convert the coordinates to the camera coordinate system. Starting from a center point of the imaging laser, a ray is fitted through the camera optical center, the intersection of the ray and the laser plane at the emission position is calculated, which is the scanned object point, and then the operation is repeated for all other laser line center points, and finally the point cloud of a laser line is reconstructed; T4: rotate the laser transmitter (1) to reach the next emission position, and execute step 3 again; T5: continuously repeat T4 until the laser transmitter (1) reaches the target emission position, and finally obtain the point cloud of the laser line corresponding to all emission positions; T6: if only one camera captures images, the final point cloud is obtained. If both left and right cameras capture images, you can choose to keep only the point cloud data of the left camera or the right camera as the final point cloud data, or you can fuse the point cloud data of the left and right cameras to obtain point cloud data with a larger field of view.

2. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, characterized in that, In S5, the laser plane refers to the plane on which the laser line projected by the laser transmitter (1) at an emission position is located.

3. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, characterized in that, In the T6, the point cloud data fusion includes the following steps: according to the parameters obtained by the double target determination, the point cloud coordinates in the right camera coordinate system are transformed into the left camera coordinate system, if a point exists only in one of the cameras, the point is retained, if a point exists in both the left and right cameras, the point in the left camera is retained, and the point in the right camera is removed.

4. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, characterized in that, In the S2, the laser emitter (1) has a series of fixed emission positions, the positions of the emission positions are determined by the angle sensor (2) on the laser emitter (1), each time the laser emitter (1) driven by the motor (3) can only move from one emission position to the next emission position, the position of each emission position relative to the two cameras is strictly fixed and will not change, and the relative position between the left and right cameras is also relatively fixed, at any emission position, the laser emitter (1) can send a trigger signal to the left and right cameras, after the cameras receive the trigger signal, they will immediately capture an image and then transmit the image to the PC for subsequent processing.

5. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, wherein, When calibrating, an arc-shaped calibration block is needed to assist, and at least two positions of the calibration block need to be calibrated.

6. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, wherein, In the S1, after the camera calibration, the internal parameter matrix of the left camera can be obtained: and the internal parameter matrix of the right camera: wherein, and respectively represent the imaging center point coordinates of the left and right cameras; α l and α r respectively represent the effective focal lengths of the left and right cameras along the u axis; β l and β r respectively represent the effective focal lengths of the left and right cameras along the v axis; γ l and γ r respectively represent the tilt coefficients of the u axis and the v axis of the left and right cameras; and the rotation matrix R and the translation vector t between the left and right cameras.

7. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, wherein, In the S5, the center points of the laser lines in the image are extracted, and their three-dimensional coordinates are reconstructed by binoculars, all the points are located on an arc line, and a laser plane is fitted by using these points; The intrinsic matrix between the left and right cameras is calculated: E=t^R Taking the center point p of the laser line in the left camera l Using the epipolar constraint equation l T Ep l =0 where l T represents the set of points in the right camera that can be matched, which together form an epipolar line; the intersection of the epipolar line and the laser line in the right camera is the real matched point; Then the real coordinates of the point in the camera coordinate system can be obtained by using the triangle similarity principle; The coordinates of all center points on a laser line in the camera coordinate system are obtained by using the above steps, and since the laser line is on a curved surface, the reconstructed points are not on a straight line, so a plane can be fitted: A i x+B i y+C i z+D i =0 Wherein, the subscript i represents the i-th emission position from left to right.

8. The method of calibrating and generating point cloud based on binocular single line laser according to claim 1, wherein, In the T3, the center points of the laser lines are extracted, their coordinates in the image coordinate system are calculated, and then the coordinates are converted to the camera coordinate system, starting from an imaging laser center point, passing through the camera optical center, a ray is fitted, the intersection point of the ray and the laser plane of the emission position is calculated, which is the scanned object point, and then the operation is repeated for all the remaining laser line center points, and finally the point cloud of a laser line is reconstructed. After conversion to the camera coordinate system, the center point coordinates can be represented as (x0, y0, z0), and the straight line equation and the laser plane equation of the emission position are combined: The result of the equation set is the coordinates of the scanned object point in the camera coordinate system.

Images