A precise three-dimensional measurement method and device based on laser and monocular vision fusion

Abstract

The application discloses a kind of precision three-dimensional measurement method and device based on laser and monocular vision fusion.The measurement method is as follows: the relative position of monocular camera and fast mirror deflection center is calibrated;Then laser ranging module collects depth information, determines the pixel coordinates of laser ranging point through monocular camera, and obtains the three-dimensional information of laser ranging point under the camera coordinate system by combining depth information and pixel coordinates;Determine whether the fast mirror scanning process is finished, if not, the fast mirror continues to deflect, and repeat the above measurement process, if finished, output three-dimensional point cloud.The method of the application obtains the coordinate information of laser ranging point using camera, combines the advantages of camera and laser measurement, realizes pixel-level accuracy in two-dimensional plane direction, and achieves precision three-dimensional measurement in depth direction with laser ranging accuracy, and the implementation cost is relatively low, which is conducive to technology promotion and application popularization.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional measurement, and in particular to a precision three-dimensional measurement method and apparatus based on the fusion of laser and monocular vision. Background Technology

[0002] With the rapid development of sensor and computer vision technologies, many tasks such as autonomous driving, robotics, and remote sensing have placed high demands on precise 3D measurement methods. Existing mainstream 3D measurement methods are divided into camera-based passive 3D measurement and lidar-based active 3D measurement. Monocular cameras can measure accurate azimuth and pitch angles of a target, but they cannot recover the scale information of real 3D motion from image data, thus failing to obtain reliable, high-precision 3D measurement information. Lidar, on the other hand, can obtain high-precision distance information of a target, but its two-dimensional resolution is relatively low, and for mainstream scanning lidar, its 3D measurement accuracy heavily depends on the angle measurement device.

[0003] A single sensor is insufficient for high-precision 3D measurement. Existing methods combine the measurement results of multiple sensors to improve measurement accuracy. This involves first using LiDAR and cameras to perform measurements separately, and then using appropriate algorithms to fuse and calculate the results. However, this approach has limitations in terms of computational resource consumption and measurement accuracy. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing laser and monocular vision fusion measurement technologies and provide a precision 3D measurement method and device based on laser and monocular vision fusion. This invention utilizes a camera to obtain the angular information of laser measurement points, effectively avoiding the reliance on high-precision angle measurement devices in scanning lidar, achieving pixel-level accuracy in the two-dimensional plane and precision 3D measurement comparable to laser ranging accuracy in the depth direction.

[0005] This invention is implemented as follows:

[0006] First, this invention provides a precision three-dimensional measurement method based on the fusion of laser and monocular vision, comprising the following steps:

[0007] S1, jointly calibrate the deflection centers of the monocular camera and the fast-reflection mirror to obtain calibration parameters including the camera intrinsic parameter matrix K and the position information (x, y, y) of the fast-reflection mirror deflection center in the camera coordinate system. f y f , z f );

[0008] S2, the laser ranging module emits visible laser light, which is then deflected by a fast-reflecting mirror;

[0009] S3, the camera acquires the pixel information of the laser ranging point emitted by the laser ranging module, and at the same time records the depth information measured by the corresponding laser ranging module;

[0010] S4. Based on the camera intrinsic parameter matrix obtained in S1, the position information of the fast-reflecting mirror deflection center in the camera coordinate system, and the pixel information and depth information obtained in S3, the three-dimensional coordinate information of the laser ranging module measurement point in the camera coordinate system is calculated.

[0011] S5 determines whether the fast-reflection mirror scanning process has ended. If it has not ended, the fast-reflection mirror deflects and repeats S2, S3 and S4. If it has ended, it outputs the three-dimensional coordinate information of all ranging points, i.e., the three-dimensional point cloud.

[0012] Furthermore, step S1 specifically includes:

[0013] S11. Using a chessboard pattern calibration board, the monocular camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameter matrix K.

[0014] S12, the laser beam is deflected using a fast-reflecting mirror, and the laser ranging point is deflected to the corners of different checkerboard grids. The depth information (d) of the three laser spots (spot a, spot b, and spot c) is recorded respectively. a d b and d c ) and pixel information (u a v a u b v b and u c v c Since the grid spacing is fixed, the distance L between light spot a and light spot b can be obtained. ab The distance L between light spot b and light spot c bc The distance L between light spot a and light spot c ac Therefore, the position information of the deflection center of the fast-reflecting mirror in the camera coordinate system (x f y f , z f The calculation is as follows:

[0015]

[0016] Where t1, t2, and t3 are real numbers greater than 0 to be determined.

[0017] Furthermore, step S4 specifically includes:

[0018] S41, the method for calculating the three-dimensional coordinate information (x, y, z) of the measurement point of the laser ranging module is as follows:

[0019]

[0020] Where t is a real number greater than 0 to be determined, K is the intrinsic parameter matrix, u and v are the pixel information of the ranging point obtained by the camera, d is the depth information measured by the laser ranging module, and x f y f , z f This refers to the position information of the deflection center of the fast-reflecting mirror in the camera coordinate system.

[0021] The present invention also provides a precision three-dimensional measurement device based on laser and monocular vision fusion, comprising:

[0022] A laser ranging module is used to measure depth information and emit a visible light spot;

[0023] Quick-reflecting mirrors are used for beam deflection.

[0024] A camera used to acquire pixel information of the laser measurement point;

[0025] The data processing unit is used for image processing, 3D information acquisition, and output of 3D point clouds.

[0026] Furthermore, the camera, quick-reflecting mirror, and laser rangefinder module are connected to the data processing unit via a data cable.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] (1) The present invention uses a camera to obtain the angle information of the laser measurement point, which effectively solves the problem that the accuracy of scanning lidar three-dimensional measurement depends on a high-precision angle measurement device.

[0029] (2) This invention provides a method for laser and monocular vision fusion, which realizes the conversion of measurement results of camera and laser in different coordinate systems to the same coordinate system by calibrating the position of the deflection center of fast mirror in the camera coordinate system.

[0030] (3) The present invention has a simple structure and is easy to operate. It effectively improves the accuracy of three-dimensional measurement, achieving pixel-level accuracy in the two-dimensional plane direction and comparable to the accuracy of laser ranging in the depth direction. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of the steps of a precision three-dimensional measurement method based on laser and monocular vision fusion according to an embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram illustrating the calculation of the three-dimensional coordinate information of the measurement point of the laser ranging module in the camera coordinate system in an embodiment of the present invention.

[0034] Figure 3 A schematic diagram of the structure of a precision three-dimensional measurement device based on laser and monocular vision fusion according to an embodiment of the present invention.

[0035] Attached reference numerals: 1-Laser ranging module, 2-Quick-reflection mirror, 3-Camera, 4-Data processing unit. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0037] like Figure 1 and Figure 2 As shown, the present invention provides a precision three-dimensional measurement method based on laser and monocular vision fusion, comprising the following steps:

[0038] S1, Install the camera, laser rangefinder module, and fast-reflecting mirror, ensuring the deflection center of the fast-reflecting mirror is as close as possible to the camera's optical center. Here, O-XYZ represents the camera coordinate system, i.e., the world coordinate system of this system. l -X l Y l Z l For the laser measurement coordinate system, O o -uv represents the camera pixel coordinate system. Joint calibration of the camera and the fast-reflecting mirror deflection center yields calibration parameters including the camera intrinsic matrix K and the position information of the fast-reflecting mirror deflection center in the camera coordinate system (x...). f y f , z f The camera used is a monocular camera.

[0039] S2, the laser ranging module emits visible laser light, which is then deflected by a fast-reflecting mirror;

[0040] S3, the camera acquires the pixel information of the laser ranging point A emitted by the laser ranging module, that is, gives the pixel coordinates (u, v) of the laser measurement point A in the pixel coordinate system, and records the corresponding depth information d measured by the laser ranging module.

[0041] S4, based on the camera intrinsic parameter matrix K obtained in S1 and the position information (x) of the fast-reflection mirror deflection center in the camera coordinate system.f y f , z f The pixel information (u, v) and depth information d obtained from S3 are used to calculate the three-dimensional coordinate information (x, y, z) of the ranging point A of the laser ranging module in the camera coordinate system.

[0042] S5 determines whether the fast-reflection mirror scanning process has ended. If it has not ended, the fast-reflection mirror deflects and repeats S2, S3 and S4. If it has ended, the three-dimensional coordinate information of all ranging points, i.e., the three-dimensional point cloud, is output.

[0043] Furthermore, step S1 specifically includes:

[0044] S11. Using a chessboard pattern calibration board, the monocular camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameter matrix K.

[0045] S12, the laser beam is deflected using a fast-reflecting mirror, and the laser spot is deflected to the corner points of different checkerboard squares. The depth information (d) of the three laser spots (spot a, spot b, and spot c) is recorded respectively. a d b and d c ), pixel information (u a v a u b v b and u c v c Since the checkerboard grid spacing is fixed, the spacing Lac between spot a and spot b, and the spacing L between spot b and spot c can be obtained. bc The distance L between light spot a and light spot c ac Therefore, the position information (x) of the fast-reflecting mirror deflection center in the camera coordinate system is obtained. f y f , z f The calculation is as follows:

[0046]

[0047] Where t1, t2, and t3 are real numbers greater than 0 to be determined.

[0048] Furthermore, step S4 specifically includes:

[0049] S41, the method for calculating the three-dimensional coordinate information (x, y, z) of the ranging point A of the laser ranging module is as follows:

[0050]

[0051] Where t is a real number greater than 0 to be determined, K is the intrinsic parameter matrix, u and v are the pixel information of the ranging point obtained by the camera, d is the depth information measured by the laser ranging module, and x f y f , z f This refers to the position information of the deflection center of the fast-reflecting mirror in the camera coordinate system.

[0052] like Figure 3 As shown, the present invention also provides a precision three-dimensional measurement device based on laser and monocular vision fusion, comprising:

[0053] Laser ranging module 1 is used to measure depth information and emit a visible light spot;

[0054] Quick-reflecting mirror 2, used for beam deflection;

[0055] Camera 3 is used to acquire the spot pixel information of the laser measurement point;

[0056] Data processing unit 4 is used for image processing, 3D information acquisition, and output of 3D point clouds.

[0057] Furthermore, the camera, quick-reflecting mirror, and laser rangefinder module are connected to the data processing unit via a data cable.

[0058] During operation, the deflection centers of the monocular camera 1 and the fast-reflecting mirror 2 are first jointly calibrated. Next, the laser ranging module emits visible laser light to measure depth information and obtains pixel information of the laser ranging point through the camera. Then, the data processing unit combines the calibration information, depth information, and pixel information to calculate and save the three-dimensional information of the ranging point. Finally, it determines whether the fast-reflecting mirror scanning process is complete. If not, the fast-reflecting mirror deflects and measures and saves the three-dimensional information of the laser ranging point after deflection. If the process is complete, a three-dimensional point cloud is output.

[0059] As can be seen from the above specific implementation scheme, the present invention is a precision 3D measurement method and device based on the fusion of laser and monocular vision. Compared with existing technologies, it avoids the problem that the accuracy of scanning lidar 3D measurement depends on high-precision angle measurement devices. At the same time, the present invention has a simple structure and is easy to operate, effectively combining the advantages of camera and laser measurement to achieve pixel-level accuracy in the two-dimensional plane and precision 3D measurement comparable to laser ranging accuracy in the depth direction.

[0060] The specific embodiments described above are only used to explain and illustrate the claims of the present invention and do not constitute a limitation on the claims. Those skilled in the art should understand that any simple modifications, variations, or substitutions made based on the technical solutions of the present invention to obtain new technical solutions will fall within the protection scope of the present invention.

Claims

1. A precision three-dimensional measurement method based on laser and monocular vision fusion, characterized in that, Includes the following steps: S1, jointly calibrate the deflection centers of the monocular camera and the fast-reflecting mirror to obtain calibration parameters including the camera intrinsic parameter matrix K and the position information of the fast-reflecting mirror deflection center in the camera coordinate system. ); S2, the laser ranging module emits visible laser light, which is then deflected by a fast-reflecting mirror; S3, the monocular camera acquires pixel information of the laser ranging point emitted by the laser ranging module. Simultaneously, the depth information measured by the corresponding laser ranging module is recorded. ; S4, based on the camera intrinsic parameter matrix K obtained in S1 and the position information of the fast-reflection mirror deflection center in the camera coordinate system ( ), pixel information obtained in S3 ( ) and depth information The three-dimensional coordinate information (x, y, z) of the laser ranging module's ranging point in the camera coordinate system is calculated. S5: Determine whether the fast-reflection mirror scanning process has ended. If it has not ended, the fast-reflection mirror deflects and repeats S2, S3 and S4. If it has ended, output the three-dimensional coordinate information of all ranging points, i.e., the three-dimensional point cloud. The S1 step specifically includes: S11. Using a chessboard pattern calibration board, the monocular camera is calibrated using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameter matrix K. S12, using a fast-reflecting mirror, deflects the laser beam and moves the laser ranging point to the corners of different checkerboard grids, recording the depth information of three laser spots, spot a, spot b, and spot c. , and and pixel information ( ), ( )and( Since the grid spacing is fixed, the spacing between light spots a and b can be obtained. The distance between light spot b and light spot c The distance between light spot a and light spot c Therefore, the position information of the deflection center of the fast-reflecting mirror in the camera coordinate system ( The calculation is as follows: in, Let be a real number greater than 0 to be found; S4 specifically includes: S41, the method for calculating the three-dimensional coordinate information (x, y, z) of the ranging point of the laser ranging module is as follows: Where t is a real number greater than 0 to be determined, K is the intrinsic parameter matrix, and u and v are the pixel information of the ranging points obtained by the camera. The depth information measured by the laser ranging module. This refers to the position information of the deflection center of the fast-reflecting mirror in the camera coordinate system.

2. A measuring device used in the precision three-dimensional measurement method of claim 1, characterized in that, include: A laser ranging module is used to measure depth information and emit a visible light spot; Quick-reflecting mirrors are used for beam deflection. A monocular camera is used to acquire the pixel information of the laser measurement point. The data processing unit is used for image processing, 3D information acquisition, and output of 3D point clouds.

3. The measuring device as described in claim 2, characterized in that, The monocular camera, fast-reflecting mirror, and laser ranging module are connected to the data processing unit via a data cable.

Images