Hybrid lidar measurement method and system

Abstract

The application discloses a hybrid laser radar measurement method and system. The hybrid laser radar measurement method comprises the following steps: taking a picture of a measurement object by using a single-line laser radar, and acquiring a grayscale image returned by the single-line laser radar; identifying a region of interest from the grayscale image; determining three-dimensional coordinates of a light strip center in a camera coordinate system according to non-distortion light strip center coordinates and a light plane coefficient; and reconstructing the region of interest according to the three-dimensional coordinates of the light strip center in the camera coordinate system, so as to obtain reconstructed three-dimensional point cloud data. The application can improve the reliability and robustness of the obtained three-dimensional point cloud data.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, specifically to a hybrid lidar measurement method and system. Background Technology

[0002] With the launch of Microsoft's Kinect depth camera, consumer-grade depth cameras began to enter the public eye. Initially, depth cameras were only used in the gaming industry, but later, with the increasing demand for 3D information from various industries, they began to be widely used in facial recognition payment, obstacle avoidance mapping for robotic vacuum cleaners, obstacle avoidance for AVG vehicles, 3D modeling, and other industries.

[0003] Currently, the mainstream depth cameras on the market mainly fall into the following categories: first, pure binocular stereo vision; second, light coding technology; and third, time-of-flight (TOF) cameras. Pure binocular stereo vision requires a large amount of computation and is highly sensitive to texture information; its accuracy increases with the measurement distance. Light coding technology also projects speckle patterns onto the object being measured, so its accuracy is also affected by distance and is easily affected by ambient light in outdoor environments. TOF cameras measure distance by measuring the time it takes for light to travel, thereby reconstructing the point cloud of the object being measured. Compared to the other two principles, this principle of depth cameras has higher integration, lower cost, and a longer measurement distance.

[0004] Time-of-flight (TOF) sensors can be categorized into indirect time-of-flight (iTOF) and direct time-of-flight (dTOF) based on their timing principles. iTOF indirectly measures the time of flight of light by measuring phase shift, rather than directly measuring it. iTOF emits modulated infrared light into the scene, and the sensor receives the light reflected back from the object being measured. The phase difference between the emitted and received signals is calculated based on the accumulated charge over the exposure (integration) time, thus determining the depth of the target object. The core components of dTOF include a laser, a single-photon avalanche diode (SPAD), and a time-to-digital converter (TDC). A SPAD is a photoelectric avalanche diode with single-photon detection capability, generating current even with a weak light signal. In a dTOF module, the laser emits pulse waves into the scene, and the SPAD receives the pulse waves reflected back from the target object. The TDC records the time of flight of each received light signal, i.e., the time interval between the emitted and received pulses. dToF transmits and receives N light signals within a single frame measurement time, and then performs histogram statistics on the recorded N flight times. The flight time t with the highest frequency is used to calculate the depth of the object under test.

[0005] Depending on their application scenarios, Time-of-Flight (TOF) can be further divided into single-point TOF, single-line TOF, and area array TOF. Single-point TOF can only obtain distance information and can output measurement information in real time; single-line TOF can obtain the three-dimensional coordinates of a line and can output measurement information in real time; area array TOF can output the same number of three-dimensional coordinates as the chip resolution, but the frame rate is lower.

[0006] In some high-reflection scenes, when measuring black objects, TOF cameras are easily affected, resulting in low reliability of the output depth values. Summary of the Invention

[0007] In view of this, this application provides a hybrid lidar measurement method and system to solve the problem that in some high-reflection scenes, when measuring black objects, the TOF camera is easily affected and the reliability of the output depth value is low.

[0008] This application discloses a hybrid lidar measurement method, which includes the following steps:

[0009] A single-line lidar is used to photograph the object being measured, and a grayscale image returned by the single-line lidar is obtained.

[0010] Identify the region of interest from the grayscale image;

[0011] The three-dimensional coordinates of the light stripe center in the camera coordinate system are determined based on the distortion-free light stripe center coordinates and the light plane coefficient.

[0012] The region of interest is reconstructed based on the three-dimensional coordinates of the center of the light stripe in the camera coordinate system, resulting in reconstructed three-dimensional point cloud data.

[0013] Optionally, the region of interest includes areas containing black objects and / or overexposed areas.

[0014] Optionally, identifying the region of interest from the grayscale image includes: identifying a light bar region from the grayscale image; calculating the sum of grayscale values ​​of each pixel within the light bar region; and determining the light bar region as the region of interest if the sum of grayscale values ​​is less than a first threshold or greater than a second threshold.

[0015] Optionally, determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the center of the distortion-free light stripe and the light plane coefficient includes:

[0016]

[0017]

[0018]

[0019] Where (X,Y,Z) represent the three-dimensional coordinates of the light stripe center in the camera coordinate system, (a,b,c,d) represent the coefficients of the light plane equation in the camera coordinate system, (u,v) represent the coordinates of the distortion-free light stripe center, (u0,v0) represent the coordinates of the principal point, and f x f represents the focal length of a single-line lidar in the x-direction of the camera coordinate system. y This represents the focal length of the single-line lidar in the y-direction within the camera coordinate system.

[0020] Optionally, before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the light stripe center without distortion and the light plane coefficient, the hybrid lidar measurement method further includes: using laser triangulation to correct the distortion of the light stripe center in the region of interest to obtain the coordinates of the light stripe center without distortion.

[0021] Optionally, before determining the three-dimensional coordinates in the camera coordinate system, the hybrid lidar measurement method further includes: obtaining the rotation matrix R and translation matrix T between the single-line lidar and the calibration plate; and, based on the rotation matrix R and the translation matrix T, adjusting the multiple first coordinates P of the calibration plate. i By transforming to the camera coordinate system, multiple coordinates P are obtained. i The corresponding second coordinate P i cFor each of the second coordinates P i c Perform least-squares plane fitting to obtain each of the second coordinates P. i c A set of plane equation coefficients are obtained for each target. The target coordinates corresponding to the center coordinates of the light stripe in the planes determined by each set of plane equation coefficients are obtained. Least square plane fitting is performed on each target coordinate to obtain the light plane coefficients.

[0022] Optionally, obtaining the rotation matrix R and translation matrix T between the single-line lidar and the calibration plate includes: converting multiple coordinates P of the calibration plate... i Projecting the image onto the image plane in the camera coordinate system yields the homogeneous coordinates p of the corner points. i Construct the homogeneous coordinates of the corner points p i With coordinate P i The coordinate error equation between the two is obtained by solving the coordinate error equation using the LM optimization algorithm, thus obtaining the rotation matrix R and the translation matrix T.

[0023] Optionally, the corner point homogeneous coordinates p i p i =K·(RP) i +T); The coordinate error equation is: Where K represents the intrinsic parameter matrix of the single-line lidar, N represents the number of first coordinates on the calibration board, and min represents finding the minimum value.

[0024] Optionally, the step of... for each of the second coordinates P i c Perform least-squares plane fitting to obtain each of the second coordinates P. i c The corresponding set of plane equation coefficients includes: setting the second coordinate P i c The corresponding plane equation, based on the second coordinate P i c The plane error expression is constructed from the corresponding plane equation; the partial derivative equations corresponding to the partial derivatives of the plane error expression with respect to each independent variable are obtained; the partial derivative equations are solved using Cramer's rule to obtain the partial derivative solutions; and a set of plane equation coefficients are calculated based on the partial derivative solutions.

[0025] This application also provides a hybrid lidar measurement system, the hybrid lidar measurement system comprising:

[0026] The acquisition module is used to take a picture of the measurement object using a single-line lidar and acquire the grayscale image returned by the single-line lidar.

[0027] The recognition module is used to identify the region of interest from the grayscale image;

[0028] The determination module is used to determine the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient;

[0029] The reconstruction module is used to reconstruct the region of interest based on the three-dimensional coordinates of the center of the light stripe in the camera coordinate system, thereby obtaining the reconstructed three-dimensional point cloud data.

[0030] The hybrid lidar measurement method and system described in this application employ a single-line lidar to photograph the measurement object, acquiring a grayscale image returned by the single-line lidar. The region of interest (ROI) is identified from the grayscale image. The three-dimensional coordinates of the light stripe center in the camera coordinate system are determined based on the distortion-free light stripe center coordinates and the light plane coefficient. The ROI is then reconstructed based on these three-dimensional coordinates, resulting in reconstructed three-dimensional point cloud data. This improves the reliability and robustness of the obtained three-dimensional point cloud data. Furthermore, by processing only the ROI, the output rate of the single-line lidar data is effectively guaranteed. The entire hybrid lidar measurement process combines the time-of-flight method and the laser triangulation method, constructing a hybrid lidar system that enables high-speed point cloud data output and robust point cloud data output in complex scenarios. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic flowchart of a hybrid lidar measurement method according to an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of the ROI region according to an embodiment of this application;

[0034] Figure 3a and Figure 3b This is a schematic diagram of an image captured according to an embodiment of this application;

[0035] Figure 4 This is a schematic diagram of a hybrid lidar measurement system according to an embodiment of this application. Detailed Implementation

[0036] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. In the absence of conflict, the following embodiments and their technical features can be combined with each other.

[0037] The first aspect of this application provides a hybrid lidar measurement method, with reference to... Figure 1 As shown, the hybrid lidar measurement method includes steps S110 to S140.

[0038] S110, a single-line lidar is used to photograph the object being measured, and a grayscale image returned by the single-line lidar is obtained.

[0039] The aforementioned single-line lidar can include lidar composed of a TOF chip, which can be a part of a TOF camera. In some cases, single-line lidar can accurately acquire depth information of the object being measured, thus enabling measurement; however, when the object being measured includes a black area, is a black object, or there is overexposure during the photographing of the object, the data output by the single-line lidar is unreliable, requiring identification of these unreliable areas from the grayscale image.

[0040] Specifically, a single-line lidar takes a picture of the object being measured and can return a depth map and a grayscale image. Step S110 above can obtain the grayscale image to accurately identify the region of interest from the grayscale image.

[0041] S120, Identify the region of interest (ROI) from the grayscale image.

[0042] Optionally, the region of interest includes regions containing black objects and / or overexposed regions. For regions containing black objects and / or overexposed regions, the results reconstructed by a single-line lidar (or by a TOF camera) are unreliable. Therefore, defining these regions as regions of interest and using lidar triangulation to reconstruct the 3D point cloud data of the region of interest improves the accuracy of the final point cloud data.

[0043] In one example, identifying the region of interest from the grayscale image includes:

[0044] Identify light bar regions from the grayscale image, where the light bar regions include the areas on the image plane of the measured object corresponding to the light emitted by the single-line lidar;

[0045] Calculate the sum T of the gray values ​​of all pixels within the light stripe area;

[0046] If the sum of the gray values ​​is less than the first threshold T low Or greater than the second threshold T high That is, T <T low Or T>T high If T is found to contain a black object or be overexposed, the light stripe area is identified as the region of interest; if T low <=T<=T high If no overexposed or black objects are found, laser triangulation will not be used for reconstruction.

[0047] S130, determine the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the center of the distortion-free light stripe and the light plane coefficient.

[0048] In one example, determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient includes:

[0049]

[0050]

[0051]

[0052] Where (X,Y,Z) represent the three-dimensional coordinates of the light stripe center in the camera coordinate system, (a,b,c,d) represent the coefficients of the light plane equation in the camera coordinate system, (u,v) represent the coordinates of the distortion-free light stripe center, (u0,v0) represent the coordinates of the principal point, and f x f represents the focal length of a single-line lidar in the x-direction of the camera coordinate system. y This represents the focal length of the single-line lidar in the y-direction within the camera coordinate system. The coefficients of the light plane equation (a, b, c, d) can be determined by fitting multiple points on the light plane to the data. The distortion-free center coordinates (u, v) of the light stripe can be determined by correcting the distortion of the light stripe center coordinates. Like the principal point coordinates (u0, v0), the focal length f of the single-line lidar in the x-direction within the camera coordinate system... x The focal length f of a single-line lidar in the y-direction of the camera coordinate system. y It can be determined through methods such as camera calibration.

[0053] S140, the region of interest is reconstructed based on the three-dimensional coordinates of the light stripe center in the camera coordinate system to obtain the reconstructed three-dimensional point cloud data.

[0054] The above steps can be performed using laser triangulation to reconstruct the ROI region. Since laser triangulation only processes fixed ROI regions, this can effectively ensure the data output rate of single-line lidar.

[0055] The aforementioned hybrid lidar measurement method uses a single-line lidar to photograph the object being measured, obtaining a grayscale image returned by the single-line lidar. The region of interest (ROI) is identified from the grayscale image. The three-dimensional coordinates of the light stripe center in the camera coordinate system are determined based on the undistorted light stripe center coordinates and the light plane coefficient. The ROI is then reconstructed based on these three-dimensional coordinates, resulting in reconstructed three-dimensional point cloud data. This method improves the reliability and robustness of the obtained three-dimensional point cloud data, and by processing only the ROI, it effectively ensures the data output rate of the single-line lidar.

[0056] In one embodiment, before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the light stripe center without distortion and the light plane coefficient, the hybrid lidar measurement method further includes: using laser triangulation to correct the distortion of the light stripe center in the region of interest to obtain the coordinates of the light stripe center without distortion.

[0057] In this embodiment, the Steger method can be used to extract the center coordinates of the light stripe in the ROI region of the grayscale image, as shown in Figure 2. Then, distortion correction is performed on the center coordinates of the light stripe to obtain the center coordinates of the light stripe without distortion.

[0058] In one embodiment, before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient, the hybrid lidar measurement method further includes:

[0059] Obtain the rotation matrix R and translation matrix T between the single-line lidar and the calibration plate;

[0060] Based on the rotation matrix R and the translation matrix T, the multiple first coordinates P of the calibration plate are... i By transforming to the camera coordinate system, multiple coordinates P are obtained. i The corresponding second coordinate P i c ;

[0061] For each of the second coordinates P i c Perform least-squares plane fitting to obtain each of the second coordinates P. i c Each corresponds to a set of plane equation coefficients;

[0062] Obtain the target coordinates in the planes determined by the coefficients of each set of plane equations, corresponding to the center coordinates of the light stripe.

[0063] The light plane coefficients are obtained by performing least-squares plane fitting on each target coordinate.

[0064] Optionally, the intrinsic and extrinsic parameters of the single-line lidar light source can be calibrated using relevant calibration methods. In this embodiment, the single-line lidar light source can be turned off, the corresponding area light source can be turned on, and the image of the calibration board can be acquired. At this time, the image of the calibration board can be used as a reference. Figure 3a As shown. Then turn off the area light source and turn on the single-line lidar light source to acquire an image of the calibration board at this point. This image of the calibration board can be used as a reference. Figure 3b As shown. Since the single-line lidar has been calibrated, the intrinsic parameters of the camera (the camera corresponding to the single-line lidar) can be obtained. The coordinates on the calibration board are precisely known. Therefore, the pose between the single-line lidar and the calibration board is calculated using an optimized PNP, which includes the rotation matrix R and the translation matrix T.

[0065] In one example, obtaining the rotation matrix R and translation matrix T between the single-line lidar and the calibration board includes:

[0066] The multiple coordinates P of the calibration plate i Projecting the image onto the image plane in the camera coordinate system yields the homogeneous coordinates p of the corner points. i ;

[0067] Construct the homogeneous coordinates of the corner points p i With coordinate P i The coordinate error equation between them;

[0068] The coordinate error equation is solved using the LM optimization algorithm to obtain the rotation matrix R and the translation matrix T.

[0069] Specifically, the corner point homogeneous coordinates p i p i =K·(RP) i +T);

[0070] Specifically, by constructing an error equation and minimizing the error, the corresponding coordinate error equation is:

[0071] Where K represents the intrinsic parameter matrix of the single-line lidar, N represents the number of first coordinates on the calibration board, and min represents finding the minimum value. The rotation matrix R and translation matrix T can be obtained by solving the coordinate error equation using the LM optimization algorithm.

[0072] In one example, the multiple first coordinates P of the calibration plate are determined according to the rotation matrix R and the translation matrix T. i By transforming to the camera coordinate system, multiple coordinates P are obtained. i The corresponding second coordinate P i c , including: P i c =RP i +T.

[0073] In one example, the Steger algorithm can be used to extract the center of the light stripe in the corresponding light stripe image and perform light plane calibration. The process of extracting the center of the light stripe using the Steger algorithm includes: first, performing Gaussian filtering on the light stripe image; then, obtaining the first-order and second-order Hessian matrices for each pixel; solving for the two eigenvalues ​​of the Hessian matrix; if one eigenvalue is much larger than the other (e.g., the difference between the two eigenvalues ​​is greater than a preset difference threshold), then the point is considered the center of the light stripe; and finally, the sub-pixel coordinates of the light stripe center are calculated based on the normal vector corresponding to the largest eigenvalue. Optionally, the relevant formula is as follows:

[0074]

[0075]

[0076]

[0077] (tn x ,tn y )∈[-0.5,0.5]×[-0.5,0.5],

[0078] (p x ,p y )=(x+tn x ,y+tn y ),

[0079] In the above formula, Let be the first derivative of the point (x,y) in the x and y directions; Let be the second derivative of (x,y) in the x and y directions; t is a coefficient, if (tn) x ,tn y If )∈[-0.5,0.5]×[-0.5,0.5], then the sub-pixel coordinates (p) of the center of the light stripe can be obtained. x ,p y );(n x ,n y ) is the normal vector corresponding to the larger eigenvalue of the second-order Hessian matrix corresponding to the point (x,y).

[0080] In one example, the second coordinate P is... i c Perform least-squares plane fitting to obtain each of the second coordinates P. i c The corresponding set of plane equation coefficients includes:

[0081] Set the second coordinate P ic The corresponding plane equation, based on the second coordinate P i c The corresponding plane equations are used to construct the plane error expression;

[0082] Obtain the partial derivative equations corresponding to the partial derivatives of the plane error expression with respect to each independent variable;

[0083] The partial derivative equations are solved using Cramer's rule to obtain the solutions to the partial derivatives.

[0084] Calculate a set of plane equation coefficients based on each of the partial derivative solutions.

[0085] Specifically, the second coordinate P i c The coordinates in the camera coordinate system are (x i y i , z i Let the second coordinate P be... i c The corresponding plane equation is: a'x + b'y + c'z + 1 = 0.

[0086] Based on the above plane equations, the corresponding plane error equations can be derived:

[0087] Second coordinate P i c If the number of elements is N1, taking the partial derivative of the plane error equation yields partial derivative equations that may include:

[0088]

[0089]

[0090]

[0091] Simplifying the above partial derivative equation, we get:

[0092]

[0093]

[0094]

[0095] This system of equations is ultimately a linear system, so Cramer's rule is used to solve it. The corresponding partial derivative solutions can include D, D1, D2, and D3:

[0096]

[0097]

[0098]

[0099]

[0100] The coefficients (a0, b0, c0, d0) of the plane equations corresponding to each partial derivative solution include: d0 = 1.

[0101] In one example, the plane defined by the coefficients of each set of plane equations lies in the camera coordinate system. The target coordinates are (X... c ,Y c Z c The process of determining this includes:

[0102]

[0103]

[0104]

[0105] Where (u,v) represents the coordinates of the center of the distortion-free fringe, (u0,v0) represents the coordinates of the principal point, and f x f represents the focal length of a single-line lidar in the x-direction of the camera coordinate system. y This represents the focal length of the single-line lidar in the y-direction within the camera coordinate system.

[0106] The above hybrid LiDAR measurement method uses a single-line LiDAR to photograph the object being measured, acquiring a grayscale image returned by the single-line LiDAR. The region of interest (ROI) is identified from the grayscale image. Based on the distortion-free light stripe center coordinates and the light plane coefficient, the 3D coordinates of the light stripe center in the camera coordinate system are determined. The ROI is then reconstructed based on these 3D coordinates, resulting in reconstructed 3D point cloud data. This method improves the reliability and robustness of the obtained 3D point cloud data. Furthermore, by processing only the ROI, it effectively ensures the data output rate of the single-line LiDAR. The entire hybrid LiDAR measurement method combines the time-of-flight method and laser triangulation principles to construct a hybrid LiDAR system, enabling high-speed point cloud data output and robust point cloud data output in complex scenarios.

[0107] This application provides a hybrid lidar measurement device in a second aspect, such as... Figure 4 As shown, the hybrid lidar measurement device includes:

[0108] The acquisition module 110 is used to take a picture of the measurement object using a single-line lidar and acquire the grayscale image returned by the single-line lidar.

[0109] The recognition module 120 is used to identify the region of interest from the grayscale image;

[0110] The determination module 130 is used to determine the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient;

[0111] The reconstruction module 140 is used to reconstruct the region of interest based on the three-dimensional coordinates of the center of the light stripe in the camera coordinate system, so as to obtain the reconstructed three-dimensional point cloud data.

[0112] Specific limitations regarding the hybrid lidar measurement device can be found in the limitations of the hybrid lidar measurement method described above, and will not be repeated here. Each unit in the aforementioned hybrid lidar measurement device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each unit.

[0113] Although this application has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and drawings. This application includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components, the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this specification shown herein.

[0114] That is, the above description is only an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made using the content of this application’s specification and drawings, such as the combination of technical features between different embodiments, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of this application.

[0115] Furthermore, it should be understood that in the description of this application, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Additionally, for structural elements with the same or similar characteristics, this application may use the same or different reference numerals for identification. Moreover, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0116] In this application, the term "exemplary" is used to mean "serving as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as more preferred or advantageous than other embodiments. This application has been provided above to enable any person skilled in the art to implement and use it. Various details have been set forth in the above description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other embodiments, well-known structures and processes will not be described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

Claims

1. A hybrid lidar measurement method, characterized in that, The hybrid lidar measurement method includes the following steps: A single-line lidar is used to photograph the object being measured, and a grayscale image returned by the single-line lidar is obtained. Identify the region of interest from the grayscale image; The three-dimensional coordinates of the light stripe center in the camera coordinate system are determined based on the distortion-free light stripe center coordinates and the light plane coefficient. The region of interest is reconstructed based on the three-dimensional coordinates of the center of the light stripe in the camera coordinate system to obtain the reconstructed three-dimensional point cloud data. The region of interest includes areas containing black objects and / or overexposed areas; The step of identifying the region of interest from the grayscale image includes: identifying a light bar region from the grayscale image; calculating the sum of grayscale values ​​of each pixel within the light bar region; and determining the light bar region as the region of interest if the sum of grayscale values ​​is less than a first threshold or greater than a second threshold. Before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the light stripe center without distortion and the light plane coefficient, the hybrid lidar measurement method further includes: using laser triangulation to correct the distortion of the light stripe center in the region of interest to obtain the coordinates of the light stripe center without distortion; the step of using laser triangulation to correct the distortion of the light stripe center in the region of interest includes: using the Steger method to extract the coordinates of the light stripe center in the region of interest and correcting the distortion of the light stripe center coordinates.

2. The hybrid lidar measurement method according to claim 1, characterized in that, The step of determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient includes: ， ， ， Where (X,Y,Z) represents the three-dimensional coordinates of the light stripe center in the camera coordinate system, and (a,b,c,d) represents the coefficients of the light plane equation in the camera coordinate system. Indicates the coordinates of the center of the distortion-free light stripe. Represents the principal point coordinates. This represents the focal length of the single-line lidar in the x-direction in the camera coordinate system. This represents the focal length of the single-line lidar in the y-direction within the camera coordinate system.

3. The hybrid lidar measurement method according to claim 1, characterized in that, Before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient, the hybrid lidar measurement method further includes: Obtain the rotation matrix between the single-line lidar and the calibration plate. Translation matrix ; According to the rotation matrix and the translation matrix The calibration plate will have multiple first coordinates By transforming to the camera coordinate system, multiple coordinates are obtained. The corresponding second coordinate ; For each of the second coordinates Perform least-squares plane fitting to obtain each of the second coordinates. Each corresponds to a set of plane equation coefficients; Obtain the target coordinates in the planes determined by the coefficients of each set of plane equations, corresponding to the center coordinates of the light stripe. The light plane coefficients are obtained by performing least-squares plane fitting on each target coordinate.

4. The hybrid lidar measurement method according to claim 3, characterized in that, The rotation matrix between the single-line lidar and the calibration plate is obtained. Translation matrix ,include: Multiple coordinates of the calibration plate Projecting the image onto the image plane in the camera coordinate system yields the homogeneous coordinates of the corner points. ; Construct corner homogeneous coordinates With coordinates The coordinate error equation between them; The coordinate error equation is solved using the LM optimization algorithm to obtain the rotation matrix. and the translation matrix .

5. The hybrid lidar measurement method according to claim 4, characterized in that, The corner point homogeneous coordinates for: ; The coordinate error equation is as follows: ; in, This represents the intrinsic parameter matrix of the single-line lidar. This indicates the number of first coordinates on the calibration plate. This indicates that the minimum value is being sought.

6. The hybrid lidar measurement method according to claim 3, characterized in that, The second coordinates Perform least-squares plane fitting to obtain each of the second coordinates. The corresponding set of plane equation coefficients includes: Set the second coordinate The corresponding plane equation, based on the second coordinate. The corresponding plane equations are used to construct the plane error expression; Obtain the partial derivative equations corresponding to the partial derivatives of the plane error expression with respect to each independent variable; The partial derivative equations are solved using Cramer's rule to obtain the solutions to the partial derivatives. Calculate a set of plane equation coefficients based on each of the partial derivative solutions.

7. A hybrid lidar measurement system, characterized in that, The hybrid lidar measurement system includes: The acquisition module is used to take a picture of the measurement object using a single-line lidar and acquire the grayscale image returned by the single-line lidar. The recognition module is used to identify the region of interest from the grayscale image; The determination module is used to determine the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the distortion-free light stripe center coordinates and the light plane coefficient; The reconstruction module is used to reconstruct the region of interest based on the three-dimensional coordinates of the center of the light stripe in the camera coordinate system, and obtain the reconstructed three-dimensional point cloud data. The region of interest includes areas containing black objects and / or overexposed areas; identifying the region of interest from the grayscale image includes: identifying light bar areas from the grayscale image; calculating the sum of grayscale values ​​of each pixel within the light bar area; if the sum of grayscale values ​​is less than a first threshold or greater than a second threshold, then the light bar area is determined to be the region of interest; Before determining the three-dimensional coordinates of the light stripe center in the camera coordinate system based on the coordinates of the light stripe center without distortion and the light plane coefficient, the method further includes: using laser triangulation to correct the distortion of the light stripe center in the region of interest to obtain the coordinates of the light stripe center without distortion; the step of using laser triangulation to correct the distortion of the light stripe center in the region of interest includes: using the Steger method to extract the coordinates of the light stripe center in the region of interest and correcting the distortion of the light stripe center coordinates.

Images