LiDAR-based 3D reconstruction methods, devices, equipment, and storage media

Abstract

This invention relates to the field of 3D reconstruction technology, and more particularly to a 3D reconstruction method, apparatus, device, and storage medium based on lidar. The invention acquires first and second roll angle information of a 2D lidar at the same timestamp at different times. Based on the same timestamp from the previous and current times, a second rotation angle corresponding to the same timestamp of the 2D lidar at the current time is calculated. Based on a first compensation angle and the second compensation information, a linear interpolation algorithm is used to obtain the corresponding target compensation information. Based on the target compensation information, the 3D coordinates of the point cloud are obtained. This invention calculates the 3D coordinates of the point cloud based on a 2D lidar, reducing the cost of acquiring the 3D coordinates. By compensating for the roll angle of the acquired 2D lidar, the influence of the roll angle error of the 2D lidar on the 3D coordinates of the point cloud is reduced, thus improving the accuracy of the 3D coordinates of the point cloud.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method, apparatus, device, and storage medium based on lidar. Background Technology

[0002] 3D environment reconstruction technology is a crucial technology in current applications such as mobile robots and autonomous vehicles, playing a vital role in enabling robots to perceive 3D environmental information and plan subsequent motion strategies. Currently, mainstream 3D scene reconstruction technologies are mainly divided into two categories based on the sensors used: one category uses vision sensors, such as monocular cameras, binocular cameras, and RGB-D cameras; the other category is based on LiDAR sensors, such as single-line LiDAR sensors and multi-line LiDAR sensors. Vision sensors require processing large amounts of data, have a short measurement distance, and are easily affected by ambient lighting, generally limiting their application to indoor environments. LiDAR sensors offer high measurement accuracy and fast data acquisition speed, but common multi-line LiDAR sensors such as Velodyne-32 and Velodyne-64 are very expensive, posing a significant obstacle to the widespread application of 3D reconstruction technology.

[0003] Currently, for some application scenarios with low real-time requirements, a feasible way to reduce costs is to complete 3D reconstruction by combining 2D LiDAR with angle information. However, there is a gimbal lock problem in the process of acquiring angle information, which makes the 3D coordinate accuracy of the point cloud obtained by combining 2D LiDAR with angle information low. Therefore, how to improve the accuracy of 3D coordinates in the process of completing 3D reconstruction by combining 2D LiDAR with angle information has become an urgent problem to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a 3D reconstruction method, apparatus, device, and storage medium based on lidar to address the aforementioned technical problems and solve the issue of low 3D coordinate accuracy of point clouds when using 2D lidar for 3D reconstruction.

[0005] A first aspect of this application provides a three-dimensional reconstruction method based on lidar, the method comprising:

[0006] Obtain the first roll angle information of the two-dimensional lidar at the previous time when it was the same time stamp as the preset measurement tool, and the second roll angle information of the two-dimensional lidar at the current time when it was the same time stamp as the preset measurement tool;

[0007] Based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known first rotation angle information of the lidar at the previous moment, the second rotation angle information corresponding to the same timestamp of the current moment of the two-dimensional lidar is calculated.

[0008] Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation information obtained from the second roll angle information and the second rotation angle information, the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment is obtained through a linear interpolation algorithm.

[0009] Based on the target compensation information, the roll angle information of the two-dimensional lidar at each unit time stamp is compensated to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp;

[0010] Based on the target roll angle information and the polar coordinates of the point cloud at each unit timestamp obtained by the two-dimensional lidar scan, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation.

[0011] A second aspect of this application provides a three-dimensional reconstruction device based on lidar, the device comprising:

[0012] The same timestamp roll angle acquisition module is used to acquire the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous time, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current time;

[0013] The rotation angle information determination module is used to calculate the second rotation angle information corresponding to the same timestamp of the current moment of the two-dimensional lidar based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known first rotation angle information of the lidar at the previous moment.

[0014] The target compensation information determination module is used to obtain the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment, based on the first compensation angle obtained based on the first roll angle information and the first rotation angle information, and the second compensation information obtained based on the second roll angle information and the second rotation angle information, through a linear difference algorithm.

[0015] The target roll angle information determination module is used to compensate the roll angle information of the two-dimensional lidar at each unit time stamp according to the target compensation information, so as to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp.

[0016] The point cloud 3D coordinate determination model is used to obtain the 3D coordinates of the point cloud based on the target roll angle information and the polar coordinates of the point cloud at each unit timestamp obtained by the 2D lidar scan, through coordinate transformation.

[0017] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lidar-based three-dimensional reconstruction method as described in the first aspect.

[0018] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lidar-based three-dimensional reconstruction method as described in the first aspect.

[0019] The advantages of this invention compared to the prior art are:

[0020] The algorithm acquires the first roll angle information of the 2D LiDAR at the same timestamp as the preset measuring tool at the previous moment, and the second roll angle information of the 2D LiDAR at the same timestamp as the preset measuring tool at the current moment. Based on the same timestamps of the previous and current moments, the preset rotational angular velocity of the 2D LiDAR, and the known first rotation angle information of the LiDAR at the previous moment, the second rotation angle information corresponding to the same timestamp of the 2D LiDAR at the current moment is calculated. Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation information obtained from the second roll angle information and the second rotation angle information, a linear interpolation algorithm is used to obtain the same timestamps of the previous and current moments. The invention calculates the target compensation information corresponding to the roll angle information of the 2D LiDAR at each unit time stamp between the same timestamps. Based on the target compensation information, the roll angle information of the 2D LiDAR at each unit time stamp is compensated to obtain the target roll angle information of the 2D LiDAR at each unit time stamp. Based on the target roll angle information and the polar coordinates of the point cloud obtained by the 2D LiDAR scanning at each unit time stamp, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation. In this invention, the three-dimensional coordinates of the point cloud are calculated based on the 2D LiDAR, which reduces the cost of obtaining the three-dimensional coordinates of the point cloud. By compensating for the roll angle of the obtained 2D LiDAR, the influence of the roll angle error of the 2D LiDAR on the three-dimensional coordinates of the point cloud is reduced, thereby improving the accuracy of the three-dimensional coordinates of the point cloud. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an application environment for a three-dimensional reconstruction method based on lidar provided in an embodiment of the present invention;

[0023] Figure 2 This is a schematic flowchart of a three-dimensional reconstruction method based on lidar provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic flowchart of a three-dimensional reconstruction method based on lidar provided in an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of a three-dimensional reconstruction device based on lidar provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0029] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0031] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0033] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0034] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0035] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0036] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0037] An embodiment of the present invention provides a three-dimensional reconstruction method based on lidar, which can be applied to, for example... Figure 1 In this application environment, the client communicates with the server. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0038] See Figure 2 This is a flowchart illustrating a lidar-based 3D reconstruction method according to an embodiment of the present invention. The lidar-based 3D reconstruction method described above can be applied to... Figure 1 The server in the above-mentioned configuration connects to the corresponding client, such as... Figure 2 As shown, the lidar-based 3D reconstruction method may include the following steps.

[0039] S201: Obtain the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measurement tool at the previous time, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measurement tool at the current time.

[0040] In step S201, the data obtained by the preset measuring tool and the two-dimensional lidar can carry timestamps. The timestamp of the two-dimensional lidar is used to characterize the time when the point cloud data is output from the lidar, and the timestamp in the preset measuring tool is used to characterize the time when the roll angle information data of the two-dimensional lidar is output. The same timestamp means that the preset measuring tool and the two-dimensional lidar output point cloud data and roll angle information data at the same time.

[0041] In this embodiment, the preset measurement tool is an inertial navigation system (INS). The INS obtains the pose data of the two-dimensional lidar. During the 2D lidar measurement process, the INS can detect the operation of the 2D lidar, thereby obtaining its pose information. Specifically, the pose data may include X (coordinates on the X-axis), Y (coordinates on the Y-axis), Z (coordinates on the Z-axis), Roll (the angle of rotation around the Z-axis, also known as the roll angle), Pitch (the angle of rotation around the X-axis, also known as the pitch angle), and Yaw (the angle of rotation around the Y-axis, also known as the yaw angle). Thus, the roll angle information of the 2D lidar is obtained. The pose data output by the INS can carry a timestamp, which is used to characterize the time when the pose data was output from the INS.

[0042] Two-dimensional lidar scanning obtains the polar coordinates of the point cloud. The point cloud data carries a timestamp, allowing the corresponding time to be obtained for each frame of point cloud data scanned by the two-dimensional lidar. By recording the time when the inertial navigation system and the two-dimensional lidar simultaneously output data, the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measurement tool at the previous moment, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measurement tool at the current moment, can be obtained.

[0043] Optionally, the first roll angle information of the two-dimensional lidar at the previous time (the same timestamp as the preset measuring tool) and the second roll angle information of the two-dimensional lidar at the current time (the same timestamp as the preset measuring tool) are obtained, including:

[0044] Acquire the timestamp of each frame output by the 2D LiDAR and the timestamp of each frame output by the preset measurement tool;

[0045] By using a preset filter mechanism, the timestamps of each frame output by the two-dimensional lidar are filtered to match the timestamps of each frame output by the preset measurement tool, and the filtering result is obtained.

[0046] Based on the filtering results, the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous time is obtained, as well as the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current time.

[0047] In this embodiment, each frame timestamp output by the two-dimensional lidar and each frame timestamp output by the preset measurement tool are input into the filter. Filtering rules are set in the filter, and when the filtering rules are met, the corresponding timestamp that meets the filtering rules is output through the filter.

[0048] The filtering rule can be a time threshold. When the time difference between the timestamp output by the 2D LiDAR and the timestamp output by the preset measurement tool is less than the time threshold, the corresponding timestamp will be output to obtain the filtering result.

[0049] Based on the filtering results, multiple sets of timestamps output by the 2D LiDAR and timestamps output by the preset measurement tool can be obtained. These sets of identical timestamps are then sorted by time to identify two adjacent sets. These two sets of identical timestamps are used as the timestamp for the previous moment and the timestamp for the current moment, respectively. Based on these two timestamps, the first roll angle information and the second roll angle information for the corresponding timestamps are obtained.

[0050] S202: Based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known first rotation angle information of the lidar at the previous moment, the second rotation angle information corresponding to the same timestamp of the two-dimensional lidar at the current moment is calculated.

[0051] In step S202, the preset rotational angular velocity of the two-dimensional lidar is the angular velocity of the two-dimensional lidar during scanning. Based on the time difference between the same timestamp of the previous moment and the same timestamp of the current moment and the angular velocity, the rotation angle of the two-dimensional lidar in the time difference is obtained, and the rotation angle information of the two-dimensional lidar at the current moment is obtained.

[0052] In this embodiment, the rotation angle of the 2D lidar corresponding to the same timestamp at the previous moment is used as the first rotation angle information, and the rotation angle of the 2D lidar corresponding to the same timestamp at the current moment is used as the second rotation angle information. The first rotation angle and the second rotation angle are the pose information of the 2D lidar obtained by calculation.

[0053] S203: Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation information obtained from the second roll angle information and the second rotation angle information, the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time stamp between the same timestamp of the previous moment and the same timestamp of the current moment is obtained through a linear difference algorithm.

[0054] In step S203, there is an error between the first roll angle information and the second roll angle information of the two-dimensional lidar obtained by the preset measurement tool. The first roll angle information and the second roll angle information are compensated using the calculated first rotation angle information and the second rotation angle information of the two-dimensional lidar to obtain the first compensation information and the second compensation information. Based on the first compensation information and the second compensation information, the unit timestamp between the same timestamp at the previous moment and the same timestamp at the current moment are compensated to obtain the target compensation information corresponding to each unit timestamp.

[0055] In this embodiment, the preset measurement tool is an inertial navigation system (INS). The INS measures the first roll angle at the same timestamp of the previous moment and the second roll angle at the same timestamp of the current moment. The time interval between the previous timestamp and the current timestamp is divided into multiple time units, each time unit corresponding to a specific timestamp. The time unit is determined using the INS measurement frequency. A linear interpolation algorithm is used to obtain the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each time point between the previous and current timestamps.

[0056] Optionally, the compensation angle obtained based on the first roll angle information and the first rotation angle information, and the second compensation information obtained based on the second roll angle information and the second rotation angle information, include:

[0057] By calculating the average, the first average angle corresponding to the first roll angle information and the first rotation angle information, and the second average angle corresponding to the second roll angle information and the second rotation angle information are obtained;

[0058] The angle difference between the first average angle and the first roll angle information is used as the first compensation angle, and the angle difference between the second average angle and the second roll angle information is used as the second compensation angle.

[0059] In this embodiment, when calculating the compensation angle information, the compensation angle is calculated by taking the average of the roll angle and the rotation angle. By calculating the average, the first average angle corresponding to the first roll angle information and the first rotation angle information, and the second average angle corresponding to the second roll angle information and the second rotation angle information are obtained. The angle difference between the first average angle and the first roll angle information is taken as the first compensation angle, and the angle difference between the second average angle and the second roll angle information is taken as the second compensation angle.

[0060] Optionally, target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit timepoint between the same timestamp of the previous time and the same timestamp of the current time is obtained through a linear interpolation algorithm, including:

[0061] Construct the first coordinates based on the same timestamp from the previous moment and the first compensation angle;

[0062] Construct a second coordinate based on the same timestamp of the next moment and the second compensation angle;

[0063] Based on the first and second coordinates, the difference formula corresponding to the linear interpolation is obtained through calculation;

[0064] Based on the difference formula, the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time stamp between the same timestamp of the previous moment and the same timestamp of the next moment is obtained.

[0065] In this embodiment, the time corresponding to the timestamp is used as the horizontal axis and the compensation angle is used as the vertical axis. A first coordinate is constructed based on the same timestamp of the previous moment and the first compensation angle. A second coordinate is constructed based on the same timestamp of the next moment and the second compensation angle. A linear equation is constructed based on the first coordinate and the second coordinate, and the target compensation angle corresponding to each unit timestamp in the linear equation is calculated.

[0066] S204: Based on the target compensation information, the roll angle information of each unit time stamp of the two-dimensional lidar is compensated to obtain the target roll angle information of each unit time stamp of the two-dimensional lidar.

[0067] In step S204, the roll angle information of each unit time stamp of the two-dimensional lidar is compensated in order to reduce the error of the roll angle obtained by measurement, thereby increasing the accuracy of the three-dimensional coordinates of the point cloud obtained based on the roll angle.

[0068] In this embodiment, the roll angle information in each unit time stamp is added to the corresponding target compensation information to obtain the target roll angle information of each unit time stamp of the two-dimensional lidar. The obtained target roll angle information is a roll angle with high accuracy.

[0069] Optionally, based on the target compensation information, the roll angle information of the two-dimensional lidar at each unit time stamp is compensated to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp, including:

[0070] Obtain the roll angle information of the two-dimensional lidar with the closest time distance to each unit timestamp, and use it as the roll angle information of the two-dimensional lidar for each unit timestamp;

[0071] Based on the roll angle information and target compensation information of each unit time stamp of the two-dimensional lidar, the target roll angle information of each unit time stamp of the two-dimensional lidar is calculated.

[0072] In this embodiment, the operating frequency of the two-dimensional lidar is much lower than that of the inertial navigation system. Therefore, the unit timestamp of the two-dimensional lidar is much larger than that of the inertial navigation system. The timestamp of the corresponding frame of the inertial navigation system that is closest to each unit timestamp of the two-dimensional lidar is taken as the unit timestamp of the two-dimensional lidar. The obtained unit timestamps of the two-dimensional lidar are matched with the information of each frame of the two-dimensional lidar. The corresponding unit timestamps of the two-dimensional lidar are taken as the target roll angle information of each frame of the two-dimensional lidar. The target roll angle information is compensated by the target compensation information to obtain the target roll angle information of each unit timestamp of the two-dimensional lidar.

[0073] S205: Based on the target roll angle information and the polar coordinates of the point cloud obtained from each unit time stamp of the two-dimensional lidar scan, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation.

[0074] In step S205, the polar coordinates of the point cloud obtained by the two-dimensional lidar scanning for each unit timestamp are the distance and angle between the point cloud in each frame of the two-dimensional lidar and the two-dimensional lidar. Through coordinate transformation, the polar coordinates of the point cloud are converted into the three-dimensional coordinates of the point cloud.

[0075] In this embodiment, the coordinate transformation formula is a transformation formula obtained in advance through known transformation rules, as shown in equations (1), (2), and (3).

[0076] x=dis*α*cosRoll (1)

[0077] y=dis*α (2)

[0078] z = dis * α * sinRoll (3)

[0079] In equation (1), x is the x-axis coordinate in three-dimensional coordinates, y is the y-axis coordinate in three-dimensional coordinates, z is the z-axis coordinate in three-dimensional coordinates, dis is the distance between the point cloud and the two-dimensional lidar in polar coordinates, α is the angle between the point cloud and the two-dimensional lidar in polar coordinates, and Roll is the target roll angle information.

[0080] The algorithm acquires the first roll angle information of the 2D LiDAR at the same timestamp as the preset measuring tool at the previous moment, and the second roll angle information of the 2D LiDAR at the same timestamp as the preset measuring tool at the current moment. Based on the same timestamps of the previous and current moments, the preset rotational angular velocity of the 2D LiDAR, and the known first rotation angle information of the LiDAR at the previous moment, the second rotation angle information corresponding to the same timestamp of the 2D LiDAR at the current moment is calculated. Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation information obtained from the second roll angle information and the second rotation angle information, a linear interpolation algorithm is used to obtain the same timestamps of the previous and current moments. The invention calculates the target compensation information corresponding to the roll angle information of the 2D LiDAR at each unit time stamp between the same timestamps. Based on the target compensation information, the roll angle information of the 2D LiDAR at each unit time stamp is compensated to obtain the target roll angle information of the 2D LiDAR at each unit time stamp. Based on the target roll angle information and the polar coordinates of the point cloud obtained by the 2D LiDAR scanning at each unit time stamp, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation. In this invention, the three-dimensional coordinates of the point cloud are calculated based on the 2D LiDAR, which reduces the cost of obtaining the three-dimensional coordinates of the point cloud. By compensating for the roll angle of the obtained 2D LiDAR, the influence of the roll angle error of the 2D LiDAR on the three-dimensional coordinates of the point cloud is reduced, thereby improving the accuracy of the three-dimensional coordinates of the point cloud.

[0081] See Figure 3 This is a flowchart illustrating a three-dimensional reconstruction method based on lidar provided in an embodiment of the present invention, as shown below. Figure 3 The lidar-based 3D reconstruction method may include the following steps:

[0082] S301: Obtain the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous time, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current time;

[0083] S302: Based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known rotation angle of the lidar at the previous moment, the first rotation angle information corresponding to the same timestamp of the previous moment and the second rotation angle information corresponding to the same timestamp of the current moment of the two-dimensional lidar are calculated.

[0084] S303: Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation information obtained from the second roll angle information and the second rotation angle information, the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment is obtained through a linear difference algorithm.

[0085] S304: Based on the target compensation information, the roll angle information of each unit time stamp of the two-dimensional lidar is compensated to obtain the target roll angle information of each unit time stamp of the two-dimensional lidar.

[0086] S305: Based on the target roll angle information and the polar coordinates of the point cloud obtained by each unit time stamp of the two-dimensional lidar scan, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation.

[0087] The contents of steps S301 to S305 are the same as those of steps S201 to S205, and can be referred to the description of steps S201 to S205, which will not be repeated here.

[0088] S306: Obtain the pose transformation relationship between adjacent frames obtained by two-dimensional lidar scanning through a preset matching method;

[0089] S307: Based on the pose transformation relationship between adjacent frames, the three-dimensional coordinates of the point cloud are transformed to the same coordinate system to achieve three-dimensional reconstruction.

[0090] In this embodiment, matching is performed based on the 3D coordinates of each point cloud frame. For two point clouds using their own coordinate systems, a spatial transformation matrix is ​​calculated between them to unify them into the same coordinate system, achieving data stitching between the point clouds. The main implementation method is to perform point-to-point matching on two point clouds, calculate the spatial transformation matrix, and iterate this process until convergence is met. For example, given two point clouds P and M, where point cloud P has n points and point set M has m points, since the two point clouds are obtained by sampling the environment separately, the number of points in the two frames may not be the same. These two point clouds are point cloud images of the same environment or object from different perspectives, but they have overlapping features. By obtaining the pose transformation relationship between the two point clouds, they are transformed into the same coordinate system. The pose transformation relationship is a rotation and translation relationship. The rotation and translation matrices between the two point clouds are calculated using the ICP matching algorithm, transforming the two point clouds into the same coordinate system and achieving 3D reconstruction.

[0091] Please see Figure 4 , Figure 4 This is a schematic diagram of a three-dimensional reconstruction device based on lidar provided in an embodiment of the present invention. In this embodiment, the terminal includes units used for execution... Figures 2 to 3 The steps in the corresponding embodiments. Please refer to the details. Figures 2 to 3 as well as Figures 2 to 3 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 4 The reconstruction device 40 includes: a roll angle acquisition module 41 with the same timestamp, a rotation angle information determination module 42, a target compensation information determination module 43, a target roll angle information determination module 44, and a point cloud three-dimensional coordinate determination model 45.

[0092] The same timestamp roll angle acquisition module 41 is used to acquire the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous moment, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current moment;

[0093] The rotation angle information determination module 42 is used to calculate the first rotation angle information corresponding to the same timestamp of the previous moment and the same timestamp of the current moment based on the same timestamp of the previous moment and the preset rotation angular velocity of the two-dimensional lidar and the known rotation angle of the lidar at the previous moment.

[0094] The target compensation information determination module 43 is used to obtain the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment by using a linear difference algorithm based on the first compensation angle obtained based on the first roll angle information and the first rotation angle information, and the second compensation information obtained based on the second roll angle information and the second rotation angle information.

[0095] The target roll angle information determination module 44 is used to compensate the roll angle information of the two-dimensional lidar for each unit time stamp based on the target compensation information, so as to obtain the target roll angle information of the two-dimensional lidar for each unit time stamp.

[0096] The point cloud 3D coordinate determination model 45 is used to obtain the point cloud 3D coordinates based on the target roll angle information and the polar coordinates of each unit time stamp obtained by the 2D lidar scan, through coordinate transformation.

[0097] Optionally, the aforementioned same timestamp roll angle acquisition module 41 includes:

[0098] The acquisition unit is used to acquire the timestamp of each frame of the two-dimensional lidar.

[0099] The filtering unit is used to filter the timestamp of each frame of the two-dimensional lidar through a preset filter mechanism to obtain the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous time, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current time.

[0100] Optionally, the target compensation information determination module 43 mentioned above includes:

[0101] The mean calculation unit is used to obtain, through mean calculation, a first mean angle corresponding to the first roll angle information and the first rotation angle information, and a second mean angle corresponding to the second roll angle information and the second rotation angle information;

[0102] The compensation unit is used to take the angle difference between the first average angle and the first roll angle information as the first compensation angle, and the angle difference between the second average angle and the second roll angle information as the second compensation angle.

[0103] Optionally, the target compensation information determination module 43 mentioned above includes:

[0104] The first coordinate construction unit is used to construct the first coordinate based on the same timestamp of the previous moment and the first compensation angle;

[0105] The second coordinate construction unit is used to construct the second coordinate based on the same timestamp of the next moment and the second compensation angle;

[0106] The difference formula acquisition unit is used to calculate the difference formula corresponding to the linear interpolation based on the first coordinate and the second coordinate.

[0107] The calculation unit is used to obtain the target compensation information corresponding to the roll angle information of the two-dimensional lidar for each unit time stamp between the same timestamp of the previous time and the same timestamp of the next time, based on the difference formula.

[0108] Optionally, the target roll angle information determination module 44 includes:

[0109] The roll angle information acquisition unit for each unit timestamp is used to acquire the roll angle information of the two-dimensional lidar with the closest time distance to each unit timestamp, and use it as the roll angle information of the two-dimensional lidar for each unit timestamp.

[0110] The target roll angle information calculation unit is used to calculate the target roll angle information of the two-dimensional lidar at each unit time stamp based on the roll angle information and target compensation information of the two-dimensional lidar.

[0111] Optionally, the above-mentioned point cloud 3D coordinate determination model 45 includes:

[0112] The unit timestamp acquisition unit is used to acquire each unit timestamp;

[0113] The polar coordinate acquisition unit is used to acquire the polar coordinates of the target point cloud obtained by the two-dimensional lidar at each unit timestamp.

[0114] The coordinate transformation unit is used to obtain the three-dimensional coordinates of the point cloud by means of coordinate transformation based on the polar coordinates of the target point cloud and the target roll angle information.

[0115] Optionally, the reconstruction device 40 also includes:

[0116] The matching unit is used to obtain the pose transformation relationship between adjacent frames obtained by two-dimensional lidar scanning through a preset matching method.

[0117] The 3D reconstruction point cloud coordinate determination unit is used to transform the 3D coordinates of the point cloud to the same coordinate system according to the transformation relationship between adjacent frames, so as to realize 3D reconstruction.

[0118] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0119] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 5As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, which, when executed by the processor, implements the steps in any of the above embodiments of the lidar-based 3D reconstruction methods.

[0120] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0121] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0122] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0123] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0124] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0127] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A three-dimensional reconstruction method based on lidar, characterized in that, The reconstruction method includes: Obtain the first roll angle information of the two-dimensional lidar at the previous time when it was the same time stamp as the preset measurement tool, and the second roll angle information of the two-dimensional lidar at the current time when it was the same time stamp as the preset measurement tool; Based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known first rotation angle information of the lidar at the previous moment, the second rotation angle information corresponding to the same timestamp of the current moment of the two-dimensional lidar is calculated. Based on the first compensation angle obtained from the first roll angle information and the first rotation angle information, and the second compensation angle obtained from the second roll angle information and the second rotation angle information, the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment is obtained by linear interpolation algorithm. Based on the target compensation information, the roll angle information of the two-dimensional lidar at each unit time stamp is compensated to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp; Based on the target roll angle information and the polar coordinates of the point cloud at each unit timestamp obtained by the two-dimensional lidar scan, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation.

2. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, The acquisition of the first roll angle information of the two-dimensional lidar at the previous time with the same timestamp as the preset measuring tool, and the second roll angle information of the two-dimensional lidar at the current time with the same timestamp as the preset measuring tool, includes: Obtain the timestamp of each frame output by the two-dimensional lidar and the timestamp of each frame output by the preset measurement tool; By using a preset filter mechanism, the timestamps of each frame output by the two-dimensional lidar and the timestamps of each frame output by the preset measurement tool are filtered to obtain the filtering result. Based on the filtering results, the first roll angle information of the two-dimensional lidar at the previous time with the same timestamp as the preset measuring tool, and the second roll angle information of the two-dimensional lidar at the current time with the same timestamp as the preset measuring tool are obtained.

3. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, The first compensation angle obtained based on the first roll angle information and the first rotation angle information, and the second compensation angle obtained based on the second roll angle information and the second rotation angle information, include: By calculating the average, a first average angle corresponding to the first roll angle information and the first rotation angle information, and a second average angle corresponding to the second roll angle information and the second rotation angle information are obtained; The angle difference between the first average angle and the first roll angle information is used as the first compensation angle, and the angle difference between the second average angle and the second roll angle information is used as the second compensation angle.

4. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, The target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment, obtained through a linear interpolation algorithm, includes: Construct the first coordinates based on the same timestamp of the previous moment and the first compensation angle; Construct a second coordinate based on the same timestamp of the current moment and the second compensation angle; Based on the first coordinate and the second coordinate, the interpolation formula corresponding to the linear interpolation is calculated. Based on the interpolation formula, target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment is obtained.

5. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, The step of compensating the roll angle information of the two-dimensional lidar at each unit time stamp based on the target compensation information to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp includes: Obtain the roll angle information of the two-dimensional lidar that is closest to the time distance corresponding to each unit timestamp, and use it as the roll angle information of the two-dimensional lidar for each unit timestamp; Based on the roll angle information of each unit time stamp of the two-dimensional lidar and the target compensation information, the target roll angle information of each unit time stamp of the two-dimensional lidar is calculated.

6. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, The point cloud polar coordinates, obtained from each unit timestamp of the point cloud based on the target roll angle information and the two-dimensional lidar scan, are transformed to obtain the three-dimensional coordinates of the point cloud, including: Obtain the timestamp of each unit; Based on each unit timestamp, obtain the polar coordinates of the target point cloud obtained by the two-dimensional lidar scanning at each unit timestamp; Based on the polar coordinates of the target point cloud and the target roll angle information, the three-dimensional coordinates of the point cloud are obtained through coordinate transformation.

7. The three-dimensional reconstruction method based on lidar as described in claim 1, characterized in that, After obtaining the three-dimensional coordinates of the point cloud based on the target roll angle information and the polar coordinates of each unit time stamp obtained by the two-dimensional lidar scan, through coordinate transformation, the method further includes: The pose transformation relationship between adjacent frames obtained by the two-dimensional lidar scanning is obtained by using a preset matching method. Based on the pose transformation relationship between adjacent frames, the three-dimensional coordinates of the point cloud are transformed to the same coordinate system to achieve three-dimensional reconstruction.

8. A three-dimensional reconstruction device based on lidar, characterized in that, The device includes: The same timestamp roll angle acquisition module is used to acquire the first roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the previous time, and the second roll angle information of the two-dimensional lidar at the same timestamp as the preset measuring tool at the current time; The rotation angle information determination module is used to calculate the second rotation angle information corresponding to the same timestamp of the current moment of the two-dimensional lidar based on the same timestamp of the previous moment and the same timestamp of the current moment, as well as the preset rotation angular velocity of the two-dimensional lidar and the known first rotation angle information of the lidar at the previous moment. The target compensation information determination module is used to obtain the target compensation information corresponding to the roll angle information of the two-dimensional lidar at each unit time point between the same timestamp of the previous moment and the same timestamp of the current moment, based on the first compensation angle obtained based on the first roll angle information and the first rotation angle information, and the second compensation angle obtained based on the second roll angle information and the second rotation angle information, through a linear interpolation algorithm. The target roll angle information determination module is used to compensate the roll angle information of the two-dimensional lidar at each unit time stamp according to the target compensation information, so as to obtain the target roll angle information of the two-dimensional lidar at each unit time stamp. The point cloud 3D coordinate determination model is used to obtain the 3D coordinates of the point cloud based on the target roll angle information and the polar coordinates of the point cloud at each unit timestamp obtained by the 2D lidar scan, through coordinate transformation.

9. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the lidar-based three-dimensional reconstruction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the lidar-based three-dimensional reconstruction method as described in any one of claims 1 to 7.

Images