Unmanned aerial vehicle fusion mapping and positioning method and device, computer device and storage medium

Abstract

The application relates to a UAV fusion mapping and positioning method and device, computer equipment and a storage medium. The UAV is provided with a solid-state laser radar and a rotatable support, the rotatable support drives the solid-state laser radar to rotate, the method comprises the following steps: acquiring an original laser point cloud data frame according to the solid-state laser radar; acquiring IMU data according to the body IMU of the UAV; pre-processing the original laser point cloud data frame and the IMU data to obtain pre-processed data; acquiring a UAV pose according to the pre-processed data; and acquiring a double-layer map according to the UAV pose and the pre-processed data; wherein the double-layer map comprises a sparse feature map and a visual dense voxel map. The application can improve the precision of the UAV and reduce the quality of the UAV.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicles (UAVs), and more particularly to a method, apparatus, computer equipment, and storage medium for UAV fusion mapping and positioning. Background Technology

[0002] The flight, positioning, and mapping of drones rely on the cooperation of lidar and other devices.

[0003] LiDAR is generally divided into rotating LiDAR and solid-state LiDAR. Rotating LiDAR achieves 360-degree scanning by rotating a laser inside the LiDAR. Rotating LiDAR typically has 32 or 64 lines. Due to the size of the laser emitter itself, the size of the accompanying optical equipment, and the size of the circuitry, it is difficult for rotating LiDAR to achieve more laser beams, which limits its resolution.

[0004] Solid-state lidar typically achieves laser scanning through a high-speed vibrating galvanometer, thus enabling extremely high resolution. However, solid-state lidar has a limited field of view, usually 120 degrees to the left and right, making it difficult to achieve comprehensive detection.

[0005] To overcome the field-of-view limitations of solid-state lidar, multiple solid-state lidar units can be deployed to achieve a 360-degree field of view. However, solid-state lidar units are larger, heavier, and far more expensive than rotating lidar units, making it difficult to equip multiple solid-state lidar units on a drone.

[0006] It is evident that existing drone technologies cannot achieve high-precision fusion mapping and positioning using solid-state LiDAR. Summary of the Invention

[0007] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, the present invention provides a method, apparatus, computer equipment and storage medium for unmanned aerial vehicle (UAV) fusion mapping and positioning.

[0008] In a first aspect, the present invention provides a method for fusion mapping and positioning of a UAV, wherein the UAV is equipped with a solid-state LiDAR and a rotatable support, the rotatable support driving the solid-state LiDAR to rotate, and the method includes:

[0009] Obtain the original laser point cloud data frame from the solid-state lidar;

[0010] IMU data is acquired based on the drone's airframe IMU;

[0011] The original laser point cloud data frame and the IMU data are preprocessed to obtain preprocessed data;

[0012] The UAV pose is obtained based on the preprocessed data;

[0013] Based on the UAV pose and the preprocessed data, a two-layer map is obtained;

[0014] The two-layer map includes a sparse feature map and a visualized dense voxel map.

[0015] Optionally, the step of acquiring the original laser point cloud data frame based on the solid-state lidar includes:

[0016] Based on the phase of the rotatable bracket, the field of view corresponding to one rotation of the solid-state lidar is divided into multiple corner windows, and the boundary between two adjacent corner windows is taken as the corner window boundary.

[0017] The solid-state lidar rotates under the drive of a rotatable bracket and acquires the original point cloud.

[0018] The original point cloud is divided into multiple quasi-spiral subframes according to the corner window boundary;

[0019] The plurality of quasi-helical subframes constitute the original laser point cloud data frame.

[0020] Optionally, the width constraint condition of the corner window is:

[0021] The linear displacement of each quasi-spiral subframe does not exceed the maximum linear displacement, and the angular displacement of each quasi-spiral subframe does not exceed the maximum angular displacement.

[0022] Optionally, the step of dividing the field of view corresponding to one rotation of the solid-state lidar into multiple corner windows based on the phase of the rotatable bracket includes:

[0023] The width of the corner window is adaptively adjusted according to the degree of shaking during the flight of the drone;

[0024] The greater the shaking during flight, the smaller the width of the corner window.

[0025] Optionally, the width constraint condition of the corner window is:

[0026]

[0027] The phase span of the corner window is determined as follows:

[0028]

[0029] in, The width of the corner window. For the maximum linear displacement, The average linear velocity of the rotating electric motor. For the maximum angular displacement, The average angular velocity of the rotating motor. For lidar radio frequency, The minimum number of points in a single quasi-spiral subframe. For phase span, For the first The initial phase of a quasi-spiral subframe For the first The initial phase of a quasi-spiral subframe.

[0030] Optionally, the original laser point cloud data frame and the IMU data have timestamps.

[0031] The preprocessing of the original laser point cloud data frame and the IMU data to obtain preprocessed data includes:

[0032] Each corner window boundary is taken as the ending phase of the current quasi-spiral subframe and the starting phase of the next quasi-spiral subframe, and the starting phase of the current quasi-spiral subframe is taken as the reference time.

[0033] Based on the timestamp, the original point cloud in each of the quasi-spiral subframes is reverted to the corresponding reference time according to the IMU data to obtain the reverted data frame;

[0034] Based on the IMU rotation rate, the time deviation of the data frame after the rollback is compensated to remove distortion in the point cloud data and obtain the distortion-free data frame.

[0035] The distortion-reduced data frame is filtered using voxel filtering, statistical cluster point removal filtering, and intensity threshold filtering to obtain preprocessed data.

[0036] The phase of the callback data frame is aligned with that of the IMU data.

[0037] Optionally, the step of compensating for the time deviation of the data frame after the rollback based on the IMU rotation rate to remove distortion in the point cloud data and obtain a distortion-free data frame includes:

[0038] The encoder angular velocity is obtained based on the IMU rotation rate;

[0039] Based on the pre-integration of the IMU data and the encoder angular velocity, the relative pose of the point cloud of the post-rewind data frame at the reference time of the quasi-helical subframe is obtained.

[0040] By transforming coordinates, the point clouds of all the data frames after the rollback are transformed to the radar coordinate system or the body coordinate system to obtain spatiotemporally consistent corrected point clouds, which are used as the distortion-free data frames.

[0041] Optionally, the step of transforming the point cloud of all the dialed-back data frames to the radar coordinate system or the body coordinate system through coordinate transformation includes:

[0042] Obtain the extrinsic parameter matrix between the solid-state lidar and the machine-mounted IMU;

[0043] Obtain the current angle of the rotatable bracket;

[0044] Based on the current angle and current extrinsic parameter matrix of the rotatable bracket, the point cloud of the data frame after the rollback is mapped from the radar coordinate system to the body coordinate system, or from the body coordinate system to the radar coordinate system.

[0045] Optionally, obtaining the UAV pose based on the preprocessed data includes:

[0046] Based on the pre-integration of the IMU data, the initial values ​​of inter-frame motion are obtained, and the state prediction is obtained based on the initial values ​​of inter-frame motion.

[0047] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the coarse pose;

[0048] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map;

[0049] The local sub-map is precisely registered with the historical sub-map to obtain the accurate pose.

[0050] Optionally, obtaining a two-layer map based on the UAV pose and the preprocessed data includes:

[0051] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the sparse feature map. The sparse feature map includes: line features, area features, feature covariance, phase cover histogram of features, and number of observations.

[0052] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map.

[0053] The local sub-map is precisely registered with the historical sub-map to obtain the visualized dense voxel map.

[0054] Optionally, obtaining a two-layer map based on the UAV pose and the preprocessed data further includes:

[0055] The coarse pose and IMU prediction are tightly coupled and fused in the iterative error state Kalman filter to obtain the optimal state.

[0056] Based on the optimal state, the current keyframe in the iterative error state Kalman filter is projected onto the global coordinate system and fused into the previous two-layer map to obtain the current two-layer map.

[0057] Secondly, a drone fusion mapping and positioning device is provided, wherein the drone is equipped with a solid-state LiDAR and a rotatable support, the rotatable support driving the solid-state LiDAR to rotate, and the device includes:

[0058] The point cloud acquisition unit is used to acquire raw laser point cloud data frames based on the solid-state lidar.

[0059] The IMU data acquisition unit is used to acquire IMU data based on the UAV's airframe IMU.

[0060] The preprocessing unit is used to preprocess the original laser point cloud data frame and the IMU data to obtain preprocessed data;

[0061] The pose acquisition unit is used to acquire the pose of the UAV based on the preprocessed data.

[0062] The map acquisition unit is used to acquire a two-layer map based on the UAV pose and the preprocessed data;

[0063] The two-layer map includes a sparse feature map and a visualized dense voxel map.

[0064] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any of the preceding claims.

[0065] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the preceding claims.

[0066] This invention provides a method, apparatus, computer device, and storage medium for unmanned aerial vehicle (UAV) fusion mapping and positioning. The UAV is equipped with a solid-state lidar and a rotatable support, the rotatable support driving the solid-state lidar to rotate. The method includes: acquiring raw lidar point cloud data frames from the solid-state lidar; acquiring IMU data from the UAV's in-body IMU; preprocessing the raw lidar point cloud data frames and the IMU data to obtain preprocessed data; acquiring the UAV pose based on the preprocessed data; and acquiring a two-layer map based on the UAV pose and the preprocessed data. The two-layer map includes a sparse feature map and a visualized dense voxel map. In this embodiment, the UAV is equipped with a solid-state lidar and a rotatable support, the rotatable support driving the solid-state lidar to rotate. This eliminates the need for multiple solid-state lidars to obtain a 360-degree field of view, thus solving the problem of insufficient 360-degree left and right field of view for solid-state lidars. It also reduces the overall weight of the UAV and improves its endurance. Furthermore, this invention also provides a method for preprocessing raw laser point cloud data frames from a solid-state lidar and IMU data from an airborne IMU to obtain preprocessed data, and for acquiring UAV pose and a two-layer map based on the preprocessed data. The two-layer map includes a sparse feature map and a visualized dense voxel map. The sparse feature map has relatively low computational cost, enabling real-time localization and mapping with low computational resources; the visualized dense voxel map combines visualization with the advantages of dense mapping and voxel modeling, offering high precision, flexible scalability, enhanced observability and data reliability, and improved UAV accuracy. Attached Figure Description

[0067] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 The diagram shown illustrates the application environment of the UAV fusion mapping and positioning method according to an embodiment of the present invention.

[0070] Figure 2 The diagram shown is a flowchart illustrating the UAV fusion mapping and positioning method according to an embodiment of the present invention.

[0071] Figure 3 The diagram shown is a structural block diagram of the UAV fusion mapping and positioning device according to an embodiment of the present invention.

[0072] Figure 4 The diagram shown is an internal structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Figure 1 This is an application environment diagram of the UAV fusion mapping and localization method in one embodiment. (Refer to...) Figure 1 This UAV fusion mapping and positioning method is applied to a UAV fusion mapping and positioning system. The method includes a terminal 110 and / or a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; the mobile terminal can be at least one of a mobile phone, tablet, or laptop. The terminal 110 is mounted on the UAV.

[0075] Server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. Server 120 can be installed on the drone or not.

[0076] The UAV fusion mapping and positioning method of the present invention is applied to terminal 110 and / or server 120.

[0077] like Figure 2 As shown, in one embodiment, a method for fusion mapping and positioning of unmanned aerial vehicles (UAVs) is provided.

[0078] In this embodiment of the invention, the drone is equipped with a solid-state lidar and a rotatable bracket, and the rotatable bracket drives the solid-state lidar to rotate.

[0079] This embodiment mainly applies the method described above. Figure 1 The example is terminal 110 and / or server 120.

[0080] Reference Figure 2 The UAV fusion mapping and positioning method includes:

[0081] Step 210: Obtain the original laser point cloud data frame based on the solid-state lidar;

[0082] Step 220: Obtain IMU data from the UAV's in-flight IMU (Inertial Measurement Unit).

[0083] Step 230: Preprocess the original laser point cloud data frame and the IMU data to obtain preprocessed data;

[0084] Step 240: Obtain the UAV pose based on the preprocessed data;

[0085] Step 250: Obtain a two-layer map based on the UAV pose and the preprocessed data;

[0086] The two-layer map includes a sparse feature map and a visualized dense voxel map.

[0087] In this embodiment of the invention, the UAV is equipped with a solid-state lidar and a rotatable support. The rotatable support drives the solid-state lidar to rotate, achieving a 360-degree field of view without the need for multiple solid-state lidars. This solves the problem of insufficient 360-degree left and right field of view for solid-state lidars, reduces the overall weight of the UAV, and improves its endurance. Furthermore, this embodiment of the invention provides a method for preprocessing the raw laser point cloud data frames from the solid-state lidar and the IMU data from the airframe IMU to obtain preprocessed data. Based on the preprocessed data, a method is used to obtain the UAV pose and a two-layer map. The two-layer map includes a sparse feature map and a visualized dense voxel map. The sparse feature map has relatively low computational cost, enabling real-time localization and mapping with low computational resources. The visualized dense voxel map combines visualization with the advantages of dense mapping and voxel modeling, offering high precision, flexible expansion, enhanced observability and data reliability, and improved UAV accuracy.

[0088] In this embodiment of the invention, step 210, which involves acquiring the original laser point cloud data frame based on the solid-state lidar, includes:

[0089] Based on the phase of the rotatable bracket, the field of view corresponding to one rotation of the solid-state lidar is divided into multiple corner windows, and the boundary between two adjacent corner windows is taken as the corner window boundary.

[0090] The solid-state lidar rotates under the drive of a rotatable bracket and acquires the original point cloud.

[0091] The original point cloud is divided into multiple quasi-spiral subframes according to the corner window boundary;

[0092] The plurality of quasi-helical subframes constitute the original laser point cloud data frame.

[0093] In this embodiment of the invention, the width constraint condition of the corner window is:

[0094] The linear displacement of each quasi-spiral subframe does not exceed the maximum linear displacement, and the angular displacement of each quasi-spiral subframe does not exceed the maximum angular displacement.

[0095] In this embodiment of the invention, dividing the field of view corresponding to one rotation of the solid-state lidar into multiple corner windows according to the phase of the rotatable bracket includes:

[0096] The width of the corner window is adaptively adjusted according to the degree of shaking during the flight of the drone;

[0097] The greater the shaking during flight, the smaller the width of the corner window.

[0098] In this embodiment of the invention, the width of the corner window is adaptively adjusted. Unlike the non-rotation scheme that only captures frames according to time periods, the method of the present invention limits the upper limit of spatiotemporal distortion of a single frame, which is beneficial to subsequent distortion correction and registration.

[0099] In this embodiment of the invention, the width constraint condition of the corner window is:

[0100]

[0101] The phase span of the corner window is determined as follows:

[0102]

[0103] in, The width of the corner window. For the maximum linear displacement, The average linear velocity of the rotating electric motor. For the maximum angular displacement, The average angular velocity of the rotating motor. For lidar radio frequency, The minimum number of points in a single quasi-spiral subframe. For phase span, For the first The initial phase of a quasi-spiral subframe For the first The initial phase of a quasi-spiral subframe.

[0104] In this embodiment of the invention, the original laser point cloud data frame and the IMU data have timestamps.

[0105] Step 230, the preprocessing of the original laser point cloud data frame and the IMU data to obtain preprocessed data includes:

[0106] Each corner window boundary is taken as the ending phase of the current quasi-spiral subframe and the starting phase of the next quasi-spiral subframe, and the starting phase of the current quasi-spiral subframe is taken as the reference time.

[0107] Based on the timestamp, the original point cloud in each of the quasi-spiral subframes is reverted to the corresponding reference time according to the IMU data to obtain the reverted data frame;

[0108] Based on the IMU rotation rate, the time deviation of the data frame after the rollback is compensated to remove distortion in the point cloud data and obtain the distortion-free data frame.

[0109] The distortion-reduced data frame is filtered using voxel filtering, statistical cluster point removal filtering, and intensity threshold filtering to obtain preprocessed data.

[0110] The phase of the callback data frame is aligned with that of the IMU data.

[0111] The original point cloud within each quasi-spiral subframe is reverted to the corresponding reference time based on the IMU data for relative alignment, which is beneficial for improving the accuracy of subsequent processing.

[0112] In this embodiment of the invention, the distortion-reduced data frame is filtered according to voxel filtering, statistical rich cluster removal filtering, and intensity threshold filtering to obtain preprocessed data, including:

[0113] Based on the phase weight of the distorted data frame, the curvature and normal vectors are obtained within the quasi-spiral subframe of the distorted data frame.

[0114] Based on curvature and normal vector, the radial direction is widened and the tangential direction is tightened when the sampling rate is reduced;

[0115] Filter the downsampled and distortion-corrected data.

[0116] In this embodiment of the invention, filtering the distortion-free data frame can alleviate the anisotropic sampling bias caused by the rotation of the solid-state lidar and improve geometric consistency.

[0117] In this embodiment of the invention, the step of compensating for the time deviation of the data frame after the callback based on the IMU rotation rate to remove distortion in the point cloud data and obtain a distortion-free data frame includes:

[0118] The encoder angular velocity is obtained based on the IMU rotation rate;

[0119] Based on the pre-integration of the IMU data and the encoder angular velocity, the relative pose of the point cloud of the post-rewind data frame at the reference time of the quasi-helical subframe is obtained.

[0120] By transforming coordinates, the point clouds of all the data frames after the rollback are transformed to the radar coordinate system or the body coordinate system to obtain spatiotemporally consistent corrected point clouds, which are used as the distortion-free data frames.

[0121] In this embodiment of the invention, the step of transforming the point cloud of all the dialed-back data frames to the radar coordinate system or the body coordinate system through coordinate transformation includes:

[0122] Obtain the extrinsic parameter matrix between the solid-state lidar and the machine-mounted IMU;

[0123] Obtain the current angle of the rotatable bracket;

[0124] Based on the current angle and current extrinsic parameter matrix of the rotatable bracket, the point cloud of the data frame after the rollback is mapped from the radar coordinate system to the body coordinate system, or from the body coordinate system to the radar coordinate system.

[0125] In this embodiment of the invention, step 240, obtaining the UAV pose based on the preprocessed data, includes:

[0126] Based on the pre-integration of the IMU data, the initial values ​​of inter-frame motion are obtained, and the state prediction is obtained based on the initial values ​​of inter-frame motion.

[0127] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the coarse pose;

[0128] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map;

[0129] The local sub-map is precisely registered with the historical sub-map to obtain the accurate pose.

[0130] In this embodiment of the invention, step 250, obtaining a two-layer map based on the UAV pose and the preprocessed data, includes:

[0131] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the sparse feature map. The sparse feature map includes: line features, area features, feature covariance, phase cover histogram of features, and number of observations.

[0132] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map.

[0133] The local sub-map is precisely registered with the historical sub-map to obtain the visualized dense voxel map.

[0134] In this embodiment of the invention, step 250, obtaining a two-layer map based on the UAV pose and the preprocessed data, further includes:

[0135] The coarse pose and IMU prediction are tightly coupled and fused in the iterative error state Kalman filter to obtain the optimal state.

[0136] Based on the optimal state, the current keyframe in the iterative error state Kalman filter is projected onto the global coordinate system and fused into the previous two-layer map to obtain the current two-layer map.

[0137] like Figure 3 As shown, the present invention also provides a drone fusion mapping and positioning device, wherein the drone is equipped with a solid-state LiDAR and a rotatable support, the rotatable support driving the solid-state LiDAR to rotate, and the device includes:

[0138] Point cloud acquisition unit 310 is used to acquire raw laser point cloud data frames based on solid-state lidar.

[0139] IMU data acquisition unit 320 is used to acquire IMU data based on the airframe IMU of the UAV;

[0140] The preprocessing unit 330 is used to preprocess the original laser point cloud data frame and the IMU data to obtain preprocessed data;

[0141] Pose acquisition unit 340 is used to acquire the pose of the UAV based on the preprocessed data;

[0142] The map acquisition unit 350 is used to acquire a two-layer map based on the UAV pose and the preprocessed data;

[0143] The two-layer map includes a sparse feature map and a visualized dense voxel map.

[0144] In embodiments of the present invention, the point cloud acquisition unit 310 is further configured to:

[0145] Based on the phase of the rotatable bracket, the field of view corresponding to one rotation of the solid-state lidar is divided into multiple corner windows, and the boundary between two adjacent corner windows is taken as the corner window boundary.

[0146] The solid-state lidar rotates under the drive of a rotatable bracket and acquires the original point cloud.

[0147] The original point cloud is divided into multiple quasi-spiral subframes according to the corner window boundary;

[0148] The plurality of quasi-helical subframes constitute the original laser point cloud data frame.

[0149] In an embodiment of the present invention, the width constraint condition of the corner window is:

[0150] The linear displacement of each quasi-spiral subframe does not exceed the maximum linear displacement, and the angular displacement of each quasi-spiral subframe does not exceed the maximum angular displacement.

[0151] In embodiments of the present invention, the point cloud acquisition unit 310 is further configured to:

[0152] The width of the corner window is adaptively adjusted according to the degree of shaking during the flight of the drone;

[0153] The greater the shaking during flight, the smaller the width of the corner window.

[0154] In an embodiment of the present invention, the width constraint condition of the corner window is:

[0155]

[0156] The phase span of the corner window is determined as follows:

[0157]

[0158] in, The width of the corner window. For the maximum linear displacement, The average linear velocity of the rotating electric motor. For the maximum angular displacement, The average angular velocity of the rotating motor. For lidar radio frequency, The minimum number of points in a single quasi-spiral subframe. For phase span, For the first The initial phase of a quasi-spiral subframe For the first The initial phase of a quasi-spiral subframe.

[0159] In embodiments of the present invention, the original laser point cloud data frame and the IMU data have timestamps, and the preprocessing unit 330 is further configured to:

[0160] Each corner window boundary is taken as the ending phase of the current quasi-spiral subframe and the starting phase of the next quasi-spiral subframe, and the starting phase of the current quasi-spiral subframe is taken as the reference time.

[0161] Based on the timestamp, the original point cloud in each of the quasi-spiral subframes is reverted to the corresponding reference time according to the IMU data to obtain the reverted data frame;

[0162] Based on the IMU rotation rate, the time deviation of the data frame after the rollback is compensated to remove distortion in the point cloud data and obtain the distortion-free data frame.

[0163] The distortion-reduced data frame is filtered using voxel filtering, statistical cluster point removal filtering, and intensity threshold filtering to obtain preprocessed data.

[0164] The phase of the callback data frame is aligned with that of the IMU data.

[0165] In embodiments of the present invention, the preprocessing unit 330 is further configured to:

[0166] The encoder angular velocity is obtained based on the IMU rotation rate;

[0167] Based on the pre-integration of the IMU data and the encoder angular velocity, the relative pose of the point cloud of the post-rewind data frame at the reference time of the quasi-helical subframe is obtained.

[0168] By transforming coordinates, the point clouds of all the data frames after the rollback are transformed to the radar coordinate system or the body coordinate system to obtain spatiotemporally consistent corrected point clouds, which are used as the distortion-free data frames.

[0169] In embodiments of the present invention, the preprocessing unit 330 is further configured to:

[0170] Obtain the extrinsic parameter matrix between the solid-state lidar and the machine-mounted IMU;

[0171] Obtain the current angle of the rotatable bracket;

[0172] Based on the current angle and current extrinsic parameter matrix of the rotatable bracket, the point cloud of the data frame after the rollback is mapped from the radar coordinate system to the body coordinate system, or from the body coordinate system to the radar coordinate system.

[0173] In embodiments of the present invention, the pose acquisition unit 340 is further configured to:

[0174] Based on the pre-integration of the IMU data, the initial values ​​of inter-frame motion are obtained, and the state prediction is obtained based on the initial values ​​of inter-frame motion.

[0175] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the coarse pose;

[0176] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map;

[0177] The local sub-map is precisely registered with the historical sub-map to obtain the accurate pose.

[0178] In embodiments of the present invention, the map acquisition unit 350 is further configured to:

[0179] Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the sparse feature map. The sparse feature map includes: line features, area features, feature covariance, phase cover histogram of features, and number of observations.

[0180] Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map.

[0181] The local sub-map is precisely registered with the historical sub-map to obtain the visualized dense voxel map.

[0182] In embodiments of the present invention, the map acquisition unit 350 is further configured to:

[0183] The coarse pose and IMU prediction are tightly coupled and fused in the iterative error state Kalman filter to obtain the optimal state.

[0184] Based on the optimal state, the current keyframe in the iterative error state Kalman filter is projected onto the global coordinate system and fused into the previous two-layer map to obtain the current two-layer map.

[0185] In this embodiment of the invention, the UAV is equipped with a solid-state lidar and a rotatable support. The rotatable support drives the solid-state lidar to rotate, achieving a 360-degree field of view without the need for multiple solid-state lidars. This solves the problem of insufficient 360-degree left and right field of view for solid-state lidars, reduces the overall weight of the UAV, and improves its endurance. Furthermore, this embodiment of the invention provides a method for preprocessing the raw laser point cloud data frames from the solid-state lidar and the IMU data from the airframe IMU to obtain preprocessed data, and for acquiring the UAV pose and a two-layer map based on the preprocessed data. The two-layer map includes a sparse feature map and a visualized dense voxel map. The sparse feature map has relatively low computational cost, enabling real-time localization and mapping with low computational resources. The visualized dense voxel map combines visualization with the advantages of dense mapping and voxel modeling, offering high precision, flexible expansion, and enhanced observability and data reliability.

[0186] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the following method: acquiring raw laser point cloud data frames based on a solid-state lidar; acquiring IMU data based on the UAV's airframe IMU; preprocessing the raw laser point cloud data frames and the IMU data to obtain preprocessed data; acquiring the UAV pose based on the preprocessed data; and acquiring a two-layer map based on the UAV pose and the preprocessed data. The two-layer map includes a sparse feature map and a visualized dense voxel map.

[0187] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the following method: acquiring raw laser point cloud data frames from a solid-state lidar; acquiring IMU data from the UAV's airframe IMU; preprocessing the raw laser point cloud data frames and the IMU data to obtain preprocessed data; acquiring the UAV pose based on the preprocessed data; and acquiring a two-layer map based on the UAV pose and the preprocessed data; wherein the two-layer map includes a sparse feature map and a visualized dense voxel map.

[0188] The aforementioned UAV-integrated mapping and positioning method achieves the beneficial effect of solving the technical problems raised in the background section.

[0189] Figure 2 This is a flowchart illustrating a UAV fusion mapping and localization method in one embodiment. It should be understood that, although... Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0190] Figure 4 An internal structural diagram of a computer device in one embodiment is shown. Specifically, this computer device may be... Figure 1 Server 120 in the middle. For example... Figure 4As shown, the computer device includes a processor, memory, network interface, input device, and display screen connected via a system bus. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and may also store computer programs. When executed by the processor, these programs enable the processor to implement a UAV fusion mapping and localization method. The internal memory may also store computer programs, which, when executed by the processor, enable the processor to implement the UAV fusion mapping and localization method. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0191] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0192] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0193] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0194] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for fusion mapping and positioning of unmanned aerial vehicles (UAVs), characterized in that, The drone is equipped with a solid-state lidar and a rotatable support, the rotatable support driving the solid-state lidar to rotate, and the method includes: Obtain the original laser point cloud data frame from the solid-state lidar; IMU data is acquired based on the drone's airframe IMU; The original laser point cloud data frame and the IMU data are preprocessed to obtain preprocessed data; The UAV pose is obtained based on the preprocessed data; Based on the UAV pose and the preprocessed data, a two-layer map is obtained; The two-layer map includes a sparse feature map and a visualized dense voxel map; The step of acquiring the original laser point cloud data frame based on the solid-state lidar includes: Based on the phase of the rotatable bracket, the field of view corresponding to one rotation of the solid-state lidar is divided into multiple corner windows, and the boundary between two adjacent corner windows is used as the corner window boundary. The solid-state lidar rotates under the drive of a rotatable bracket and acquires the original point cloud. The original point cloud is divided into multiple quasi-spiral subframes based on the corner window boundary. The plurality of quasi-helical subframes constitute the original laser point cloud data frame; The width constraint condition for the corner window is: The phase span of the corner window is determined as follows: in, The width of the corner window. For the maximum linear displacement, The average linear velocity of the rotating electric motor. For the maximum angular displacement, The average angular velocity of the rotating motor. For lidar radio frequency, The minimum number of points in a single quasi-spiral subframe. For phase span, For the first The initial phase of a quasi-spiral subframe For the first The initial phase of a quasi-spiral subframe.

2. The method according to claim 1, characterized in that, The width constraint condition for the corner window is: The linear displacement of each quasi-spiral subframe does not exceed the maximum linear displacement, and the angular displacement of each quasi-spiral subframe does not exceed the maximum angular displacement.

3. The method according to claim 1, characterized in that, The method of dividing the field of view corresponding to one rotation of the solid-state lidar into multiple corner windows based on the phase of the rotatable bracket includes: The width of the corner window is adaptively adjusted according to the degree of shaking during the flight of the drone; The greater the shaking during flight, the smaller the width of the corner window.

4. The method according to claim 1, characterized in that, The original laser point cloud data frame and the IMU data have timestamps. The preprocessing of the original laser point cloud data frame and the IMU data to obtain preprocessed data includes: Each corner window boundary is taken as the ending phase of the current quasi-spiral subframe and the starting phase of the next quasi-spiral subframe, and the starting phase of the current quasi-spiral subframe is taken as the reference time. Based on the timestamp, the original point cloud in each of the quasi-spiral subframes is reverted to the corresponding reference time according to the IMU data to obtain the reverted data frame; Based on the IMU rotation rate, the time deviation of the data frame after the rollback is compensated to remove distortion in the point cloud data and obtain the distortion-free data frame. The distortion-reduced data frame is filtered using voxel filtering, statistical cluster point removal filtering, and intensity threshold filtering to obtain preprocessed data. The phase of the callback data frame is aligned with that of the IMU data.

5. The method according to claim 4, characterized in that, The step of compensating for the time deviation of the data frame after the rollback based on the IMU rotation rate to remove distortion in the point cloud data and obtain a distortion-free data frame includes: The encoder angular velocity is obtained based on the IMU rotation rate; Based on the pre-integration of the IMU data and the encoder angular velocity, the relative pose of the point cloud of the post-rewind data frame at the reference time of the quasi-helical subframe is obtained. By transforming coordinates, the point clouds of all the data frames after the rollback are transformed to the radar coordinate system or the body coordinate system to obtain spatiotemporally consistent corrected point clouds, which are used as the distortion-free data frames.

6. The method according to claim 5, characterized in that, The step of transforming the point cloud of all the recalled data frames to the radar coordinate system or the body coordinate system through coordinate transformation includes: Obtain the extrinsic parameter matrix between the solid-state lidar and the machine-mounted IMU; Obtain the current angle of the rotatable bracket; Based on the current angle and current extrinsic parameter matrix of the rotatable bracket, the point cloud of the data frame after the rollback is mapped from the radar coordinate system to the body coordinate system, or from the body coordinate system to the radar coordinate system.

7. The method according to claim 1, characterized in that, The step of obtaining the UAV pose based on the preprocessed data includes: Based on the pre-integration of the IMU data, the initial values ​​of inter-frame motion are obtained, and the state prediction is obtained based on the initial values ​​of inter-frame motion. Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the coarse pose; Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map; The local sub-map is precisely registered with the historical sub-map to obtain the accurate pose.

8. The method according to claim 7, characterized in that, The step of obtaining a two-layer map based on the UAV pose and the preprocessed data includes: Based on the state prediction, the current quasi-spiral subframe is quickly registered with the adjacent quasi-spiral subframes to obtain the sparse feature map. The sparse feature map includes: line features, area features, feature covariance, phase cover histogram of features, and number of observations. Based on the coarse pose, multiple consecutive quasi-spiral subframes are aggregated to form a local sub-map; The local sub-map is precisely registered with the historical sub-map to obtain the visualized dense voxel map.

9. The method according to claim 8, characterized in that, The step of obtaining a two-layer map based on the UAV pose and the preprocessed data further includes: The coarse pose and IMU prediction are tightly coupled and fused in the iterative error state Kalman filter to obtain the optimal state. Based on the optimal state, the current keyframe in the iterative error state Kalman filter is projected onto the global coordinate system and fused into the previous two-layer map to obtain the current two-layer map.

10. A UAV fusion mapping and positioning device, characterized in that, The drone is equipped with a solid-state lidar and a rotatable support, the rotatable support driving the solid-state lidar to rotate, and the device applies the method as described in any one of claims 1 to 9, the device comprising: The point cloud acquisition unit is used to acquire raw laser point cloud data frames based on the solid-state lidar. The IMU data acquisition unit is used to acquire IMU data based on the UAV's airframe IMU. The preprocessing unit is used to preprocess the original laser point cloud data frame and the IMU data to obtain preprocessed data; The pose acquisition unit is used to acquire the pose of the UAV based on the preprocessed data. The map acquisition unit is used to acquire a two-layer map based on the UAV pose and the preprocessed data; The two-layer map includes a sparse feature map and a visualized dense voxel map.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

Images