2D laser mapping method and device based on multi-sensor fusion, equipment and medium

Abstract

This application discloses a 2D laser mapping method, apparatus, device, and medium based on multi-sensor fusion. The method includes: acquiring laser point cloud data, inertial measurement data, and wheel speed data of the robot in the current frame; matching the laser point cloud data with a local map to determine laser matching constraints; performing pre-integration processing on the inertial measurement data to determine inertial measurement constraints; constructing a wheeled vehicle motion model based on the wheel speed data and determining wheel speed motion constraints based on the wheeled vehicle motion model; performing tightly coupled joint optimization based on the laser matching constraints, inertial measurement constraints, and wheel speed motion constraints to obtain the robot's pose data, and updating the global map based on the pose data. This technical solution achieves high-precision and robust robot localization and mapping through the tight coupling and deep fusion of lidar, IMU, and wheel speed meter, providing key technical support for reliable autonomous operation of robots in complex dynamic environments.

Description

Technical Field

[0001] This application relates to the field of autonomous localization and mapping technology for mobile robots, and in particular to a 2D laser mapping method, apparatus, device and medium based on multi-sensor fusion. Background Technology

[0002] With the increasing prevalence of mobile robots in industrial, commercial, and household settings, such as robotic vacuum cleaners, AGVs for logistics, and service robots, achieving simultaneous localization and mapping (SMR) in unknown environments and autonomous navigation based on this mapping has become a key technological bottleneck.

[0003] Currently, in the field of Simultaneous Localization and Mapping (SLAM) for mobile robots, the following solutions are commonly used. For example, a single LiDAR is used to calculate pose changes and build a map through inter-frame matching. However, this solution is prone to matching failures in scenarios with degraded features, such as long corridors or open environments, leading to unstable localization or even map construction failure. Another example is the addition of wheel speed sensors to SLAM systems to provide incremental mileage information in order to improve the continuity of short-term motion estimation. However, situations such as wheel slippage, uneven ground, or sudden steering changes can cause severe cumulative errors, reducing the overall consistency of the map.

[0004] Therefore, how to provide a technical solution that can improve the accuracy of real-time localization and map building of mobile robots is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a 2D laser mapping method, device, equipment, and medium based on multi-sensor fusion. By tightly coupling and deeply fusing lidar, IMU, and wheel speed meter, it achieves high-precision and robust robot localization and mapping, providing key technical support for the reliable autonomous operation of robots in complex dynamic environments.

[0006] According to one aspect of this application, a 2D laser mapping method based on multi-sensor fusion is provided, the method comprising: The robot acquires laser point cloud data, inertial measurement data, and wheel speed data in the current frame; wherein the laser point cloud data is collected by a 2D lidar mounted on the robot, the inertial measurement data is collected by an inertial measurement unit mounted on the robot, and the wheel speed data is collected by a wheel speed meter mounted on the robot. The laser point cloud data is matched with the local map to determine the laser matching constraints; The inertial measurement data is pre-integrated to determine the inertial measurement constraints; A wheeled vehicle motion model is constructed based on the wheel speed data, and wheel speed motion constraints are determined based on the wheeled vehicle motion model. The robot's pose data is obtained by performing tight-coupled joint optimization based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint, and the global map is updated based on the pose data.

[0007] According to another aspect of this application, a 2D laser mapping device based on multi-sensor fusion is provided, characterized in that the device comprises: The data acquisition module is used to acquire the robot's laser point cloud data, inertial measurement data, and wheel speed data in the current frame; wherein, the laser point cloud data is collected by the 2D LiDAR mounted on the robot, the inertial measurement data is collected by the inertial measurement unit mounted on the robot, and the wheel speed data is collected by the wheel speed meter mounted on the robot. The laser constraint module is used to match the laser point cloud data with the local map to determine the laser matching constraints; The IMU constraint module is used to pre-integrate the inertial measurement data to determine the inertial measurement constraints. The motion constraint module is used to construct a wheeled vehicle motion model based on the wheel speed data, and to determine wheel speed motion constraints based on the wheeled vehicle motion model. The map update module is used to perform tightly coupled joint optimization based on the laser matching constraints, the inertial measurement constraints, and the wheel speed motion constraints to obtain the robot's pose data, and update the global map based on the pose data.

[0008] According to another aspect of this application, an electronic device is provided, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the 2D laser mapping method based on multi-sensor fusion as described in any embodiment of this application.

[0009] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the 2D laser mapping method based on multi-sensor fusion as described in any embodiment of this application.

[0010] According to another aspect of this application, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the 2D laser mapping method based on multi-sensor fusion as described in any embodiment of this application.

[0011] The technical solution provided in this application acquires the robot's laser point cloud data, inertial measurement data, and wheel speed data in the current frame; matches the laser point cloud data with a local map to determine laser matching constraints; pre-integrates the inertial measurement data to determine inertial measurement constraints; constructs a wheeled vehicle motion model based on the wheel speed data, and determines wheel speed motion constraints based on the wheeled vehicle motion model; and performs tightly coupled joint optimization based on the laser matching constraints, inertial measurement constraints, and wheel speed motion constraints to obtain the robot's pose data, and updates the global map based on the pose data. This technical solution, through the tight coupling and deep fusion of LiDAR, IMU, and wheel speed meter, achieves high-precision and robust robot localization and mapping, providing key technical support for the reliable autonomous operation of robots in complex dynamic environments.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a 2D laser mapping method based on multi-sensor fusion provided in Embodiment 1 of this application.

[0015] Figure 2 This is a flowchart of a 2D laser mapping method based on multi-sensor fusion provided in Embodiment 2 of this application.

[0016] Figure 3 This is a schematic diagram of a map based on the open-karto mapping method provided in Embodiment 2 of this application.

[0017] Figure 4 This is a schematic diagram of a 2D laser mapping method based on multi-sensor fusion provided in Embodiment 2 of this application.

[0018] Figure 5 This is a comparison chart showing the computation time of a mapping method provided in Embodiment 2 of this application.

[0019] Figure 6 This is a schematic diagram of a 2D laser mapping device based on multi-sensor fusion provided in Embodiment 2 of this application.

[0020] Figure 7 This is a schematic diagram of the structure of a device for implementing a 2D laser mapping method based on multi-sensor fusion according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "current," "critical," "historical," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0024] Example 1 Figure 1 This is a flowchart of a 2D laser mapping method based on multi-sensor fusion provided in Embodiment 1 of this application. This embodiment is applicable to the real-time localization and map building of mobile robots. The method can be executed by a 2D laser mapping device based on multi-sensor fusion, which can be implemented in hardware and / or software. This 2D laser mapping device based on multi-sensor fusion can be configured in a device with data processing capabilities. Figure 1 As shown, the method includes the following steps.

[0025] S110. Acquire the robot's laser point cloud data, inertial measurement data, and wheel speed data in the current frame. The laser point cloud data is collected by a 2D LiDAR mounted on the robot, the inertial measurement data is collected by an inertial measurement unit mounted on the robot, and the wheel speed data is collected by a wheel speed meter mounted on the robot.

[0026] The current frame refers to a predefined time window. Since the 2D lidar, inertial measurement unit, and wheel speed meter operate at different frequencies (the 2D lidar has a lower frequency, while the inertial measurement unit and wheel speed meter have higher frequencies), the current frame in this application is based on the scanning cycle of the 2D lidar, typically the time period between the end of the previous laser scan and the end of the current laser scan.

[0027] Specifically, the 2D LiDAR, inertial measurement unit, and wheel speedometer on the robot can be timestamped first to ensure that the data collected by each sensor corresponds to the same moment, thus ensuring the consistency of subsequent data fusion. The current frame is determined based on the scanning cycle of the 2D LiDAR, and the entire packet of LiDAR point cloud data within the current frame is acquired, along with the inertial measurement data and wheel speed data collected by the inertial measurement unit and wheel speedometer within the current frame.

[0028] Laser point cloud data is a collection of point cloud data for all measurement points within the scanned area, represented in polar coordinates. Each point typically contains distance and angle values. Inertial measurement data mainly includes triaxial angular velocity and triaxial acceleration. The unit of triaxial angular velocity is rad / s, reflecting the rotational rate around each coordinate axis. The unit of triaxial acceleration is m / s². 2 This reflects the linear acceleration along each coordinate axis, including the gravitational component. Wheel speed data mainly includes the left wheel speed and the right wheel speed, which can be obtained by multiplying the encoder pulse frequency by a calibrated wheel radius.

[0029] S120. Match the laser point cloud data with the local map to determine the laser matching constraints.

[0030] The local map is a partial area of ​​the global map constructed from historical data. In this application, matching laser point cloud data with the local map reduces computational load and avoids interference from irrelevant areas.

[0031] Among them, laser matching constraints represent nonlinear geometric observation constraints on the robot's pose state. They can be used to quantify the inconsistency between the currently estimated robot pose and the real environment structure, provide global geometric constraints, correct inertial measurement unit drift and wheel speedometer drift, and determine the consistency of the final map.

[0032] Specifically, the laser point cloud data of the current frame is transformed into the world coordinate system, and a spatial index data structure, such as a KD-Tree, is established for the local map; each point in the laser point cloud data is searched and matched in the spatial index data structure of the local map to determine the pose transformation of each point; and laser matching constraints are determined based on the pose transformation of each point.

[0033] S130. Perform pre-integration processing on the inertial measurement data to determine the inertial measurement constraints.

[0034] Pre-integration processing refers to calculating the relative motion increments from inertial measurement data. Specifically, based on the inertial measurement data of the current frame, pre-integrated quantities such as rotation increments, velocity increments, and displacement increments between adjacent time points can be calculated in advance. Inertial measurement constraints refer to the constraints formed by introducing the pre-integrated quantities as observation factors into the optimization framework.

[0035] In this application, it is assumed that the times corresponding to two adjacent frames are t. i and t j During this period, there are several inertial measurement data. Using the high-frequency angular velocity and acceleration raw measurements of the IMU, the robot's attitude, velocity and position at the current moment are derived from the state at the previous moment by integration.

[0036] Specifically, the angular velocity is integrated to obtain the attitude change, which updates the robot's attitude at the current moment. The attitude is represented by a rotation matrix, indicating the rotation relationship between the robot's coordinate system and the world coordinate system. The robot's attitude at the current moment is used to perform coordinate compensation on the acceleration, yielding the motion acceleration in the world coordinate system. This acceleration is integrated once to update the robot's velocity at the current moment, and integrated twice to update the robot's position at the current moment.

[0037] By pre-integrating the inertial measurement data, a dense stream of inertial measurement data can be converted into inertial measurement constraints connecting two frames. During the interval between two lidar scans, the inertial measurement unit continuously propagates its state at a high frequency, providing the system with the latest pose and velocity predictions.

[0038] S140. Construct a wheeled vehicle motion model based on the wheel speed data, and determine wheel speed motion constraints based on the wheeled vehicle motion model.

[0039] Taking a differential-drive robot as an example, the robot's instantaneous motion can be viewed as rotation around an instantaneous center of rotation. The linear velocity of the robot's center of mass is determined based on the left and right wheel velocities, and the angular velocity of the robot around the vertical axis is determined based on the left and right wheel velocities and the wheelbase. The rotation increment is obtained by integrating the linear velocity, and the translation increment is obtained by integrating the angular velocity. The relative pose transformation matrix of the robot in the current frame is constructed based on the rotation and translation increments, i.e., the wheeled vehicle motion model. The wheel speed motion constraints are determined based on the robot's relative motion calculated from the wheeled vehicle motion model and the robot's relative motion estimated from the state variables.

[0040] In this application, wheel speed motion constraints are constructed based on wheel speed data, especially rotation increments, which provide stable planar motion priors for robot real-time localization. This maintains motion continuity when laser features are sparse or inertial measurement data is noisy, and can dynamically correct wheel speed errors and slippage errors through tight coupling optimization.

[0041] S150. Perform tight-coupled joint optimization based on the laser matching constraint, the inertial measurement constraint and the wheel speed motion constraint to obtain the robot's pose data, and update the global map based on the pose data.

[0042] Tightly coupled joint optimization refers to the process in multi-sensor fusion systems based on graph optimization or iterative Kalman filtering frameworks, where laser matching constraints, inertial measurement constraints, and wheel speed motion constraints from different sensors are constructed into a unified constraint factor through their corresponding physical or geometric models. Based on this unified constraint factor, an optimal robot state sequence is found that maximizes the joint probability of observations from all sensors within that state sequence.

[0043] Specifically, the objective function can be constructed by the weighted sum of squares of laser matching constraints, inertial measurement constraints, and wheel speed motion constraints. The minimum value of this objective function is then obtained, which represents the robot's pose data. Using the robot pose data optimized by multiple sensors, the laser point cloud data of the current frame is transformed from the sensor coordinate system to the world coordinate system, and the transformed laser point cloud data is then fused into the global map.

[0044] The fusion method varies depending on the different representations of the global map. For example, if the global map is a two-dimensional occupancy grid map, each world coordinate point is mapped to its corresponding grid. An inverse sensor model is used to update the occupancy probability of the grid and its rays, resulting in a global map that is ultimately a collection of probability values ​​for each grid. Alternatively, if the global map is a feature map, geometric features are extracted from the laser point cloud data of the current frame. These extracted features are then transformed into the world coordinate system using robot pose data and associated with and merged with existing features in the global map.

[0045] In this application, laser point cloud data provides a global geometric reference but has a low frequency and is distorted; inertial measurement data provides high-frequency attitude and acceleration information but suffers from integral drift; and wheel speed data provides accurate planar relative motion but is constrained by kinematic models and slippage. Through tight coupling optimization, the advantages of the three are complemented and mutually corrected, thereby obtaining a robust and accurate state estimate.

[0046] Optionally, the robot's pose data can be obtained by performing tightly coupled joint optimization based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint, including: constructing constraint equations based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint; and performing nonlinear optimization on the constraint equations based on iterative Kalman filtering to determine the robot's pose data.

[0047] In this application, a pose state vector is first constructed, and the laser matching constraint, inertial measurement constraint, and wheel speed motion constraint are unified into the same iterative Kalman filter optimization framework to form a joint constraint of pose state variables, i.e., constraint equations. Through iterative Kalman filtering, the robot's position, attitude, velocity, inertial measurement unit acceleration bias, and acceleration offset are obtained.

[0048] Specifically, inertial measurement data can be used to derive the predicted value of the current state from the optimal estimate of the previous moment; the predicted value is corrected using laser matching constraints and wheel speed motion constraints; after each update, the linearized approximation is recalculated based on the new state estimate and corrected again until the result converges to the optimal value, and finally the robot's position, attitude, velocity, inertial measurement unit acceleration bias and acceleration bias are output.

[0049] Optionally, updating the global map based on the pose data includes: constructing keyframe data based on the pose data; and inserting the keyframe data into the global map to update the global map.

[0050] Among them, keyframe data can be obtained by optimizing the laser point cloud data of the current frame based on the optimized robot pose data.

[0051] Specifically, based on pose data, the laser point cloud data of the current frame can be transformed from the 2D lidar coordinate system to the world coordinate system; feature point extraction, noise filtering, and downsampling processing are performed on the transformed point cloud data to obtain key frame data; the key frame data is then inserted into the global map to update the global map.

[0052] Feature points can be corner points, edge points, planar points, etc. Downsampling can be done using voxel grid filtering, which divides the space into tiny voxels, retaining only one point within each voxel, such as the centroid.

[0053] There are two ways to insert keyframe data into the global map. One is direct stitching, where the keyframe data is directly added to the global point cloud set. The other is incremental fusion, where for occupied grid maps, the occupancy probability of each grid is updated using an inverse sensor model. For feature maps, new features are associated with and merged with existing features.

[0054] Optionally, after updating the global map based on the pose data, the method further includes: calculating the similarity between the keyframe data and each historical keyframe data to determine the similarity calculation result; and performing loop closure detection based on each of the similarity calculation results.

[0055] Among them, loop closure detection refers to identifying whether the robot has returned to a previously visited location, which can effectively reduce cumulative drift and significantly improve the accuracy of localization and mapping.

[0056] Specifically, when keyframe data is created, one or more global descriptors, such as M2DP, can be extracted from the keyframe data to generate a global descriptor vector through the projection histograms of the point cloud on different planes.

[0057] Historical keyframe data is determined based on temporal and spatial proximity. Temporal proximity refers to an exclusion rule based on time order. It assumes that the robot cannot move from one location to a distant location and return to its original position within a very short time interval; therefore, frames that are too close in time cannot form a loop. Specifically, a time threshold can be predefined, and historical keyframe data within a nearby time period is filtered out based on the timestamp of the current keyframe data and the time threshold. Spatial proximity refers to a focusing rule based on coarse position information. It assumes that the robot's movement is continuous; therefore, possible loop locations are most likely to appear in the vicinity of the robot's current position inferred from odometry without global optimization. Specifically, a coarse position can be predefined for each keyframe data, and a search is performed from historical keyframe data based on the coarse position of the current keyframe data and a preset spatial search radius.

[0058] Similarity calculation results can refer to the geometric consistency, temporal consistency, and spatial consistency results between the current keyframe data and historical keyframe data.

[0059] Specifically, a point cloud registration algorithm can be used to match the current keyframe data with historical keyframe data to determine geometric consistency results, such as the proportion of successfully matched points to the total number of points. Spatial consistency results are determined based on whether the next frame data of the current frame and the next frame data of historical keyframes form a loop; temporal consistency results are determined based on whether the timestamps of the current keyframe data and historical keyframe data are too close.

[0060] Concurrent searching allows for the parallel processing of multiple keyframes or feature regions, giving the system better scalability.

[0061] This invention provides a 2D laser mapping method based on multi-sensor fusion. The method acquires laser point cloud data, inertial measurement data, and wheel speed data of the robot in the current frame; matches the laser point cloud data with a local map to determine laser matching constraints; pre-integrates the inertial measurement data to determine inertial measurement constraints; constructs a wheeled vehicle motion model based on the wheel speed data and determines wheel speed motion constraints based on the wheel speed model; and performs tightly coupled joint optimization based on the laser matching constraints, inertial measurement constraints, and wheel speed motion constraints to obtain the robot's pose data, and updates the global map based on the pose data. This technical solution achieves high-precision and robust robot localization and mapping through the tight coupling and deep fusion of lidar, IMU, and wheel speed measurement, providing key technical support for reliable autonomous operation of robots in complex dynamic environments.

[0062] Example 2 Figure 2 This is a flowchart of a 2D laser mapping method based on multi-sensor fusion provided in Embodiment 2 of this application. This embodiment is an optimization based on the above embodiment, specifically optimizing the process for determining laser matching constraints. Figure 2 As shown, the method in this embodiment specifically includes the following steps.

[0063] S210. Acquire the robot's laser point cloud data, inertial measurement data, and wheel speed data in the current frame. The laser point cloud data is collected by a 2D LiDAR mounted on the robot, the inertial measurement data is collected by an inertial measurement unit mounted on the robot, and the wheel speed data is collected by a wheel speed meter mounted on the robot.

[0064] S220. Perform distortion correction processing on the laser point cloud data to obtain distortion-corrected laser point cloud data.

[0065] Whether using a rotating or solid-state LiDAR, completing a full frame takes time. Within those 100 milliseconds, the robot carrying the LiDAR may have translated or rotated. This results in different points in the same frame of point cloud being acquired under different LiDAR poses. If these points are directly treated as being acquired at the same stationary moment, distorted LiDAR point cloud data will be obtained.

[0066] Specifically, high-frequency sensor data can be used to estimate the precise pose of the 2D lidar at the moment of laser emission at each laser point, and then all points can be projected onto the same coordinate system at the same reference moment to achieve distortion correction of the laser point cloud data.

[0067] Optionally, distortion correction processing is performed on the laser point cloud data to obtain distortion-corrected laser point cloud data, including: performing range filtering and reflection intensity filtering processing on the laser point cloud data to obtain preprocessed laser point cloud data; based on the timestamps of each laser point in the current frame of the preprocessed laser point cloud data, interpolating the preprocessed laser point cloud data according to the inertial measurement data to obtain distortion-corrected laser point cloud data.

[0068] Range filtering refers to removing invalid or unreliable measurement points. For example, points that are too close may be due to self-reflection or noise, while points that are too far away have larger measurement errors and may be outside the range of interest. Therefore, minimum and maximum values ​​are set to filter out noise.

[0069] Reflection intensity filtering refers to using the reflection intensity information returned by LiDAR to filter out unreliable points or retain only points on specific object surfaces. Reflection intensity is the intensity of the echo received by LiDAR, which is affected by factors such as the object's surface material, color, and angle of incidence. Therefore, by setting a reflection intensity threshold, only points within a certain reflection intensity range are retained.

[0070] In this application, high-frequency inertial measurement data is used to perform distortion correction on preprocessed laser point cloud data. Specifically, the inertial measurement data and laser point cloud data are first timestamped; starting from the beginning of the current frame, the angular velocity and acceleration in the inertial measurement data are pre-integrated to obtain a series of discrete poses; for each laser point, the pose at that moment is obtained by linear interpolation or spherical linear interpolation based on its timestamp; each laser point is transformed from the radar coordinate system at its acquisition time to the radar coordinate system at the reference time, thereby obtaining the distortion-corrected laser point cloud data.

[0071] S230. Match the distortion-corrected laser point cloud data with the local map to determine the laser matching constraints.

[0072] Specifically, the laser point cloud data after distortion correction can be matched with the local map using the ICP (Iterative Closest Point) method or the scan matching method to determine the laser matching constraints.

[0073] Optionally, matching the distortion-corrected laser point cloud data with a local map to determine laser matching constraints includes: matching the distortion-corrected laser point cloud data with a local map to determine the distance error between the observation point in the current frame and the corresponding geometric feature in the local map; and determining laser matching constraints based on the distance error.

[0074] Among them, distance error can refer to the distance error between points, the distance error between points and lines, or the distance error between points and surfaces.

[0075] Specifically, a least-squares problem, namely laser matching constraint, can be constructed based on at least one of the distance errors between points, between points and lines, and between points and surfaces. This problem minimizes the sum of the distance errors of all points by optimizing the pose of the current frame.

[0076] S240. Perform pre-integration processing on the inertial measurement data to determine the inertial measurement constraints.

[0077] S250. Construct a wheeled vehicle motion model based on the wheel speed data, and determine wheel speed motion constraints based on the wheeled vehicle motion model.

[0078] S260. Perform tight-coupled joint optimization based on the laser matching constraint, the inertial measurement constraint and the wheel speed motion constraint to obtain the robot's pose data, and update the global map based on the pose data.

[0079] This invention provides a 2D laser mapping method based on multi-sensor fusion. This method corrects the distortion of laser point cloud data to obtain distorted laser point cloud data. The distorted laser point cloud data is then matched with a local map to determine laser matching constraints. Based on these constraints, the mobile robot can achieve timely localization and map construction. This technical solution, by compensating for the influence of the robot's own motion during radar scanning, can uniformly correct point clouds collected in different actual poses within the same frame to the same reference time, effectively improving subsequent matching accuracy, map clarity, and system robustness.

[0080] Based on the above embodiments, a specific example will be used to explain the 2D laser mapping method based on multi-sensor fusion provided in this application.

[0081] For a certain region, a map is constructed using the existing open-karto method, such as... Figure 3 As shown. From Figure 3 As can be seen, there is a ghosting effect on the left side of the wall. This is because the scene is relatively empty and some walls are made of transparent glass, which causes degradation and affects the positioning accuracy.

[0082] The 2D laser mapping method based on multi-sensor fusion provided in this application was used to map it, such as... Figure 4 As shown. From Figure 4 You can see that the ghosting on the left side of the wall has disappeared.

[0083] To further verify the effectiveness of the 2D laser mapping method provided in this application, this application statistically analyzes the computation time of open-karto mapping, tightly coupled mapping, and the 2D laser mapping method provided in this application. Figure 5 As shown. From Figure 5 As can be seen, the blue line corresponds to the processing time of each keyframe in Open-Karto mapping. From laser odometry to backend optimization, the processing flow is sequential, therefore loop closure detection is required at certain times, leading to an increase in the processing time of a particular frame. The red line represents the computation time of tightly coupled mapping based on Open-Karto, with a significant reduction in the processing time of each frame. The green line represents the computation time of the 2D laser mapping provided in this application. This application incorporates parallel acceleration processing on top of tight coupling, further reducing the computation time, thus demonstrating the effectiveness of the multi-sensor fusion-based 2D laser mapping method provided in this application.

[0084] Example 3 Figure 6 This is a schematic diagram of a 2D laser mapping device based on multi-sensor fusion provided in Embodiment 4 of the present invention. Figure 6 As shown, the device includes: The data acquisition module 310 is used to acquire laser point cloud data, inertial measurement data and wheel speed data of the robot in the current frame; wherein, the laser point cloud data is collected by the 2D lidar mounted on the robot, the inertial measurement data is collected by the inertial measurement unit mounted on the robot, and the wheel speed data is collected by the wheel speed meter mounted on the robot. The laser constraint module 320 is used to match the laser point cloud data with the local map to determine the laser matching constraint; wherein, the laser matching constraint is used to represent the pose transformation constraint between the laser point cloud data and the local map. IMU constraint module 330 is used to perform pre-integration processing on the inertial measurement data to determine inertial measurement constraints; The motion constraint module 340 is used to construct a wheeled vehicle motion model based on the wheel speed data, and to determine wheel speed motion constraints based on the wheeled vehicle motion model. The map update module 350 is used to perform tightly coupled joint optimization based on the laser matching constraint, the inertial measurement constraint and the wheel speed motion constraint to obtain the robot's pose data, and update the global map based on the pose data.

[0085] The 2D laser mapping device based on multi-sensor fusion provided in this invention acquires laser point cloud data, inertial measurement data, and wheel speed data of the robot in the current frame; matches the laser point cloud data with a local map to determine laser matching constraints; pre-integrates the inertial measurement data to determine inertial measurement constraints; constructs a wheeled vehicle motion model based on the wheel speed data and determines wheel speed motion constraints based on the wheeled vehicle motion model; and performs tightly coupled joint optimization based on the laser matching constraints, inertial measurement constraints, and wheel speed motion constraints to obtain the robot's pose data, and updates the global map based on the pose data. This technical solution achieves high-precision and robust robot localization and mapping through the tight coupling and deep fusion of lidar, IMU, and wheel speed meter, providing key technical support for reliable autonomous operation of robots in complex dynamic environments.

[0086] Optionally, the laser constraint module 320 includes: The distortion correction unit is used to perform distortion correction processing on the laser point cloud data to obtain the distortion-corrected laser point cloud data. The data matching unit is used to match the distortion-corrected laser point cloud data with the local map to determine the laser matching constraints.

[0087] Optionally, the distortion correction unit includes: The data preprocessing subunit is used to perform range filtering and reflection intensity screening on the laser point cloud data to obtain preprocessed laser point cloud data. The data interpolation subunit is used to interpolate the preprocessed laser point cloud data based on the timestamps of each laser point in the current frame and according to the inertial measurement data to obtain the distortion-corrected laser point cloud data.

[0088] Optional, the data matching unit includes: The distance error determination subunit is used to match the distortion-corrected laser point cloud data with the local map to determine the distance error between the observation point in the current frame and the corresponding geometric feature in the local map. The laser matching constraint determination subunit is used to determine the laser matching constraint based on the distance error.

[0089] Optional, map update module 350, including: The constraint equation construction unit is used to construct constraint equations based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint. The pose data determination unit is used to perform nonlinear optimization solution of the constraint equations based on iterative Kalman filtering to determine the pose data of the robot.

[0090] Optionally, the map update module 350 also includes: A keyframe data construction unit is used to construct keyframe data based on the pose data; The map update unit is used to insert the keyframe data into the global map to update the global map.

[0091] Optionally, the device further includes: The frame similarity determination module is used to calculate the similarity between the keyframe data and each historical keyframe data after updating the global map based on the pose data, and to determine the similarity calculation result. The loop closure detection module is used to perform loop closure detection based on the similarity calculation results.

[0092] The 2D laser mapping device based on multi-sensor fusion provided in the embodiments of the present invention can execute the 2D laser mapping method based on multi-sensor fusion provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0093] Example 4 Figure 7 A schematic diagram of the structure of a device 10 that can be used to implement embodiments of this application is shown. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0094] like Figure 7 As shown, device 10 includes at least one processor 11 and a memory, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc., communicatively connected to at least one processor 11. The memory stores computer programs executable by at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 may also store various programs and data required for the operation of device 10. The processor 11, ROM 12, and RAM 13 are interconnected via bus 14. Input / output (I / O) interface 15 is also connected to bus 14.

[0095] Multiple components in device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as 2D laser mapping methods based on multi-sensor fusion.

[0097] In some embodiments, the 2D laser mapping method based on multi-sensor fusion can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the 2D laser mapping method based on multi-sensor fusion described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the 2D laser mapping method based on multi-sensor fusion by any other suitable means (e.g., by means of firmware).

[0098] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on a device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0102] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0103] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0104] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0105] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A 2D laser mapping method based on multi-sensor fusion, characterized in that, The method includes: The robot acquires laser point cloud data, inertial measurement data, and wheel speed data in the current frame; wherein the laser point cloud data is collected by a 2D lidar mounted on the robot, the inertial measurement data is collected by an inertial measurement unit mounted on the robot, and the wheel speed data is collected by a wheel speed meter mounted on the robot. The laser point cloud data is matched with the local map to determine the laser matching constraints; The inertial measurement data is pre-integrated to determine the inertial measurement constraints; A wheeled vehicle motion model is constructed based on the wheel speed data, and wheel speed motion constraints are determined based on the wheeled vehicle motion model. The robot's pose data is obtained by performing tight-coupled joint optimization based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint, and the global map is updated based on the pose data.

2. The method according to claim 1, characterized in that, The method further includes matching the laser point cloud data with a local map to determine laser matching constraints. The laser point cloud data is subjected to distortion correction processing to obtain distortion-corrected laser point cloud data; The distortion-corrected laser point cloud data is matched with the local map to determine the laser matching constraints.

3. The method according to claim 2, characterized in that, The laser point cloud data is subjected to distortion correction processing to obtain distortion-corrected laser point cloud data, including: The laser point cloud data is subjected to range filtering and reflection intensity screening to obtain preprocessed laser point cloud data; Based on the timestamps of each laser point in the current frame of the preprocessed laser point cloud data, the preprocessed laser point cloud data is interpolated according to the inertial measurement data to obtain the distortion-corrected laser point cloud data.

4. The method according to claim 2, characterized in that, The distortion-corrected laser point cloud data is matched with the local map to determine laser matching constraints, including: The distortion-corrected laser point cloud data is matched with the local map to determine the distance error between the observation point in the current frame and the corresponding geometric feature in the local map; The laser matching constraint is determined based on the distance error.

5. The method according to claim 1, characterized in that, Based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint, a tightly coupled joint optimization is performed to obtain the robot's pose data, including: Based on the laser matching constraint, the inertial measurement constraint, and the wheel speed motion constraint, a constraint equation is constructed; The robot's pose data are determined by nonlinear optimization of the constraint equations based on iterative Kalman filtering.

6. The method according to claim 1, characterized in that, The global map is updated based on the pose data, including: Keyframe data is constructed based on the pose data; The keyframe data is inserted into the global map to update the global map.

7. The method according to claim 6, characterized in that, After updating the global map based on the pose data, the method further includes: The similarity between the keyframe data and each historical keyframe data is calculated to determine the similarity calculation result; Based on the similarity calculation results, loop closure detection is performed.

8. A 2D laser mapping device based on multi-sensor fusion, characterized in that, The device includes: The data acquisition module is used to acquire the robot's laser point cloud data, inertial measurement data, and wheel speed data in the current frame; wherein, the laser point cloud data is collected by the 2D LiDAR mounted on the robot, the inertial measurement data is collected by the inertial measurement unit mounted on the robot, and the wheel speed data is collected by the wheel speed meter mounted on the robot. The laser constraint module is used to match the laser point cloud data with the local map to determine the laser matching constraints; The IMU constraint module is used to pre-integrate the inertial measurement data to determine the inertial measurement constraints. The motion constraint module is used to construct a wheeled vehicle motion model based on the wheel speed data, and to determine wheel speed motion constraints based on the wheeled vehicle motion model. The map update module is used to perform tightly coupled joint optimization based on the laser matching constraints, the inertial measurement constraints, and the wheel speed motion constraints to obtain the robot's pose data, and update the global map based on the pose data.

9. An electronic device, characterized in that, The device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the 2D laser mapping method based on multi-sensor fusion as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the 2D laser mapping method based on multi-sensor fusion as described in any one of claims 1-7.

Images