A real-time multimodal map generation method and system

Abstract

The present invention relates to the field of robot navigation and autonomous positioning technology, and discloses a real-time multimodal map generation method and system. The method synchronously acquires the device position and posture and the spatial point cloud through the tight coupling mechanism of the laser radar and the inertial measurement unit to form the raw data input. Based on this, the incremental point cloud registration technology is used to process the continuous point cloud stream. The dynamic spatial index architecture is used to realize the real-time organization and update of the point cloud to construct an accurate three-dimensional point cloud map. The three-dimensional point cloud is then input into the probabilistic octree model, and the logarithmic domain Bayesian update mechanism is used to analyze the spatial occupancy state to generate a three-dimensional probabilistic occupancy map. Finally, the three-dimensional occupancy map is subjected to spatial dimension conversion: the effective height interval is screened along the vertical direction, and the multi-level occupancy probabilities are integrated to generate a two-dimensional grid map, thereby realizing lossless compression from the three-dimensional environmental information to the two-dimensional navigation information.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation and autonomous positioning technology, and in particular to a real-time multimodal map generation method and system. Background Art

[0002] LiDAR (Light Detection and Ranging) technology has been widely used in environmental perception and mapping tasks. Traditional mapping methods typically employ a step-by-step process: first, pose data is acquired through loose coupling of the LiDAR and an inertial measurement unit (IMU), followed by a 3D point cloud map generated using feature matching algorithms (such as the LOAM series), and finally, a 2D raster map generated through offline conversion. However, existing methods have significant drawbacks: insufficient real-time performance, cumulative delays caused by serial processing of multiple modules, and difficulty meeting the real-time obstacle avoidance requirements of dynamic scenes; poor environmental adaptability, reliance on manually designed geometric features (such as edges / planes), prone to matching failures in open or symmetrical scenes, and a sharp drop in mapping accuracy; lack of multimodal support, and the separation of the 3D and 2D raster map generation processes, requiring the development of additional projection algorithms, increasing system complexity and errors; and high resource consumption. Global map reconstruction requires the storage of the entire historical point cloud, resulting in excessive memory usage and impracticality for deployment on embedded platforms.

[0003] Therefore, there is an urgent need for a real-time multimodal map generation method and system that is adaptable to complex environments and resource-efficient. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a real-time multimodal map generation method and system, which meets the real-time requirements of the embedded platform while ensuring the accuracy of environmental perception, and provides a technical basis for the autonomous operation of unmanned systems in complex dynamic scenarios.

[0005] The present invention provides a real-time multimodal map generation method, comprising the following steps:

[0006] S1, through the tight coupling of LiDAR and inertial measurement unit, obtain real-time pose data and original point cloud data;

[0007] S2. Based on the original point cloud data and the real-time pose data, generating a three-dimensional point cloud map by an incremental point cloud registration algorithm;

[0008] S3. Inputting the three-dimensional point cloud map into a probabilistic octree model and generating a three-dimensional probabilistic occupancy map through dynamic Bayesian probability updating;

[0009] S4. Performing height projection and probability fusion on the three-dimensional probability occupancy map to generate a two-dimensional grid map in real time.

[0010] Furthermore, in S2, the incremental point cloud registration algorithm uses an incremental distance k-dimensional tree to perform dynamic point cloud management, supporting dynamic insertion, deletion, balancing and downsampling operations of point clouds.

[0011] Furthermore, the management operation of the incremental distance k-dimensional tree includes:

[0012] The point cloud is downsampled and filtered by constraining the local map range through cubic space segmentation.

[0013] Furthermore, in S3, the update formula of the probabilistic octree model is:

[0014] ;

[0015] Where l(n) is the log probability ratio of node n, and P(n) represents the occupancy probability of node n; is the likelihood function of the sensor model, indicating that z is observed at the node n state t The conditional probability of z t is the original point cloud scanned by the lidar at time t; represents the posterior log probability accumulated to time t; n represents the node in the octree.

[0016] Furthermore, in the likelihood function of the sensor model:

[0017] The hit probability is set to 0.7, corresponding to the increment , then the update formula of the probabilistic octree model is: ;

[0018] The miss probability is set to 0.4, corresponding to the logarithmic increment Δ l miss ≈−0.405, then the update formula of the probabilistic octree model is: ;

[0019] The probabilistic octree model determines the occupancy state by using a truncation threshold: when the node occupancy probability P(n)>0.97, it is marked as occupied, and when P(n)<0.12, it is marked as idle.

[0020] Furthermore, in S4, the generation of the two-dimensional grid map includes:

[0021] S41. Set a height interval [min(z), max(z)] to filter three-dimensional voxels; S42. Calculate the two-dimensional grid probability based on the three-dimensional voxels using a probability fusion formula;

[0022] The probability fusion formula is: ;

[0023] in, l2D (x,y) represents the log-probability ratio of the two-dimensional grid; P 3D (x, y, z) represents the occupancy probability of a three-dimensional voxel, and z represents the height dimension index of the three-dimensional space; S43, the two-dimensional grid probability is normalized by a Sigmoid function to output a two-dimensional grid map.

[0024] Furthermore, the probability fusion introduces a time decay factor ɑ to perform exponential smoothing on the historical probability. The specific formula is:

[0025] ;in, Represents the two-dimensional grid probability value at time t; represents the two-dimensional grid probability value at time t-1; P new Indicates the new probability value of the 3D map projection of the current frame.

[0026] The present invention also provides a real-time multimodal map generation system for executing the above-mentioned real-time multimodal map generation method. The system is a detachable hardware system, including: a laser radar, an onboard computer, a power supply module, an external display screen, and a six-button Bluetooth keyboard;

[0027] The laser radar is tightly coupled with the inertial measurement unit to acquire real-time pose data and original point cloud data;

[0028] The onboard computer is used to execute the tightly coupled positioning and mapping algorithm and the multimodal map generation process, and receive the data from the laser radar through the data interface;

[0029] The external display screen is connected to the onboard computer via nomachine and is used to observe the map generation status in real time;

[0030] The six-button Bluetooth keyboard is used to trigger multimodal map generation and saving operations through key commands;

[0031] The power supply module is used to provide DC power to the laser radar, the onboard computer and the six-button Bluetooth keyboard;

[0032] The laser radar, onboard computer and power supply module are connected via the M1 route.

[0033] Furthermore, the six-key Bluetooth keyboard includes:

[0034] Button 1: used to generate a 3D point cloud map and save it in .pcd format;

[0035] Button 2: used to generate a three-dimensional probability occupation map and save it in .ot format;

[0036] Button 3: used to generate a 2D raster map and save it in .pgm+.yaml format;

[0037] Button 4: used to save the current map triggered by button 1;

[0038] Button 5: used to save the current map triggered by button 2;

[0039] Button 6: used to save the current map triggered by button 3.

[0040] The embodiments of the present invention have the following technical effects:

[0041] 1. Through the tightly coupled data acquisition mechanism of the LiDAR and inertial measurement unit, a deep fusion of raw point cloud and pose data is achieved, providing high-precision synchronous input for the subsequent mapping process. An incremental point cloud registration algorithm is used to directly process the raw point cloud, eliminating the reliance on manually designed features. This significantly improves the robustness of mapping in complex environments (such as low-texture or symmetrical scenes) and avoids positioning drift or map distortion caused by missing features.

[0042] 2. Through dynamic Bayesian probability updates using a probabilistic octree model, 3D point cloud data is converted into a 3D probabilistic occupancy map in real time. Its incremental update mechanism, based on logarithmic probability ratios, significantly reduces computational complexity, enabling the 3D environment modeling process to continuously adapt to dynamic scenarios. A 2D grid map generated by height projection and probabilistic fusion of the 3D probabilistic occupancy map preserves the integrity of the environmental structure through rational spatial dimension selection and probabilistic model conversion, effectively distinguishing between traversable and obstructed areas, providing a highly reliable underlying foundation for navigation decisions.

[0043] 3. A probability fusion function is used to fuse vertical information from 3D occupancy probabilities, and a time-attenuation factor is used to dynamically attenuate historical data. This further addresses the "ghosting" problem caused by residual dynamic obstacles in traditional methods, significantly enhancing the real-time performance and accuracy of maps. A height projection mechanism combined with inertial attitude compensation ensures that 2D maps maintain spatial consistency even in complex terrain, providing a critical guarantee for safe navigation of unmanned systems on slopes and bumpy roads. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a flowchart of a real-time multimodal map generation method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0047] Figure 1 This is a flow chart of a real-time multimodal map generation method provided by an embodiment of the present invention. Figure 1 , specifically including:

[0048] S1. Through the tight coupling of LiDAR and inertial measurement unit, real-time pose data and original point cloud data are obtained.

[0049] In some embodiments, a lidar (LiDAR) mounted on a device (such as a drone or robot) continuously scans its surroundings, generating a series of spatial position information representing points. This represents raw point cloud data. Simultaneously, an inertial measurement unit (IMU) attached to the device also operates, capturing the device's instantaneous motion state, such as acceleration, angular velocity, and rotational speed. This data is fused and computed to provide real-time information on the device's precise position and orientation, known as pose data. The key point here is that the LiDAR and IMU signals are tightly coupled and processed on the onboard computing platform using an extended Kalman filter. The IMU data provides an estimate of the LiDAR's motion state, compensating for distortion during LiDAR motion and ensuring geometric consistency between spatial positioning and the point cloud structure. The resulting raw point cloud and pose data are inherently a tightly linked set of input information, eliminating the "wait-and-see" latency issues that can arise with previous, step-by-step processing and laying the foundation for accurate subsequent modeling.

[0050] An extended Kalman filter (EKF) is used to fuse the IMU and LiDAR data to estimate the sensor's pose (position and orientation). The EKF is an extension of the Kalman filter for nonlinear systems. It improves the accuracy and robustness of the system by optimizing the state estimation between the LiDAR point cloud and the IMU data. The EKF estimates the current pose through a prediction and update step. The prediction step predicts the sensor's position and velocity based on the IMU data, and the update step optimizes the current pose based on the LiDAR point cloud data. The state update formula of the Kalman filter is:

[0051] ;

[0052] in, is the predicted state at the current k moment; is the state estimate at the previous moment, z k is the observation data at the current moment, is the prediction model, K k is the Kalman gain.

[0053] S2. Based on the original point cloud data and the real-time pose data, generate a three-dimensional point cloud map through an incremental point cloud registration algorithm.

[0054] In some embodiments, in S2, the incremental point cloud registration algorithm uses an incremental distance k-dimensional tree to perform dynamic point cloud management, and supports dynamic insertion, deletion, balancing and downsampling operations of point clouds.

[0055] Furthermore, the management operation of the incremental distance k-dimensional tree includes:

[0056] The point cloud is downsampled and filtered by constraining the local map range through cubic space segmentation.

[0057] Specifically, during the 3D point cloud map construction phase, the system uses an incremental processing architecture to achieve spatial registration based on the raw point cloud data collected by the LiDAR and the real-time pose information calculated by the inertial measurement unit. The core of the registration process is to dynamically manage the continuously input point cloud stream and construct a hierarchical spatial index structure through an incremental distance k-dimensional tree (ikd-Tree for short). This structure creates a root node space during the initialization phase. When new input point cloud data is continuously received, each spatial point is inserted into the corresponding child node based on the 3D coordinates. When a new point overlaps with an existing node position, the attribute value is updated; when a new point falls into a blank area, a branch path is dynamically generated. This index structure has a self-maintenance mechanism - when the subtree becomes depth-unbalanced due to high-frequency updates, the node rotation and reorganization operation is automatically triggered to maintain retrieval efficiency.

[0058] As an efficient multi-dimensional spatial index structure, the incremental distance k-dimensional tree shows good adaptability in point cloud management. Its basic principle is to divide the three-dimensional space in sequence according to the coordinate axis direction and organize the point set in a tree structure. Whenever a new point is added, the system automatically assigns it to the appropriate child node based on its spatial position to ensure the orderliness of the overall structure. At the same time, a self-balancing mechanism is set up within the tree structure. When the point density in a certain area changes or the tree depth is unbalanced, the system will automatically adjust the node distribution to maintain query efficiency. This dynamic maintenance capability is particularly important for real-time mapping because it allows the system to flexibly cope with the storage and computing pressure brought by large-scale point clouds without losing key information.

[0059] During the specific implementation process, a cube space segmentation strategy was introduced to further optimize the point cloud management process. This strategy significantly reduces the amount of invalid calculations by spatially restricting the local map range to avoid involving all data in each operation. Specifically, the system determines the area of ​​interest based on the current sensor posture and divides the area into cube units of fixed size. For example, the local map range is constrained by cube space segmentation, and the default side length can be 1000 meters; an incremental distance k-dimensional tree is independently constructed in each cube, and operations such as insertion, deletion, and downsampling are only performed on the point cloud within the area. This method not only improves the efficiency of point cloud processing, but also enhances the consistency and stability of the local map.

[0060] On this basis, the downsampling operation further improves the operating efficiency of the system. For example, the point cloud can be downsampled and filtered with a resolution of 0.5 meters. Since the point cloud collected by the lidar usually has a high density and contains a large amount of redundant information, this places high demands on computing resources and storage capabilities. To this end, the system sets sampling rules within each cube, retains the most representative points, and removes duplicate or similar points. This process not only ensures the integrity of the map features, but also effectively reduces the complexity of subsequent registration and modeling. Ultimately, through the above mechanism, the incremental point cloud registration algorithm can achieve high-quality, low-latency three-dimensional map construction under limited resource conditions, providing a stable and reliable data foundation for the subsequent probabilistic occupancy map generation.

[0061] S3. Input the three-dimensional point cloud map into a probabilistic octree model, and generate a three-dimensional probabilistic occupancy map through dynamic Bayesian probability updating.

[0062] The 3D point cloud map is imported into the probabilistic environment modeling process. At this stage, a hierarchical spatial representation structure based on an octree is used to map point cloud coordinates into discrete voxel units. Each voxel's spatial occupancy state is evaluated through a recursive probabilistic update mechanism: New observations are combined with historical probability information, and the voxel confidence is updated in real time according to a logarithmic domain probability model. The update process uses a differentiated weighting strategy to handle different observation results—positive confidence accumulation is assigned when the laser beam hits a voxel, negative confidence correction is applied when the voxel is penetrated, and the final state of the voxel is determined based on a preset confidence threshold. This outputs a 3D occupancy map with probabilistic semantics.

[0063] In some embodiments, in S3, the update formula of the probabilistic octree model is:

[0064] ;

[0065] Where l(n) is the log probability ratio of node n, and P(n) represents the occupancy probability of node n; is the likelihood function of the sensor model, indicating that z is observed at the node n statet Conditional probability (hit / miss probability); z t is the original point cloud scanned by the lidar at time t; represents the posterior log probability accumulated to time t; n represents the node in the octree.

[0066] In some embodiments, in the likelihood function of the sensor model:

[0067] The hit probability is set to 0.7, corresponding to the increment , then the update formula of the probabilistic octree model is: ;

[0068] The miss probability is set to 0.4, corresponding to the logarithmic increment Δ l miss ≈−0.405, then the update formula of the probabilistic octree model is: ;

[0069] The probabilistic octree model determines the occupancy state by using a truncation threshold: when the node occupancy probability P(n)>0.97, it is marked as occupied, and when P(n)<0.12, it is marked as idle.

[0070] Specifically, the original point cloud data is mapped to an octree structure, where each voxel node represents a spatial unit. The system generates the point cloud data z according to the scanning data z obtained by the laser radar at time t. t , combined with the current sensor model to calculate the probability that the node is occupied or idle. The likelihood function in the sensor model describes the probability of observing z under a certain real state n. t The probability of a hit or miss is set based on the probability of a hit or miss. When the laser beam penetrates a voxel and detects an obstacle, the probability of that node being occupied increases accordingly. Conversely, if the laser beam passes through but misses, the node is more likely to be an empty area. Through continuous iterative updates, the system gradually converges to a more accurate spatial distribution.

[0071] In order to improve computational efficiency, the logarithmic probability ratio is used as the update basis. In the initial state, the logarithmic probability ratio of each node is set to a neutral value, indicating that there is no prior judgment on the node state. As a new frame of data is input, the system uses the Bayesian formula to update the current posterior probability. For example, when a hit event occurs, the corresponding increment Δl hit is introduced, which increases the node's occupation tendency; in the case of a miss, Δl miss This reduces the probability of the node being occupied. This incremental update strategy avoids the numerical instability problem caused by directly manipulating the probability value, while supporting fast addition and subtraction operations, improving the overall processing speed.

[0072] To prevent excessive probability fluctuations, the system introduces a truncation threshold mechanism. When a node's occupancy probability exceeds a set upper limit (e.g., 0.97), it is considered to be in a stable occupied state; when the probability falls below a lower limit (e.g., 0.12), it is marked as idle. This mechanism effectively suppresses state mutations caused by single false detections or noise interference, making the map more robust. Furthermore, the probabilistic octree supports multi-resolution modeling, enabling low-resolution representations in distant areas while retaining high-precision details in key areas, thereby achieving a balance between storage efficiency and mapping quality.

[0073] For example, assuming that the initial occupancy probability of a certain voxel node is 0.5, the corresponding logarithmic probability ratio is 0. In a certain frame observation, the laser beam hits the node, and the system calculates Δl based on the preset hit probability of 0.7 hit And update the node's logarithmic probability ratio. If the node is continuously hit in the following frames, its probability value gradually increases until it reaches above 0.97 and is marked as occupied. On the contrary, if the node is only penetrated by the laser but not hit in several frames, the system calculates Δl based on the miss probability of 0.4 miss , gradually reducing its occupancy probability and finally marking it as idle.

[0074] S4. Performing height projection and probability fusion on the three-dimensional probability occupancy map to generate a two-dimensional grid map in real time.

[0075] In some embodiments, in S4, generating the two-dimensional grid map includes:

[0076] S41. Set the height range [min(z), max(z)] to filter 3D voxels. To generate a 2D grid map, you first need to filter out the ground (or other specified plane) area. By setting the z-axis height range, for example, 0.1m to 1m, and then projecting the 3D point cloud that meets the conditions, you can obtain a 2D grid map. S42. Based on the 3D voxels, calculate the 2D grid probability using the probability fusion formula.

[0077] The probability fusion formula is: ;

[0078] in, l 2D (x,y) represents the log-probability ratio of the two-dimensional grid; P 3D (x, y, z) represents the occupancy probability of a three-dimensional voxel, and z represents the height dimension index of the three-dimensional space.

[0079] The system dynamically adjusts the projection plane according to the attitude angle provided by the inertial measurement unit, so that the two-dimensional map generation is not affected by the tilt or motion state of the sensor. In this way, even if the data is collected in a non-ideal horizontal state, the final output map can still be guaranteed to have good geometric consistency and spatial alignment capabilities. On this basis, the system performs a probability fusion operation on the filtered three-dimensional voxels and uses the logarithmic domain fusion formula to calculate the comprehensive probability value of the corresponding two-dimensional position. This formula combines the probability information of multiple height layers and converts the occupancy probability P of different z layers in three-dimensional space into a probability value P of 1000. 3D(x,y,z) Mapped to a two-dimensional plane, and formed a unified two-dimensional logarithmic probability ratio l by logarithmic superposition 2D(x,y) , thereby effectively retaining the uncertainty characteristics of the original 3D map.

[0080] Furthermore, the probability fusion introduces a time decay factor ɑ to perform exponential smoothing on the historical probability. The specific formula is:

[0081] ;in, Represents the two-dimensional grid probability value at time t; represents the two-dimensional grid probability value at time t-1; P new Indicates the new probability value of the 3D map projection of the current frame.

[0082] This mechanism can effectively suppress the afterimages of dynamic obstacles and improve the real-time performance of the map.

[0083] S43: The two-dimensional grid probabilities are normalized using a Sigmoid function to output a two-dimensional grid map. The Sigmoid function is used to normalize the probabilities to the interval [0, 1], converting them into a standard two-dimensional grid map. This function exhibits nonlinear compression properties, effectively suppressing the influence of extreme values, resulting in an output image with good visual performance and numerical distribution.

[0084] To illustrate, consider an area with partially occupied voxels at multiple altitude levels, with the corresponding 2D grid's initial probability at an intermediate value. As multiple frames of data are continuously fed in, if the area remains stable and similar, its 2D probability gradually converges to a high value, indicating reliable occupation. If subsequent observations show that the area has cleared, the probability decreases, reflecting a changing environment.

[0085] The present invention also provides a real-time multimodal map generation system for executing the above-mentioned real-time multimodal map generation method. The system is a detachable hardware system, including: a laser radar, an onboard computer, a power supply module, an external display screen, and a six-button Bluetooth keyboard;

[0086] The laser radar is tightly coupled with the inertial measurement unit to acquire real-time pose data and original point cloud data;

[0087] The onboard computer is used to execute the tightly coupled positioning and mapping algorithm and the multimodal map generation process, and receive the data from the laser radar through the data interface;

[0088] The external display screen is connected to the onboard computer via nomachine and is used to observe the map generation status in real time;

[0089] The six-button Bluetooth keyboard is used to trigger multimodal map generation and saving operations through key commands;

[0090] The power supply module is used to provide DC power to the laser radar, the onboard computer and the six-button Bluetooth keyboard;

[0091] The laser radar, onboard computer and power supply module are connected via the M1 route.

[0092] Furthermore, the six-key Bluetooth keyboard includes:

[0093] Button 1: used to generate a 3D point cloud map and save it in .pcd format;

[0094] Button 2: used to generate a three-dimensional probability occupation map and save it in .ot format;

[0095] Button 3: used to generate a 2D raster map and save it in .pgm+.yaml format;

[0096] Button 4: used to save the current map triggered by button 1;

[0097] Button 5: used to save the current map triggered by button 2;

[0098] Button 6: used to save the current map triggered by button 3.

[0099] In the construction and operation of the real-time multimodal map generation system, the detachable hardware platform design provides high flexibility for system deployment and task execution. The system consists of a lidar, an onboard computer, a power supply module, and a six-button Bluetooth keyboard. These modules work efficiently together through standardized interfaces, making it suitable for applications such as mobile robots and drones that require high real-time environmental perception.

[0100] As the core sensor, the LiDAR is tightly coupled with the inertial measurement unit to ensure strict time synchronization between the collected raw point cloud data and the position and posture information. This design effectively improves mapping accuracy, especially in dynamic or complex motion states, and can more stably maintain spatial consistency. The LiDAR continuously scans the surrounding environment and transmits the raw point cloud and posture data to the onboard computer via the M1 line. As a dedicated communication link, the M1 line features low latency and high bandwidth, ensuring the real-time and integrity of the data stream.

[0101] The onboard computer is the control center of the entire system, responsible for executing the entire algorithmic process, including tightly coupled positioning, incremental point cloud registration, probabilistic octree modeling, and 2D grid map generation. It integrates a high-performance computing unit and an embedded operating system, supporting multi-threaded processing within the ROS (Robot Operating System) framework. After preprocessing, received data enters multiple stages, including 3D mapping, probability update, and height projection, ultimately outputting map data in multiple formats to meet the needs of different application levels.

[0102] An external display screen is connected to the computer via nomachine to observe the map generation status in real time. The external display screen can be a mobile phone or tablet.

[0103] To facilitate operation and interaction, the system introduces a six-button Bluetooth keyboard as an external command input device. Each button is assigned a clear function to enable quick triggering of map generation and saving. For example, pressing button 1 starts the construction process of the 3D point cloud map and stores the current frame in .pcd format; button 2 is used to generate and save a 3D occupancy map based on a probabilistic octree, with the output format being .ot; button 3 is responsible for generating a 2D raster map that meets navigation standards, and outputs it in the form of a .pgm image file and a .yaml configuration file. In addition, buttons 4, 5, and 6 correspond to the save confirmation operations of the three main buttons, respectively, providing a secondary confirmation mechanism to prevent data overwriting or loss due to accidental touches.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A real-time multimodal map generation method, characterized in that: The steps include: S1, obtain real-time pose data and original point cloud data by coupling the lidar with the inertial measurement unit; S2. Based on the original point cloud data and the real-time pose data, generating a three-dimensional point cloud map by an incremental point cloud registration algorithm; S3. Inputting the three-dimensional point cloud map into a probabilistic octree model and generating a three-dimensional probabilistic occupancy map through dynamic Bayesian probability updating; S4, performing height projection and probability fusion on the three-dimensional probability occupancy map to generate a two-dimensional grid map in real time; In S4, the generation of the two-dimensional grid map includes: S41. Set a height interval [min(z), max(z)] to filter three-dimensional voxels; S42. Calculate the two-dimensional grid probability based on the three-dimensional voxels using a probability fusion formula; The probability fusion formula is: ; in, l 2D (x,y) represents the log-probability ratio of the two-dimensional grid; P 3D (x, y, z) represents the occupancy probability of a three-dimensional voxel, and z represents the height dimension index of the three-dimensional space; S43, the two-dimensional grid probability is normalized by a Sigmoid function to output a two-dimensional grid map; The probability fusion introduces the time decay factor ɑ to perform exponential smoothing on the historical probability. The specific formula is: ;in, Represents the two-dimensional grid probability value at time t; represents the two-dimensional grid probability value at time t-1; P new Indicates the new probability value of the 3D map projection of the current frame.

2. A real-time multimodal map generation method according to claim 1, characterized in that: In S2, the incremental point cloud registration algorithm uses an incremental distance k-dimensional tree to perform dynamic point cloud management, and supports dynamic insertion, deletion, balancing and downsampling operations of point clouds.

3. A real-time multimodal map generation method according to claim 2, characterized in that: The management operations of the incremental distance k-dimensional tree include: The point cloud is downsampled and filtered by constraining the local map range through cubic space segmentation.

4. A real-time multimodal map generation method according to claim 1, characterized in that: In S3, the update formula of the probabilistic octree model is: ； Where l(n) is the logarithmic probability ratio of node n, P(n) represents the occupancy probability of node n; The likelihood function of the sensor model indicates that z is observed at the node n state t The conditional probability of z t is the original point cloud scanned by the lidar at time t; represents the posterior log probability accumulated to time t; n represents the node in the octree.

5. A real-time multimodal map generation method according to claim 4, characterized in that: In the likelihood function of the sensor model: The hit probability is set to 0.7, corresponding to the increment , then the update formula of the probabilistic octree model is: ; The miss probability is set to 0.4, corresponding to the logarithmic increment Δl miss ≈−0.405, then the update formula of the probabilistic octree model is: ; The probabilistic octree model determines the occupancy state by using a truncation threshold: when the node occupancy probability P(n)>0.97, it is marked as occupied, and when P(n)<0.12, it is marked as idle.

6. A real-time multimodal map generation system, configured to execute a real-time multimodal map generation method according to any one of claims 1 to 5, characterized in that: The system is a detachable hardware system, including: laser radar, onboard computer, power supply module, external display screen and six-button Bluetooth keyboard; The laser radar is tightly coupled with the inertial measurement unit to acquire real-time pose data and original point cloud data; The onboard computer is used to execute the tightly coupled positioning and mapping algorithm and the multimodal map generation process, and receive the data from the laser radar through the data interface; The external display screen is connected to the onboard computer via nomachine and is used to observe the map generation status in real time; The six-button Bluetooth keyboard is used to trigger multimodal map generation and saving operations through key commands; The power supply module is used to provide DC power to the laser radar, the onboard computer and the six-button Bluetooth keyboard; The laser radar, onboard computer and power supply module are connected via the M1 route.

7. A real-time multimodal map generation system according to claim 6, characterized in that: The six-key Bluetooth keyboard includes: Button 1: used to generate a 3D point cloud map and save it in .pcd format; Button 2: used to generate a three-dimensional probability occupation map and save it in .ot format; Button 3: used to generate a 2D raster map and save it in .pgm+.yaml format; Button 4: used to save the current map triggered by button 1; Button 5: used to save the current map triggered by button 2; Button 6: used to save the current map triggered by button 3.

Images