Semantic map construction method based on deep learning and laser SLAM

Abstract

According to the semantic map construction method based on deep learning and laser SLAM, a semantic segmentation method based on deep learning and a laser SLAM method are fused, a three-dimensional map with semantic tags is constructed in real time, and richer information support is provided for applications such as robot navigation and automatic driving. The invention provides a set of complete solution for challenges such as complexity and dynamics of an outdoor environment, large-scale and high-precision requirements, illumination and weather changes, semantic information requirements and the like.

Description

Technical Field

[0001] The present invention relates to the technical fields of robot navigation, autonomous driving, augmented reality, and environmental perception, and particularly relates to a method for constructing a semantic map based on deep learning and laser SLAM. Background Art

[0002] Environmental perception and map construction are one of the core technologies of intelligent robot technology and autonomous driving technology. The laser SLAM (Simultaneous Localization and Mapping) technology can construct a three-dimensional geometric map of the environment, but lacks the semantic understanding of the objects in the environment and is difficult to meet the navigation and decision-making requirements in complex scenarios. In recent years, deep learning technology has made remarkable progress in semantic segmentation of image and point cloud data. Incorporating semantic information into map construction can effectively improve the environmental perception and autonomous navigation capabilities of robots. However, how to effectively combine deep learning with laser SLAM technology to construct a three-dimensional map with semantic labels in real time is still a technical problem.

[0003] Particularly in outdoor environments, map construction faces the following challenges:

[0004] 1. Semantic distortion caused by dynamic object interference: Moving targets cause sudden changes in point cloud features, resulting in semantic label drift in dynamic regions;

[0005] 2. Sensor attenuation under meteorological interference: Heavy rainfall causes a decrease in the density of laser point clouds, and thick fog causes attenuation of echo signals;

[0006] 3. Contradiction between algorithm and efficiency imbalance: The semantic segmentation network requires high computing power and long computing time, which does not meet the real-time requirements.

[0007] Therefore, there is an urgent need for a method that can construct a high-precision semantic map in real time to solve the above problems. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for constructing a semantic map based on deep learning and laser SLAM. By integrating a semantic segmentation method based on deep learning with a laser SLAM method, a three-dimensional map with semantic labels is constructed in real time, providing richer information support for applications such as robot navigation and autonomous driving. In view of the challenges of the complexity and dynamics of outdoor environments, large-scale and high-precision requirements, illumination and weather changes, and semantic information requirements, the present invention proposes a complete solution.

[0009] The technical solution of the present invention includes the following steps: 1 Point cloud data collection and acquisition

[0010] Collect point cloud data directly through sensors or read it from publicly available datasets, where the point cloud data contains three-dimensional coordinate information.

[0011] The sensor is a LiDAR (Light Detection and Ranging) capable of collecting three-dimensional point cloud data, such as Velodyne, Ouster, Riegl, Hesai, RoboSense, for directly collecting the three-dimensional point cloud data of the environment; or the publicly available datasets, such as KITTI, Semantic-KITTI, NuScenes, which contain point cloud data with three-dimensional coordinate information.

[0012] 2 Real-time Point Cloud Semantic Segmentation

[0013] Input the point cloud data into a pre-trained semantic segmentation network for semantic segmentation to generate semantic labels for each point, where the semantic labels include color information for distinguishing objects of different semantic classes in the point cloud data.

[0014] The training process of the pre-trained semantic segmentation network includes:

[0015] Obtain a training dataset containing point cloud data, such as the Semantic-KITTI dataset, and create semantic labels for it;

[0016] Perform random transformations on the point cloud data, including rotation, deletion, scaling, and adding noise, to enhance the diversity and robustness of the data;

[0017] Input the randomly transformed data into the semantic segmentation network for training, calculate the error of semantic segmentation using a loss function, and update the network parameters using an optimizer through the backpropagation algorithm;

[0018] Repeat the training process until the model converges to obtain a pre-trained semantic segmentation network.

[0019] The semantic segmentation process includes:

[0020] Use the pre-trained semantic segmentation network to process the point cloud data and obtain local and global features of the point cloud through multi-level feature extraction;

[0021] Integrate features at different levels through a context information fusion module (such as an attention mechanism or multi-scale feature fusion) to enhance the representational ability of the features;

[0022] Input the fused features into a classifier to perform semantic classification on each point and generate semantic labels for each point, where the semantic labels include color information.

[0023] 3 Semantic Map Construction

[0024] Based on laser SLAM, the extracted semantic tags are fused with the point cloud data, and semantic information is assigned to the point cloud data through color rendering to construct a three-dimensional map with semantic tags in real time.

[0025] The specific process is as follows:

[0026] Receive three-dimensional point cloud data in real time and send it in parallel to a pre-trained semantic segmentation network. The semantic segmentation network performs semantic segmentation on the point cloud data to generate semantic tags (color information) and returns the results;

[0027] Combine the semantic tags returned by the semantic segmentation network with the original point cloud data. Specifically, during the mapping process, while reading the point cloud data, the corresponding semantic information is read, and the semantic information is assigned to the point cloud data through color rendering to achieve color rendering of the point cloud;

[0028] Subsequently, use the registration algorithm to input the current frame with semantic tags into the existing semantic map, and update and expand the map in real time according to the newly received data, so as to continuously construct a three-dimensional map with semantic information;

[0029] During the calculation process of laser SLAM, loop detection is performed through a semantic-geometry dual verification mechanism, and the map is globally optimized. For the processing of dynamic objects, dynamic filtering or probability modeling methods are used;

[0030] The above steps are executed in a loop when each frame of point cloud data arrives, ensuring the continuous extraction, fusion of semantic information, and real-time update and optimization of the map, so as to achieve efficient and accurate semantic map construction.

[0031] 4 Semantic-Enhanced Loop Detection and Global Optimization

[0032] Extract the global semantic features and local geometric features of the point cloud data through the semantic segmentation network, where the global semantic features are the semantic category distribution of static objects in the scene and their topological relationships, and the local geometric features are key point descriptors and surface curvature distributions;

[0033] Calculate the semantic overlap rate between candidate closed-loop frames based on the semantic features, and screen candidate frames that meet the semantic consistency conditions; perform geometric feature matching on the candidate frames, calculate the pose transformation error through the point cloud registration algorithm, and if the semantic overlap rate is higher than the first threshold and the geometric error is lower than the second threshold, it is determined that the closed loop is valid;

[0034] According to the closed-loop verification results, use the graph optimization algorithm to globally correct the historical poses and map points to improve the long-term geometric accuracy and semantic consistency of the map.

[0035] 5 Dynamic Object Detection and Map Interference Suppression

[0036] Analyze the motion trajectories of objects in consecutive point cloud frames using a semantic segmentation network. Identify dynamic objects by comparing the coordinate offsets of objects with the same semantic label, and exclude false detected static interferences by combining semantic categories.

[0037] Perform dynamic mask filtering or probability modeling on the detected dynamic objects: Dynamic mask filtering eliminates instantaneous interferences by removing dynamic point cloud data, and probability modeling estimates the spatio-temporal confidence of the dynamic region based on the Bayesian filtering algorithm and marks its motion trend and appearance probability on the map.

[0038] During the map building process, the dynamic mask filtering scheme updates the static map in real time, and the probability modeling scheme fuses historical observation data through a sliding window mechanism to achieve the interpretable expression and long-term tracking of dynamic semantic information.

[0039] 6 Visualization and Interaction

[0040] Real-time display a 3D map with semantic labels through visualization software, and the software supports users to view and operate interactively to achieve a comprehensive browsing and analysis of the semantic map.

[0041] The visualization software supports the following multiple interactive operations:

[0042] Zoom in, rotate, and pan the map to observe the map from different perspectives.

[0043] Query the semantic information of a specific area, including semantic categories and their corresponding color information.

[0044] Dynamically adjust the accuracy and resolution of the map display to adapt to different browsing needs.

[0045] Select and read specific point cloud data for detailed analysis of a local area.

[0046] Support playing back the construction process of the semantic map along the time axis to show the changes of the map over time.

[0047] Support exporting the semantic map in multiple formats (such as point cloud files, image files, or 3D model files). Description of the Drawings

[0048] Appendix Figure 1 Is the schematic diagram of the laser SLAM semantic mapping method based on the deep learning network of the present invention.

[0049] Appendix Figure 2 Is the schematic diagram of the fusion of laser SLAM mapping and semantic segmentation results. Detailed Embodiments

[0050] 1 System Hardware Architecture and Data Acquisition

[0051] 1.1 Sensor Selection and Data Generation Mechanism

[0052] The core sensor of this system selects the Helios 32 lidar from RoboSense, and its technical parameters are strictly matched to the requirements of dynamic environment mapping. The 32-beam design of this lidar forms a 70° field of view in the vertical direction (-55° to +15°), and performs a 360° scan in the horizontal direction with a resolution of 0.1° to 0.4°, with a point cloud output of 1,200,000 points per second. The basis for hardware selection is as follows:

[0053] Horizontal Resolution Optimization: At a distance of 10 meters, the minimum horizontal angular resolution of 0.1° corresponds to an interval of 17.5 millimeters, achieving centimeter-level feature matching accuracy and effectively improving the reliability of SLAM mapping loop detection.

[0054] Vertical Field of View Design: The extended vertical coverage range (-35° to +15°) can simultaneously detect ground obstacles (the lowest -35° corresponds to a ground height of 5 cm) and objects up to three stories high (an object 12 meters high at 50 meters away at a 15° elevation angle), completely covering special targets such as construction vehicles (height 3.5 meters) and traffic lights (height 5.5 meters) on urban roads.

[0055] Scanning Frequency 20Hz: By enhancing the point cloud density through the dual-echo mode and cooperating with the motion compensation algorithm, the overlap rate of adjacent frame point clouds can still be maintained above 35% at a vehicle speed of 60 km / h, meeting the requirements of real-time incremental mapping in complex scenarios.

[0056] Anti-interference Ability: It meets the Class 1 eye-safe standard and has a multi-lidar anti-crosstalk function built-in, adapting to stable operation in strong reflection environments such as tunnels.

[0057] 1.2 Point Cloud Data Generation Mechanism

[0058] Every time the lidar completes a scanning cycle (50 ms), it outputs a frame of point cloud data, and the data structure is defined as:

[0059] P i =(x i ,y i ,z i ,r i ,t i ) (i = 1, 2,..., N)

[0060] Where:

[0061] i is the point cloud number;

[0062] (x i ,y i ,z i) is the three-dimensional coordinate in the Cartesian coordinate system (unit: meter), and the accuracy is determined by the radar calibration parameters, with a typical value of ±2 cm;

[0063] r i is the reflection intensity (normalized value 0 - 1), reflecting the material characteristics of the target surface (such as the reflectivity of metal is 0.8 - 1.0, and that of asphalt pavement is 0.2 - 0.3);

[0064] t i is the timestamp accurate to 0.1 ms, which is synchronized with the IMU data through the hardware trigger signal to ensure that the time alignment error is less than 0.5 ms.

[0065] 1.3 Multi-source data fusion and preprocessing

[0066] When using public datasets (such as Semantic-KITTI), the following preprocessing process needs to be executed:

[0067] (1) Coordinate system alignment

[0068] Convert the local coordinate system of the lidar (Right-Up-Front, RUF) to the global coordinate system (East-North-Up, ENU), and the transformation matrix is:

[0069]

[0070] Rotation matrix calculation: Based on the initial attitude angles (roll pitch θ, yaw ψ) of the IMU (Inertial Measurement Unit), generate the rotation matrix R = R z (ψ)·R y (θ)·R x (φ)

[0071] Translation vector calculation: Convert the GPS (Global Positioning System) longitude and latitude coordinates (longitude λ, latitude elevation h) to the global ENU (east-west coordinate E, northward coordinate N, upward coordinate U) coordinates through UTM (Universal Transverse Mercator) as the translation vector Initial reference point The projection is (E0, N0, U0).

[0072] (2) Point cloud denoising and motion distortion correction

[0073] Statistical Outlier Removal (SOR): Calculate the coordinate mean of points within the 50-neighborhood of each point and the standard deviation Remove outliers that satisfy || p i - μ i || > z · ||σ i ||, effectively eliminating environmental noises such as rain and fog (reflection intensity 0.1 - 0.3).

[0074] Motion distortion compensation: Using the angular velocity ω and linear acceleration a of the IMU, through the displacement correction amount and the rotation matrix R(t) ≈ I + ω^t, correct the point cloud coordinate p corrected = R(t)^-1(p raw - Δp), eliminating geometric distortions caused by the movement of the carrier.

[0075] Among them, k is the number of neighborhood points, with a default value of 50; z is the standard deviation threshold coefficient, with a default value of 2.0; t is the relative time within the point cloud frame (unit: s); v0 is the initial velocity vector (unit: m / s).

[0076] 2 Deep learning network training and optimization

[0077] 2.1 Physical meaning and implementation of data augmentation strategies

[0078] To improve the robustness of the model to interference in the actual scenario, data augmentation needs to simulate noises, occlusions, and perspective changes in the real environment:

[0079] (1) Rotation perturbation around the Z-axis (±45°)

[0080] Simulate the observation perspective change when the vehicle turns, enhancing the model's feature extraction ability that is insensitive to directions. The rotation angle θ follows a uniform distribution, where α is the maximum perturbation angle (default value α = 15°):

[0081] θ ∼ U(-α, α)

[0082] For the original point p = [x, y, z] T The rotated point cloud coordinates (x′, y′, z′) are calculated as:

[0083]

[0084] (2) Poisson point cloud deletion

[0085] Simulate the signal attenuation effect of long-distance LiDAR (Light Detection and Ranging), and the deletion intensity is controlled by the parameter λ = 0.02. For a point cloud containing N points in one frame, the number of remaining points follows a Poisson distribution:

[0086] N retain ~Poisson(λ, N)

[0087] For example, when N = 10,000, the expected value of the retained points is 0.02 × 10,000 = 200 points.

[0088] (3) Gaussian noise injection

[0089] To simulate sensor measurement errors and environmental disturbances (such as air turbulence), independent and identically distributed Gaussian noise is added to each point coordinate:

[0090] ∈ x , ∈ y , ∈ z ~N(0, σ 2 )

[0091]

[0092] The noise standard deviation σ is set to be slightly higher than the inherent error of the sensor. For example, σ = 0.03 m (the inherent error of the sensor is 0.02 m) to cover environmental disturbances.

[0093] 2.2 Hierarchical design of the network architecture

[0094] The deep learning network used in this example is PointNet++. The PointNet++ network includes four-level feature extraction modules and a multi-scale attention mechanism. The specific structure is as follows:

[0095] (1) Set Abstraction (SA) module

[0096] The first-level SA: Sample 512 points, search radius 0.1 m, and use a 3-layer MLP (Multi-Layer Perceptron) (32-64-128 dimensions) to extract local geometric features (such as edges, curvature).

[0097] The second-level SA: Sample 128 points, search radius 0.2 m, and the MLP dimensions are increased to 64-128-256 dimensions to capture object contour features (such as vehicle boundaries).

[0098] The third-level SA: Sample 32 points, search radius 0.4 m, introduce a spatial attention mechanism to enhance the semantic context awareness ability.

[0099] The fourth-level SA: Globally sample 1 point to generate a 256-dimensional scene-level feature vector.

[0100] (2) Spatial attention mechanism

[0101] After the third-level SA module, multi-scale features are fused through multi-head attention:

[0102]

[0103] MultiHead(Q, K, V) = Concat(head1, ..., head h )W O

[0104] where Q, K, and V are the query, key, and value matrices respectively, d k is the feature dimension (default value 64), and h is the number of attention heads (default value 4). After concatenating the output of multi-head attention with the original features, the dimension is reduced through a 1×1 convolution, and finally the fused semantic feature F fused = Conv 1×1 (Concat(F orig , F attn )) is generated.

[0105] 2.3 Loss Function Design and Training Optimization

[0106] (1) Class-Balanced Cross-Entropy Loss

[0107] To address the problem of unbalanced semantic class distribution, a frequency-weighted factor is introduced:

[0108]

[0109] where f c is the frequency of occurrence of class c in the training set. For example, the frequency of the road class f road = 0.35, and its weight w road = 1 / log(1.55) ≈ 2.3; the frequency of pedestrians f ped = 0.02, and the weight w ped = 1 / log(1.22) ≈ 4.1.

[0110] (2) Training Parameter Configuration

[0111] Optimizer: AdamW (Adam with Weight Decay, Adam optimizer with weight decay), initial learning rate 3×10 -4 , weight decay 0.01, to prevent overfitting.

[0112] Learning rate scheduling: Cosine annealing strategy, cycle 50 epochs, minimum learning rate 1×10 -6 .

[0113] Regularization: Dropout rate is 0.3 and is applied to the fully connected layer;

[0114] Label smoothing: The coefficient is 0.1 to improve the generalization of the model.

[0115] 3 Semantic-Geometric Fusion Map Construction

[0116] 3.1 Real-Time Semantic Label Fusion

[0117] (1) Parallel Processing Pipeline

[0118] Thread 1 (20Hz): Receive and preprocess the raw point cloud, including timestamp alignment, coordinate system transformation, and outlier removal.

[0119] Thread 2 (15Hz): Semantic segmentation inference, using the TensorRT (NVIDIA's deep learning inference optimization library) acceleration engine to load the pre-trained model, and the single-frame inference time ≤ 65ms.

[0120] Thread 3 (10Hz): Semantic coloring and map update, adopting a double-buffering mechanism to avoid data competition and ensure real-time performance.

[0121] (2) Semantic Coloring Rules

[0122] Classify the point cloud into 20 predefined semantic classes according to the segmentation results, and assign a unique RGB color code to each class. For example:

[0123] Dynamic vehicle: Red (255, 0, 0)

[0124] Vegetation: Green (0, 255, 0)

[0125] Building: Gray (128, 128, 128)

[0126] The color information is stored in a 32-bit integer. The lower 24 bits represent the RGB value, and the upper 8 bits are reserved for extended semantic attributes (such as confidence and motion state).

[0127] 3.2 Semantic-Enhanced ICP Registration Algorithm

[0128] The traditional ICP algorithm only optimizes the geometric distance. This method introduces a semantic consistency constraint term, and the objective function is:

[0129]

[0130] Where:

[0131] T: Stiffness transformation matrix (rotation + translation);

[0132] p i ,q i : Corresponding points in the source point cloud and the target point cloud

[0133] δ(s i ,s j ):Semantic consistency constraint term, defined as

[0134] λ: Semantic item weight (determined by grid search, default λ = 10.0).

[0135] Steps of algorithm implementation:

[0136] (1) Coarse semantic consistency matching: Voxel downsampling (0.2 m) is performed on the source point cloud and the target point cloud, and only point pairs with consistent semantic labels are retained.

[0137] (2) Geometric fine registration: On the basis of coarse matching, the traditional ICP algorithm is used to optimize the rigid body transformation matrix T T.

[0138] (3) Convergence condition: Iterate 50 times or the change rate of registration error < 1e -5 .

[0139] 3.3 Map update and maintenance strategy

[0140] (1) Static map update

[0141] Voxel filtering: The point cloud is divided into voxels of 0.1 m, and the weighted average of the point coordinates within each voxel is taken:

[0142]

[0143] where: ω i is the time decay weight, t i is the difference between the point cloud timestamp and the current time, τ is the decay coefficient (default value 30 s), and M is the number of points within the voxel.

[0144] Dynamic point filtering: Dynamically detect dynamic objects and generate a binary mask, and clean the historical dynamic point cloud data every 3 seconds to avoid memory overflow.

[0145] (2) Glo Figure 1 bal consistency check

[0146] Perform a global semantic consistency check every 5 seconds. If the semantic category distribution in a certain area deviates from the historical data by more than the threshold (e.g., a large number of vegetation labels appear in the building area), trigger a local re-optimization process.

[0147] 4 Closed-loop detection optimization

[0148] 4.1 Semantic-geometric double verification mechanism

[0149] (1) Semantic consistency screening

[0150] Semantic histogram construction: Map 20 types of semantic tags to a histogram H current (Current frame) and H hist (Historical key frame), and the frequency of each category is normalized to a probability distribution.

[0151] Overlap rate calculation

[0152]

[0153] The default threshold is set to η overlap = 0.65. Experiments show that 85% of the false matches can be excluded (verified based on the KITTI dataset). If Overlap > η overlap , it is considered that the semantic environments of the two frames are similar.

[0154] (2) Geometric consistency verification

[0155] Multi-resolution ICP (Iterative Closest Point) registration:

[0156] Coarse registration: Downsample the point cloud to a 0.2-meter voxel, iterate 20 times, and the convergence threshold is 0.1 meter.

[0157] Fine registration: Point cloud at the original resolution, iterate 50 times, and the convergence threshold is 0.05 meter.

[0158] Geometric error calculation

[0159]

[0160] If E geo < η geo , it is determined that the loop closure is established.

[0161] 4.2 Global pose graph optimization

[0162] (1) Pose graph construction

[0163] Vertices: The pose T of each key frame i ∈ SE(3) (Special Euclidean group, including rotation and translation).

[0164] Odometer constraint edges: The relative pose transformation between adjacent frames Covariance matrix ∑odom.

[0165] Loop closure constraint edges: The relative pose transformation T between loop-matched frames i，k , covariance matrix ∑loop

[0166] (2) Optimization objective function

[0167]

[0168] Wherein:

[0169] ρ(·) is the Huber robust kernel function, defined as follows (threshold δ = 0.1), suppressing the influence of abnormal closed-loop constraints.

[0170] ∑ -1 is the inverse of the covariance matrix, reflecting the confidence of the constraint.

[0171] (3) Optimizer configuration:

[0172] Use the Levenberg-Marquardt (LM) solver of g2o (General Graph Optimization library), with a maximum number of iterations of 100 times, and the pose error is reduced by an average of 72%.

[0173] 5 Dynamic object detection and spatio-temporal modeling

[0174] 5.1 Instantaneous filtering

[0175] (1) Dynamic mask generation

[0176] For the detected dynamic objects, generate a spherical mask area r = 0.5m.

[0177] (2) Morphological dilation

[0178] Use a circular structuring element K (radius r dilate = 0.2m) to dilate the mask To avoid the remaining edge points, where represents the dilation operation, used to eliminate the serrations at the mask boundary.

[0179] 5.2 Probability modeling

[0180] (1) Spatio-temporal occupancy probability model:

[0181] The dynamic region probability P(p, t) is defined as:

[0182]

[0183] Wherein:

[0184] N(p, t): The number of observations of dynamic objects at position p within the time window [t - τ, t];

[0185] N total : The total observation number normalization factor;

[0186] λ: The time decay coefficient (default λ = 0.1s -1 ;

[0187] tlsat : The last observation time.

[0188] (2) Dynamic area annotation:

[0189] Every T update = 30s to update the probability map. If P(p, t)>η p (Default η p = 0.7) is marked as a high - probability dynamic area (such as a bus lane, crosswalk).

[0190] 6 Interactive visualization system

[0191] 6.1 Multimodal interaction function design

[0192] (1) Viewpoint control

[0193] 6 - degree - of - freedom manipulation: Supports translation, rotation, and zoom operations. Smooth viewpoint switching is achieved using quaternion interpolation:

[0194]

[0195] Among them:

[0196] q start , q end : The starting and ending quaternions;

[0197] θ: The angle between the two quaternions, θ = arccos(q start , q end );

[0198] t: The interpolation coefficient, t ∈ [0, 1].

[0199] Viewpoint following mode: Automatically adjusts the viewpoint to follow the movement of dynamic objects, with a delay ≤ 100ms.

[0200] (2) Semantic information query

[0201] Point - level query: Click on any point cloud to display the semantic category, confidence level, and historical trajectory.

[0202] Region statistics: Select a region to output the number of point clouds of each category, spatial distribution, and proportion.

[0203] 6.2 Map export and compatibility

[0204] (1) Point cloud format (PLY)

[0205] Contains XYZ coordinates, RGB colors, and semantic labels

[0206] (2) 3D model (OBJ + MTL)

[0207] The OBJ file defines the mesh vertices and faces.

[0208] The MTL file maps semantic colors to materials (e.g., material_Vehicle_Red corresponds to RGB(255, 0, 0)).

Claims

1. A semantic map construction method based on deep learning and laser SLAM, characterized in that: include: Directly collect point cloud data through sensors or read point cloud data from public data sets, wherein the point cloud data includes three-dimensional coordinate information; Inputting the point cloud data into a pre-trained semantic segmentation network for semantic segmentation to generate a semantic label for each point, wherein the semantic label includes color information for distinguishing objects of different semantic categories in the point cloud data; Based on laser SLAM, the extracted semantic labels are fused with point cloud data, semantic information is given to point cloud data through color rendering, and a 3D map with semantic labels is constructed in real time. In the process of building a three-dimensional map with semantic labels, the semantic segmentation network is used to optimize the loop detection of SLAM, and the closed-loop hypothesis is generated and double-verified by combining semantic consistency analysis and geometric feature matching to improve the robustness of closed-loop detection and reduce the accumulated error of long-term mapping; At the same time, the semantic segmentation network is used to detect and model dynamic objects, identify dynamic objects based on semantic labels and temporal point cloud sequences, and selectively filter or annotate their spatiotemporal variation characteristics, thereby reducing the impact of dynamic interference on map consistency; The three-dimensional map with semantic labels is displayed in real time through visualization software, and the software supports user interactive viewing and operation to achieve comprehensive browsing and analysis of the semantic map.

2. A semantic map construction method based on deep learning and laser SLAM as claimed in claim 1, characterized in that: The sensor is a laser radar (LiDAR) capable of collecting three-dimensional point cloud data, such as Velodyne, Ouster, Riegl, Hesai, RoboSense, which is used to directly collect three-dimensional point cloud data of the environment; or the public data set, such as KITTI, Semantic-KITTI, NuScenes, which contains point cloud data with three-dimensional coordinate information.

3. A semantic map construction method based on deep learning and laser SLAM as claimed in claim 1, characterized in that: The training process of the pre-trained semantic segmentation network includes: Get a training dataset containing point cloud data, such as the Semantic-KITTI dataset, and create semantic labels for it; Randomly transform the point cloud data, including rotation, deletion, scaling, and adding noise, to enhance the diversity and robustness of the data; The randomly transformed data is input into the semantic segmentation network for training, the error of semantic segmentation is calculated using the loss function, and the network parameters are updated using the optimizer through the back propagation algorithm; The training process is repeated until the model converges to obtain a pre-trained semantic segmentation network.

4. A semantic map construction method based on deep learning and laser SLAM as claimed in claim 1, characterized in that: The semantic segmentation process includes: Use the pre-trained semantic segmentation network to process the point cloud data and obtain the local and global features of the point cloud through multi-level feature extraction; Integrate features at different levels through context information fusion modules (such as attention mechanism or multi-scale feature fusion) to enhance the representation ability of features; The fused features are input into the classifier, semantic classification is performed on each point, and a semantic label of each point is generated, wherein the semantic label includes color information.

5. A semantic map construction method based on deep learning and laser SLAM as claimed in claim 1, characterized in that: Based on laser SLAM, the extracted semantic labels are fused with point cloud data, and semantic information is given to point cloud data through color rendering, so as to construct a three-dimensional map with semantic labels. The specific process is as follows: Receive 3D point cloud data in real time and send it to the pre-trained semantic segmentation network in parallel. The semantic segmentation network performs semantic segmentation on the point cloud data, generates semantic labels (color information), and returns the results; The semantic labels returned by the semantic segmentation network are combined with the original point cloud data. Specifically, in the process of building a map, the corresponding semantic information is read while reading the point cloud data, and the semantic information is given to the point cloud data through color rendering to achieve color rendering of the point cloud. Subsequently, the current frame with semantic labels is input into the existing semantic map using a registration algorithm, and the map is updated and expanded in real time based on the newly received data, thereby continuously building a three-dimensional map with semantic information; In the laser SLAM calculation process, loop detection is performed through the semantic-geometric dual verification mechanism described in claim 7, and the map is globally optimized. The dynamic filtering or probabilistic modeling method described in claim 8 is used to process dynamic objects; The above steps are executed cyclically when each frame of point cloud data arrives, ensuring that the extraction and fusion of semantic information and the real-time updating and optimization of the map are continuously carried out, thereby achieving efficient and accurate semantic map construction.

6. A semantic map construction method based on deep learning and laser SLAM as claimed in claim 1, characterized in that: The visualization software supports the following interactive operations: 1) Zoom, rotate, and pan the map to view it from different perspectives; 2) Query the semantic information of a specific area, including the semantic category and its corresponding color information; 3) Dynamically adjust the accuracy and resolution of map display to meet different browsing needs; 4) Select and read specific point cloud data for detailed analysis of local areas; 5) Support playback of the semantic map construction process according to the timeline to show how the map changes over time; 6) Supports exporting semantic maps into multiple formats (such as point cloud files, image files or 3D model files).

7. The method according to claim 1, characterized in that: The optimization of the semantic segmentation network for SLAM loop detection includes the following steps: The global semantic features and local geometric features of the point cloud data are extracted through a semantic segmentation network, wherein the global semantic features are the semantic category distribution and topological relationship of static objects in the scene, and the local geometric features are the key point descriptors and the surface curvature distribution; Based on the semantic features, the semantic overlap rate between the candidate closed-loop frames is calculated, and the candidate frames that meet the semantic consistency condition are screened; geometric feature matching is performed on the candidate frames, and the pose transformation error is calculated by a point cloud registration algorithm, and if the semantic overlap rate is higher than a first threshold and the geometric error is lower than a second threshold, the closed loop is determined to be valid; According to the closed-loop verification results, a graph optimization algorithm is used to globally correct historical poses and map points to improve the long-term geometric accuracy and semantic consistency of the map.

8. The method according to claim 1, characterized in that: The detection and modeling of dynamic objects by the neural network includes the following steps: The semantic segmentation network is used to analyze the motion trajectory of objects in continuous point cloud frames, and dynamic objects are identified by comparing the coordinate offsets of objects with the same semantic label, and static interference caused by false detection is eliminated by combining semantic categories. Perform dynamic mask filtering or probabilistic modeling on detected dynamic objects: Dynamic mask filtering eliminates instantaneous interference by removing dynamic point cloud data, while probabilistic modeling estimates the spatiotemporal confidence of dynamic areas based on the Bayesian filtering algorithm and marks their movement trends and occurrence probabilities on the map; During the mapping process, the dynamic mask filtering scheme updates the static map in real time, and the probabilistic modeling scheme fuses historical observation data through a sliding window mechanism to achieve interpretable expression and long-term tracking of dynamic semantic information.

9. The method according to claim 1 or 7, characterized in that: The pre-trained deep learning network is a three-dimensional segmentation network with point cloud sparsity processing capability and multi-scale feature fusion mechanism, such as PointNet++, RandLA-Net, KPConv network architecture; The neural network must meet the following requirements: 1) Support single forward reasoning of large-scale point cloud input, and the processing delay is less than 1.5 times the sensor data acquisition frame rate; 2) It has a hierarchical downsampling and feature propagation structure that can extract local geometric details and global semantic context of point clouds; 3) Integrate a timing analysis module to capture the motion trajectory characteristics of dynamic objects through 3D convolution or recurrent neural network (RNN); 4) The output layer contains open set recognition capabilities, which assigns specific semantic labels to point clouds of unknown categories to avoid misclassification interference.

10. The method according to claim 1 or 5, characterized in that: The laser SLAM algorithm is an optimized framework that supports semantic information fusion and highly robust closed-loop detection, such as LOAM, LIO-SAM, and Cartographer algorithms; The laser SLAM needs to meet the following requirements: 1) Provide a tightly coupled interface between the original point cloud and the semantic label, and realize the synchronous transmission of semantic and geometric data through color channels or additional data fields; 2) Integrate the probabilistic map representation method, using the octree or TSDF model to be compatible with the probability update of semantic labels; 3) It has a multi-hypothesis closed-loop detection mechanism, which allows the simultaneous processing of geometric feature matching and semantic consistency verification tasks; 4) Support dynamic object mask input to eliminate the interference of dynamic area point cloud on pose solution in the pose estimation stage.

Images