Lidar slam method based on multi-modal structural semantic features

Abstract

The application discloses a laser radar SLAM method based on a multi-modal structure semantic feature, and comprises the following steps: acquiring laser radar point cloud data; performing cylindrical point cloud segmentation on the laser radar point cloud data based on an ERFNet method; performing ground point cloud segmentation based on a BEV projection view method; performing planar point cloud segmentation based on a region growing method; fitting the cylindrical point cloud into a cylindrical model, and fitting the ground point cloud and the planar point cloud into planar models respectively; acquiring a matching relationship between different cylindrical models, and acquiring a matching relationship between different planar models, and constructing a structure semantic map; storing the structure semantic map based on a voxel plane; and updating the structure semantic map based on a model matching and map incremental updating algorithm. The application effectively improves the laser radar SLAM positioning precision, and proposes an intuitive multi-modal structure semantic map, so that the construction and real-time updating of three kinds of structure semantic model maps can be realized.

Description

Technical Field

[0001] This invention belongs to the field of ground feature extraction and matching technology, and in particular relates to a lidar SLAM method based on multimodal structural semantic features. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is one of the core technologies for robot navigation and control, and also one of the mainstream research methods in current robot localization technology. With the continuous development of autonomous driving technology, LiDAR SLAM technology is being used more and more widely in unmanned vehicles.

[0003] Currently, LiDAR SLAM feature matching is mainly divided into traditional point, line, and surface feature matching methods and semantic structure feature matching methods. Point, line, and surface feature matching methods typically perform feature matching on geometric features extracted within the scene. They have wide applications and good real-time performance, but in feature-sparse scenarios, their matching is easily affected by point cloud noise, leading to reduced accuracy.

[0004] In SLAM algorithms based on structured semantic feature matching, structural semantic features such as cylinders, ground surfaces, and planes are typically used for matching, achieving higher matching accuracy compared to SLAM algorithms based on point, line, and surface features. However, extracting structured semantic features is time-consuming, leading to reduced real-time performance. Furthermore, current SLAM algorithms based on structured semantic feature matching usually employ a single structural semantic feature, while real-world environments typically contain multiple structural semantic features. Therefore, SLAM algorithms based on a single structural semantic feature struggle to achieve optimal matching results. Summary of the Invention

[0005] The purpose of this invention is to provide a lidar SLAM method based on multimodal structural semantic features to solve the problems existing in the prior art.

[0006] To achieve the above objectives, this invention provides a lidar SLAM method based on multimodal structural semantic features, comprising the following steps:

[0007] Acquire lidar point cloud data;

[0008] The laser radar point cloud data is segmented into cylindrical point clouds based on the ERFNet method.

[0009] The ground point cloud segmentation of the lidar point cloud data is performed based on the BEV projection view method.

[0010] The lidar point cloud data is segmented into planar point clouds based on the region growing method.

[0011] The cylindrical point cloud is fitted into a cylindrical model, and the ground point cloud and the planar point cloud are fitted into planar models respectively;

[0012] The cylindrical model and the planar model are projected onto the world coordinate system to obtain the matching relationship between different cylindrical models and the matching relationship between different planar models, thus constructing a structural semantic map;

[0013] The structural semantic map is stored based on a voxel plane;

[0014] The structural semantic map is updated based on model matching and map incremental update algorithms.

[0015] Optionally, the process of performing cylindrical point cloud segmentation on the lidar point cloud data based on the ERFNet method includes:

[0016] The lidar point cloud data is converted into a distance image. The pixels in the distance image are divided into grids based on the density of the lidar point cloud. Each grid is labeled to obtain the cylindrical point cloud.

[0017] Optionally, the process of performing ground point cloud segmentation on the lidar point cloud data based on the BEV projection view method includes:

[0018] The lidar point cloud data is projected into a BEV view, and the point cloud coordinates after projection onto the BEV view are obtained. The point cloud is divided into a concentric circle grid in the xy plane, and the point cloud with the lowest height in each grid is retained as ground features.

[0019] Optionally, the process of performing planar point cloud segmentation on the lidar point cloud data based on the region growing method includes:

[0020] The lidar point cloud data is converted into a distance image, a pixel set is constructed, and the distance image is divided into grids.

[0021] Non-planar meshes with missing points and discontinuous distances between adjacent pixels are removed, and the remaining meshes are divided into planar meshes based on principal component analysis and mean square error method.

[0022] The remaining mesh is grown to obtain the planar point cloud.

[0023] Optionally, the process of growing the remaining grid includes:

[0024] A dynamic normal histogram is constructed, and the normal vectors of the remaining mesh are represented by polar coordinates and azimuth angles. The main directions of the normals of the remaining mesh are iteratively retrieved in the dynamic normal histogram, and the mesh with the smallest mean square error in the plane is given priority as the seed mesh for region growth.

[0025] The grown mesh is removed from the dynamic normal histogram. The current seed mesh and its neighboring meshes are iteratively checked to see if they are on the same plane. If they are on the same plane, the grown mesh is used as the seed mesh for iterative region growing until there are no neighboring meshes that meet the conditions.

[0026] The planar point cloud is obtained based on the ratio of the second largest eigenvalue to the smallest eigenvalue of the planar covariance matrix.

[0027] Optionally, the process of obtaining the matching relationship between different cylindrical models includes:

[0028] The average distance between three feature points at different heights between two cylindrical models is taken as the threshold distance between the cylinders. When the distance between the two cylinders is less than the threshold distance, the matching relationship between the two cylinders is established.

[0029] Optionally, the process of obtaining the matching relationship between different planar models includes:

[0030] The point cloud of the planar model is stored as a planar map. The coordinate information of the current planar point is obtained. A KD tree is used to search for several nearest points in the planar map. A distance threshold is constructed. If the distance between the nearest points is less than the distance threshold, the planar indices corresponding to several nearest points are extracted. The map plane with the most nearest points is matched with the current planar point.

[0031] Optionally, the process of storing the structural semantic map based on the voxel plane includes:

[0032] Construct voxel parameters, store the structural semantic map according to the voxel parameters, and when performing map matching or update operations, only operate on the small planes within the corresponding voxel, and store multiple planes within the same voxel.

[0033] Optionally, the process of updating the structural semantic map based on model matching and map incremental update algorithms includes:

[0034] Project the structural semantic map onto the world system to obtain the projected map, calculate the distance between the projected map and all map models, and extract the minimum distance value and the corresponding map model.

[0035] A distance threshold is constructed. When the minimum distance value is less than the distance threshold, it is considered a successful match. The feature points are superimposed and refitted to obtain a new map model. The new map model replaces the map model in the original map. If the match is unsuccessful, the projected map is added to the model map as a new map model.

[0036] The technical effects of this invention are as follows:

[0037] This invention proposes a LiDAR SLAM method based on multimodal structural semantic features. By extracting three types of structured semantic features—cylinder, ground, and plane—to constrain pose estimation, it effectively improves the positioning accuracy of LiDAR SLAM. Furthermore, this invention maintains an intuitive multimodal structural semantic map, enabling the construction and real-time updating of the three structural semantic model maps. Compared to point, line, and surface feature-based SLAM algorithms such as A-LOAM, LEGO-LOAM, LIO-SAM, and FAST-LIO, the proposed algorithm achieves higher positioning accuracy both indoors and outdoors and is more adaptable to different scenarios. Attached Figure Description

[0038] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0039] Figure 1 This is an architecture diagram of the SLAM method based on multimodal structural semantic features in an embodiment of the present invention;

[0040] Figure 2 This is an example diagram of ground point cloud mesh division in an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the matching relationship of the cylindrical model in an embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of voxel-based storage in an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0045] like Figure 1-4As shown, this embodiment proposes a multimodal structural semantic feature LiDAR SLAM method, which can jointly extract and match cylindrical, ground, and planar features. First, a grid-based multimodal structural semantic feature extraction method is proposed to effectively extract cylindrical, ground, and planar features. Then, matching cost functions are constructed for each of the three modal features to achieve joint matching of multimodal features. Next, a parameterized description model for the three modal features is established, and an incremental map storage and update method is designed accordingly. Furthermore, this embodiment conducts outdoor and indoor tests and compares and analyzes the results with algorithms such as A-LOAM, LEGO-LOAM, LIO-SAM, and FAST-LIO.

[0046] This embodiment proposes a SLAM method based on multimodal structural semantic features. It extracts and fits cylindrical, ground, and planar models by semantically segmenting LiDAR point clouds, which are then used for global optimization of the odometry to improve positioning accuracy. The overall algorithm architecture includes four modules: input, semantic segmentation, feature matching, and map maintenance.

[0047] like Figure 1 As shown, the input includes a LiDAR point cloud and a velocity prediction module, and the output is a LiDAR odometry and a structural semantic map. In the semantic segmentation module, structural semantic features are extracted from the LiDAR point cloud, resulting in cylindrical, ground, and planar point clouds for the current frame. Models corresponding to these three features are then fitted to each. In the feature matching module, the three structural semantic models for the current frame are matched with their corresponding structural semantic maps to obtain the correspondence between the current frame and the map, which is used to globally optimize the LiDAR odometry for the current frame. In the map maintenance module, the optimized current frame pose and structural semantic model are used to update the corresponding structural semantic map.

[0048] 1. A multimodal structural semantic feature extraction method for cylinders, ground surfaces, and planes:

[0049] (1) Grid-based multimodal semantic model segmentation and extraction:

[0050] Semantic segmentation refers to extracting meaningful objects from raw input data. Currently, semantic feature extraction typically involves directly inputting LiDAR point clouds or distance images for subsequent processing; however, this method is computationally intensive and affects the algorithm's real-time performance. This embodiment proposes a grid-based multimodal semantic model segmentation and extraction method, which converts LiDAR point cloud information into a grid image and then extracts three structural semantic features.

[0051] 1) Cylindrical point cloud segmentation based on ERFNet:

[0052] This embodiment uses ERFNet for cylindrical point cloud segmentation and extraction. The input to ERFNet is an h×w range image converted from the LiDAR point cloud, using the formula shown below. Based on the density of the LiDAR point cloud, multiple pixels in the range image are divided into a grid, effectively improving cylindrical segmentation efficiency. ERFNet ultimately assigns a label to each grid in the image, indicating whether the grid belongs to a specific cylinder.

[0053]

[0054] Where (u,v) represent the horizontal and vertical coordinates of the distance image, respectively, (w,h) represent the width and height of the distance image, respectively, i.e., w is the number of points scanned in one revolution of the lidar, h is the number of lines scanned by the lidar, and f = f up +f down This indicates the vertical field of view of the lidar. This represents the distance value for each point.

[0055] 2) Ground point cloud segmentation based on BEV projection view:

[0056] Considering the possibility of slopes or other complex terrain in actual situations, and assuming that the ground is a local plane and extracting ground features, this embodiment projects the lidar point cloud into a BEV view, as shown in the figure.

[0057]

[0058] Where (r,θ) are the coordinates of the current point projected onto the BEV view, (r n ,θ n ) indicates that the projection is r n ×θ n The grid is defined by R, where R is the maximum radius of the point cloud projection, p is the coordinate of the current projection point, and o is the current coordinate of the carrier.

[0059] Considering the unevenness of the ground, the point cloud is divided into concentric circular grids in the xy plane. Figure 2 An example of ground point cloud mesh generation is provided. Finally, only the lowest 10% of points in each mesh are retained as ground features to reduce computational cost.

[0060] 3) Planar point cloud segmentation based on region growing:

[0061] Considering the efficiency and accuracy of planar point cloud extraction, this embodiment designs a region growing method based on a gridded distance image for the segmentation and extraction of planar point clouds. Similar to the processing of point clouds by cylindrical semantic segmentation, the radar point cloud is converted into a distance image, and the m×n pixels are set into a grid to improve processing efficiency.

[0062] First, a coarse planar fit is performed on the grid image, and grids that meet the following conditions are classified as non-planar grids:

[0063] i. Having obvious missing points: A grid should contain m·n pixels. If the number of pixels in the current grid is less than m·n / 2, it is defined as having obvious missing points.

[0064] ii. Discontinuous distance between adjacent pixels: If the distance values ​​of adjacent pixels in the same grid differ too much and the direction of change is not monotonic, it is defined as discontinuous distance between adjacent pixels.

[0065] Next, non-planar meshes are removed, and the remaining meshes are fitted to a plane using Principal Component Analysis (PCA). The eigenvalues ​​of the covariance matrix are λ1, λ2, and λ3, with corresponding eigenvectors ξ1, ξ2, and ξ3, where λ1 < λ2 < λ3. The fitted mean square error (MSE) is then λ1, and the plane normal vector is ξ1. If the following conditions are met, the mesh is classified as a planar mesh.

[0066] MSE<(σ d +ε) 2 (3)

[0067] Where, σ d ε represents the standard uncertainty of the average distance of the point cloud within the grid, and ε represents a tolerance coefficient.

[0068] Finally, the remaining mesh is grown to extract the planar structure, as shown in Table 1.

[0069] Table 1

[0070]

[0071] A dynamic normal histogram H is established in spherical coordinates, where the normal vectors of the planar mesh are represented by polar coordinates and azimuth angles. The dominant directions of the planar mesh normals are iteratively retrieved from the histogram, and the corresponding set of planar meshes C is extracted. The planar mesh s with the smallest MSE is preferentially used as the seed mesh for region growing. Simultaneously, the mesh set R after region growing is deleted from the histogram H to facilitate finding the dominant directions of the remaining planar mesh normals for the next region growing iteration. The region growing algorithm iteratively checks whether the current seed mesh and its adjacent meshes are on the same plane. If they are on the same plane, the growing mesh is used as the seed mesh for iterative region growing until no adjacent meshes meet the conditions. Adjacent meshes c that satisfy the following conditions are considered coplanar with the seed mesh s:

[0072] The angle between the normal vectors is and c is less than θM .

[0073] ii.c The distance from the center point to s is less than the dynamic distance threshold D(s), and its value is positively correlated with the distance between the two grids.

[0074] Finally, the ratio r of the second largest eigenvalue to the smallest eigenvalue of the planar covariance matrix is ​​used. λ Assess the planarity of the entire surface after the region is grown; if r λ If it is large enough, mark the current face as a plane.

[0075] (2) Model fitting and parameter estimation based on feature point clouds:

[0076] After extracting the corresponding structural semantic features, the features need to be fitted into corresponding models for subsequent matching with the structural semantic map. In this embodiment, the extracted cylindrical features are fitted into a cylindrical model, and the ground and planar features are fitted into planar models respectively.

[0077] a) Cylindrical model:

[0078] In this embodiment, the cylindrical feature points are fitted with parameters c = (ρ, φ, ν, α, κ) to represent the cylindrical model, n c Let ρ be the normal vector of the cylinder at the point closest to the origin, and φ be the unit vector along the direction of the cylinder's central axis. Here, ρ represents the distance from the origin to the nearest point on the cylinder, and φ represents the vector n. c The angle between the projection of the vector n onto the xy-plane and the x-axis, ν represents the vector n. c The angle between the z-axis and the z-axis, α, represents the angle between a and n. c The angle between the partial derivatives with respect to ν, where 1 / κ represents the radius of the cylinder. The normal vector n mentioned above... c The axis a of the cylinder's center can be calculated from the parameters (ρ, φ, ν, α, κ) using the following formula:

[0079] n c =(cosφsinν,sinφsinν,cosν) (4)

[0080]

[0081] in, For n c The partial derivative with respect to ν, By n c The partial derivative with respect to φ is derived.

[0082] The distance d from any point p to cylinder c c for:

[0083]

[0084] Given a set of cylinders of the same type, the quantity of which is δ. i,k+1 cylindrical feature points Each can be utilized The cylinder parameter c is estimated by least-squares optimization using the sum of distances to the cylinder. However, when κ is small, the distances from a point to the cylinder may produce singularities, causing problems in the optimization process. Therefore, for a feature point p and a cylinder c, the following distances can be used... Approximate distance d from the point to the cylinder c :

[0085]

[0086] Given that each cylinder has δ i Given a set of feature points, the least squares optimization problem is as follows:

[0087]

[0088] b) Planar model:

[0089] This embodiment locally fits ground features and planar features into a planar model, using the parameter π = (n p Define the plane: {x| ,β) <x,n p >+β=0}, where n p Let β be the normal vector of the plane, and β be the offset.

[0090] The distance d from any point p to the plane π π for:

[0091]

[0092] Here, x0 represents any point on the plane.

[0093] Given a set of feature points belonging to the same ground or plane There are m features, and a 3×m matrix H is represented using the coordinates of the feature points. The center of the planar feature points, matrix H satisfy For matrix H Singular value decomposition (SVD) is used to estimate the planar model.

[0094] H =UΣV T (10)

[0095] in, Normal vector n p The estimate corresponds to the left singular vector of the minimum singular value of U. This is derived from the plane normal vector n. p and the center of the plane The offset β in the plane equation can be easily obtained.

[0096] 2. Semantic map-based matching and global optimization:

[0097] Matching semantic models and semantic maps involves projecting the semantic model onto the world coordinate system W, then finding the model in the semantic map with the smallest distance to the current semantic model that is less than a certain threshold, and establishing a matching relationship between the two semantic models. The calculation method for the distance between semantic models varies depending on the specific model being matched.

[0098] a) Cylindrical model:

[0099] In real-world scenarios, cylindrical models often have a certain angle of inclination. Therefore, when calculating the distance between two cylindrical models, it is inappropriate to directly calculate the distance between feature points of the two cylindrical models at the same height as the distance between the cylinders. Instead, the average distance between the two cylindrical models at three different heights should be taken as the distance between the cylinders. Figure 3 As shown in Table 2, a matching relationship is established based on the distance between the current cylindrical model and the cylindrical map model.

[0100] Table 2

[0101]

[0102] The cylindrical model C extracted by traversing the point cloud of the current frame c After converting it to the world coordinate system W, in the cylindrical map C m The KD-tree (K-dimensional Tree) recently found the most recent map model c. m If the distance between the two cylinders is less than the threshold l m Then, establish the matching relationship M between the two cylinders. c .

[0103] b) Planar model:

[0104] In this embodiment, the stored planar map is in point cloud form, that is, the point cloud of the planar model in the map is stored as a planar map, and each point is composed of parameters (x, y, z, i), where (x, y, z) are the coordinate information of the point, and i is the plane index to which the current point belongs. The specific matching method is shown in Table 3:

[0105] Table 3

[0106]

[0107] Iterate through the point cloud of the current frame to extract the planar points P c After converting it to the world coordinate system W, on the plane map P mThe KD-tree recently searched 10 points {p1, p2, ..., p...} 10 If the distance to the nearest point is less than the threshold l m Then extract the plane indices {i1,i2,…,i} corresponding to the 10 points. 10}, establish a matching relationship M between the map plane with the largest number of points and the current plane points. p .

[0108] Given that there are N cylinders in the current frame, and δ feature points for each cylinder. i Given γ local ground features and η planes, the pose of the current frame can be estimated by optimizing the following nonlinear least squares problem.

[0109]

[0110] Where, λ t , λ g , λ p These are weighting coefficients used to adjust the magnitude of the impact of the distance function on the optimization results for each model.

[0111] 3. Incremental updating and maintenance of structural semantic map:

[0112] (1) Voxel-based planar map storage method:

[0113] Considering that large planes have poor fitting properties to scene structures, using fewer parameters to represent large regions will result in the loss of a lot of detailed information, such as region boundaries. Furthermore, improper threshold settings may lead to fitting non-planar regions as large planes, thus significantly impacting the final mapping and localization. Therefore, this embodiment proposes a voxel-based planar map, storing the extracted large planes in voxels of a certain size. During matching or updating operations, only the smaller planes within the corresponding voxel need to be operated on, and multiple planes can be stored within the same voxel. A schematic diagram is shown below. Figure 4 This not only avoids losing too much scene detail information, but also improves the accuracy of matching and positioning.

[0114] (2) Model-based matching and incremental map updates:

[0115] Currently, map updates typically employ keyframe point cloud overlay and downsampling, which is computationally intensive. This embodiment designs a general model matching and incremental map update algorithm for structural semantic models. Based on the matching results between the current frame and the map, only the semantic structural model of the current frame is incrementally updated into the structural semantic map, effectively reducing the computational cost of map updates, as shown in Table 4:

[0116] Table 4

[0117]

[0118]

[0119] Traversing the semantic structure model of the current frame O c The carrier pose optimized using least squares The current traversal model o c Projected onto the world system W to obtain o W , calculate o W Distances between all map models and extract the minimum distance value d. m and its corresponding map model o d If d m If the value is less than a certain threshold, it is considered a successful match, and o is set to... W Feature point superposition o d Feature points are identified and refitted to obtain a new model. n Replace the o in the original map d Unlike using "o" directly. W Replace o d Feature point overlay and refitting can reduce the model's fitting error and improve matching accuracy. If a match is unsuccessful, then o W It is added to the model map as a new map model.

[0120] As the map expands and updates, the number of semantic map models increases. However, an excessively large semantic map used for matching with the current frame's model can slow down the matching process. Therefore, the semantic map used for matching employs a KD-tree nearest search approach, searching for a certain number of models closest to the current position to generate corresponding semantic sub-maps for matching. This reduces computational cost while maintaining matching accuracy.

[0121] 4. Experimental Results and Analysis:

[0122] To verify the algorithm in this embodiment, it was deployed on an experimental platform for prototype testing. An unmanned vehicle was used as the carrier, equipped with an Ouster OS1-64 LiDAR and a microcontroller with an Intel Core i9 8950 CPU and an NVIDIA GTX 1650 GPU. The Ouster OS1-64 LiDAR incorporated an InvenSense ICM-20948 IMU. Outdoor experiments were conducted in a campus environment, while indoor experiments were conducted in a corridor environment. RTK positioning results were used as the ground truth for outdoor experiments, and total station positioning results were used as the ground truth for indoor experiments.

[0123] (1) Outdoor scene experiment:

[0124] In outdoor scenarios, the autonomous vehicle travels along a path in a campus environment, and its trajectory is approximately rectangular. The error analysis is shown in Table 5.

[0125] Table 5

[0126]

[0127] Through outdoor scene experiments, the experimental data shows that:

[0128] The algorithm in this embodiment can accurately extract cylindrical structures (trees) and planar structures (building walls) in the scene and construct a complete structural semantic map.

[0129] The algorithm in this embodiment improves the positioning accuracy in both the horizontal and vertical directions compared to traditional algorithms. The positioning accuracy is improved by 40.89%, 19.21%, 24.73%, and 15.61% compared to A-LOAM, LEGO-LOAM, LIO-SAM, and FAST-LIO, respectively.

[0130] (2) Indoor scene experiment:

[0131] In an indoor scenario, the autonomous vehicle travels in a loop within an indoor corridor, with a trajectory that approximates a rectangle. Error analysis is shown in Table 6.

[0132] Table 6

[0133]

[0134] Through indoor scene experiments, the experimental data shows that:

[0135] The algorithm in this embodiment can accurately extract the planar structure (walls) in the corridor environment and construct a complete structural semantic map.

[0136] The root mean square error in both the horizontal and vertical directions of the algorithm in this embodiment is reduced compared to the classic algorithm. The positioning accuracy is improved by 54.76%, 60.39%, 10.60%, and 5.06% compared to A-LOAM, LEGO-LOAM, LIO-SAM, and FAST-LIO, respectively.

[0137] This invention proposes a SLAM method based on multimodal structural semantic features. First, a multimodal feature extraction front-end is designed to extract three structural semantic information elements—cylinder, ground, and plane—from LiDAR point clouds. Second, matching cost functions for cylinder, ground, and plane features are constructed to match the three structural semantic information elements with the corresponding constructed structural semantic maps. Finally, storage and update methods based on parametric models are designed for each of the three structural semantic information elements, enabling the updating and maintenance of the structural semantic maps. Ultimately, the algorithm of this embodiment is validated through outdoor and indoor experiments, and its accuracy is compared with commonly used LiDAR SLAM algorithms such as A-LOAM, LEGO-LOAM, LIO-SAM, and FAST-LIO.

[0138] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lidar SLAM method based on multimodal structural semantic features, characterized in that, Includes the following steps: Acquire lidar point cloud data; The laser radar point cloud data is segmented into cylindrical point clouds based on the ERFNet method. The ground point cloud segmentation of the lidar point cloud data is performed based on the BEV projection view method. The lidar point cloud data is segmented into planar point clouds based on the region growing method. The cylindrical point cloud is fitted into a cylindrical model, and the ground point cloud and the planar point cloud are fitted into planar models respectively; The cylindrical model and the planar model are projected onto the world coordinate system to obtain the matching relationship between different cylindrical models and the matching relationship between different planar models, thus constructing a structural semantic map. The structural semantic map is stored based on a voxel plane; The structural semantic map is updated based on model matching and map incremental update algorithms.

2. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of performing cylindrical point cloud segmentation on the lidar point cloud data based on the ERFNet method includes: The lidar point cloud data is converted into a distance image. The pixels in the distance image are divided into grids based on the density of the lidar point cloud. Each grid is labeled to obtain the cylindrical point cloud.

3. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of performing ground point cloud segmentation on the lidar point cloud data based on the BEV projection view method includes: The lidar point cloud data is projected into a BEV view, and the point cloud coordinates after projection onto the BEV view are obtained. The point cloud is divided into a concentric circle grid in the xy plane, and the point cloud with the lowest height in each grid is retained as ground features.

4. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of performing planar point cloud segmentation on the lidar point cloud data based on the region growing method includes: The lidar point cloud data is converted into a distance image, a pixel set is constructed, and the distance image is divided into grids. Non-planar meshes with missing points and discontinuous distances between adjacent pixels are removed, and the remaining meshes are divided into planar meshes based on principal component analysis and mean square error method. The remaining mesh is grown to obtain the planar point cloud.

5. The lidar SLAM method based on multimodal structural semantic features according to claim 4, characterized in that, The process of growing the remaining grid includes: A dynamic normal histogram is constructed, and the normal vectors of the remaining mesh are represented by polar coordinates and azimuth angles. The main directions of the normals of the remaining mesh are iteratively retrieved in the dynamic normal histogram, and the mesh with the smallest mean square error in the plane is given priority as the seed mesh for region growth. The grown mesh is removed from the dynamic normal histogram. The current seed mesh and its neighboring meshes are iteratively checked to see if they are on the same plane. If they are on the same plane, the grown mesh is used as the seed mesh for iterative region growing until there are no neighboring meshes that meet the conditions. The planar point cloud is obtained based on the ratio of the second largest eigenvalue to the smallest eigenvalue of the planar covariance matrix.

6. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of obtaining the matching relationship between different cylindrical models includes: The average distance between three feature points at different heights between two cylindrical models is taken as the threshold distance between the cylinders. When the distance between the two cylinders is less than the threshold distance, the matching relationship between the two cylinders is established.

7. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of obtaining the matching relationship between different planar models includes: The point cloud of the planar model is stored as a planar map. The coordinate information of the current planar point is obtained. A KD tree is used to search for several nearest points in the planar map. A distance threshold is constructed. If the distance between the nearest points is less than the distance threshold, the planar indices corresponding to several nearest points are extracted. The map plane with the most nearest points is matched with the current planar point.

8. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of storing the structural semantic map based on the voxel plane includes: Construct voxel parameters, store the structural semantic map according to the voxel parameters, and when performing map matching or update operations, only operate on the small planes within the corresponding voxel, and store multiple planes within the same voxel.

9. The lidar SLAM method based on multimodal structural semantic features according to claim 1, characterized in that, The process of updating the structural semantic map based on model matching and incremental map update algorithms includes: Project the structural semantic map onto the world system to obtain the projected map, calculate the distance between the projected map and all map models, and extract the minimum distance value and the corresponding map model. A distance threshold is constructed. When the minimum distance value is less than the distance threshold, it is considered a successful match. The feature points are superimposed and refitted to obtain a new map model. The new map model replaces the map model in the original map. If the match is unsuccessful, the projected map is added to the model map as a new map model.

Images