Point cloud global positioning method fusing semantic information and geometric features

Abstract

The invention provides a semantic information and geometric feature fused point cloud global positioning method, and belongs to the field of robot positioning. The method comprises the following steps: firstly, screening and initializing candidate positions based on semantic information of a prior map point cloud so as to avoid a large amount of calculation load caused by invalid candidate positions; secondly, constructing a semantic topological graph of the priori map point cloud and the laser radar scanning point cloud, calculating the topological similarity between the candidate position and the laser radar scanning point cloud, and coarsely positioning the ground robot to several areas with high topological similarity; and finally, constructing a point cloud global descriptor encoder network, extracting a point cloud global descriptor on the candidate position after coarse positioning, and calculating the similarity between the point cloud global descriptor and the global descriptor of the laser radar scanning point cloud to complete fine positioning. According to the method, the ground robot is guided to perform accurate positioning based on the aerial view angle map, and the fundamental weakness of a traditional method in coping with challenges such as large scale, violent view angle change and scene repetition is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of robot positioning, and relates to a point cloud global positioning method fusing semantic information and geometric features. BACKGROUND

[0002] Before a ground robot is guided by a large-scene aerial-view outdoor prior map to perform an autonomous environment exploration task, the position of the ground robot in the prior map needs to be determined, that is, the ground robot is positioned, that is, the coordinate system of the ground robot is unified with the coordinate system of the aerial-view prior map, and a coordinate conversion mode of the ground robot coordinate system and the prior map coordinate system is established. When the air-ground collaborative task starts, because the unmanned aerial vehicle is not limited to taking off from the starting position of the ground robot, the initial position guess of the ground robot in the aerial-view prior map cannot be obtained, and the laser radar scanning of the ground robot needs to be matched and positioned with the prior map. Traditional methods mostly use position recognition based on point cloud geometric features, such as a Chinese invention patent with the application number CN202510673273.0, which discloses a laser SLAM loop correction method based on scene matching, aiming to comprehensively describe the global characteristics of point cloud and improve the scene matching accuracy. However, the traditional method only focuses on the geometric features of the point cloud scene, such as point, line, and surface information. Under the conditions of great difference in air-ground scene perspective and scene repetition, the geometric information changes dramatically and loses its uniqueness, which easily causes positioning and matching failure. SUMMARY

[0003] In view of the global positioning of the point cloud map under a large-scene aerial view, the application provides a point cloud global positioning method fusing semantic information and geometric features.

[0004] To achieve the above object, the application provides the following solutions.

[0005] A point cloud global positioning method fusing semantic information and geometric features, comprising the following steps.

[0006] S1, based on the semantic information of the prior map point cloud, screening and initializing candidate positions;

[0007] S2, constructing a semantic topology graph of the prior map point cloud and the laser radar scanning point cloud, calculating the topology similarity between the candidate positions and the laser radar scanning point cloud, and coarsely positioning the ground robot to several regions with high topology similarity;

[0008] S3, constructing a point cloud global descriptor encoder network, extracting point cloud global descriptors at the candidate positions after coarse positioning, and calculating the similarity of the point cloud global descriptors with the laser radar scanning point cloud global descriptors to complete fine positioning.

[0009] Specifically, the following steps are included.

[0010] S1, filtering and initializing candidate positions based on semantic information of the prior map point cloud;

[0011] The prior map point cloud and the laser radar scanning point cloud are acquired, and both have semantic labels. Meanwhile, the laser radar scanning point cloud further includes position information three-dimensional coordinates and instance labels of points in the point cloud.

[0012] In the prior map point cloud, a region with a semantic label of "ground" is filtered out. Candidate positions are uniformly initialized at intervals in the filtered region, wherein the interval distance is 3.0-3.5 m.

[0013] S2, constructing a semantic topology graph of the prior map point cloud and the laser radar scanning point cloud, calculating the topological similarity between the candidate positions and the laser radar scanning point cloud, and coarsely positioning the ground robot to several regions with high topological similarity.

[0014] S2.1, respectively constructing a semantic topology graph of the prior map point cloud and the laser radar scanning point cloud to save semantic topology information.

[0015] For an instance object in the laser radar scanning point cloud , its original point cloud is represented as a semantic node containing semantic topology information, and the semantic topology information includes a point cloud geometric position center and a semantic category label , wherein =1, 2,..., n, n is the number of instances contained in the laser radar scanning point cloud, and the semantic topology graph of the laser radar scanning point cloud is represented as ={ ... ,... }.

[0016] For an instance object of any candidate position in the prior map point cloud , its original point cloud is also represented as a semantic node containing semantic topology information, and the semantic topology information includes a point cloud geometric position center , a semantic category label , and a distribution range around the geometric position center , wherein =1, 2,..., m, m is the number of instances contained in the corresponding candidate position, and represent the minimum edge radius and the maximum edge radius of the instance respectively, and the semantic topology graph of any candidate position of the prior map point cloud is represented as ={ ... ,..., }.

[0017] S2.2 Calculate the topological similarity between the candidate locations and the LiDAR scan point cloud.

[0018] For instance objects in the LiDAR scan point cloud and instance objects located near candidate positions in the prior map point cloud If the instance object Geometric position center Located in the instance object Geometric position center The radius of the vicinity is If the two semantic nodes are within the same ring, then they are considered to have the same semantic label, that is, the two semantic nodes have a matching relationship.

[0019] Compare each candidate's position one by one All semantic nodes in The matching relationship of each semantic node in the text is determined, and each candidate position is calculated. The topological similarity M between each candidate location and the LiDAR scanned point cloud is obtained by calculating the proportion of semantic nodes with matching relationships to the total number of semantic nodes. The formula is as follows:

[0020] , =1,2,...,n, =1,2,...,m

[0021] in, ,express and The spacing, (·) indicates the number of elements in the set.

[0022] S2.3, Set the semantic topological similarity threshold Candidate positions with a topological similarity greater than a set threshold are selected as coarse localization candidate positions, where the threshold range is 0.5-0.8.

[0023] S3. Construct a point cloud global descriptor encoder network to extract point cloud global descriptors at candidate locations after coarse localization and calculate their similarity with the global descriptors of the point cloud scanned by the LiDAR to complete fine localization.

[0024] S3.1, Construct the point cloud global descriptor encoder network;

[0025] The point cloud global descriptor encoder network comprises two parallel local feature extraction branches and a global descriptor generator. Specifically, the two local feature extraction branches are a point-based convolutional branch and a voxel-based convolutional branch. The point-based convolutional branch consists of a multilayer perceptron (MLP), including an input layer, hidden layers, and an output layer, used to extract fine-grained features of the point cloud. The hidden layers include shallow and deep MLPs. The voxel-based convolutional branch is responsible for aggregating information at different resolutions, including a preprocessing layer, a normalization layer, a sparse 3D convolutional layer, and an upsampling layer, used to extract macroscopic structural features of the point cloud. The global descriptor encoder includes a fusion layer and a full aggregation layer, used to aggregate the local features generated by the two local feature extraction branches with the semantic and instance information of the point cloud to obtain the final point cloud global descriptor.

[0026] S3.2, Generate global descriptors for candidate positions.

[0027] At each candidate location, a LiDAR scanning point cloud is generated using GazeboVelodyne simulation. The point-based convolutional branch and the voxel-based convolutional branch in the point cloud global descriptor encoder network are input simultaneously to output the feature matrix and feature pyramid, respectively. These two are then input into the fusion layer of the global descriptor generator to finally output the global descriptor of the candidate location.

[0028] After the generated LiDAR scan point cloud is input via a voxel-based convolutional branch, the preprocessing layer first transforms the LiDAR scan point cloud into a coordinate system with the point cloud centroid as the origin; the normalization layer then normalizes it to eliminate the influence of point cloud scale differences. For any point cloud... One point Its normalized point cloud can be represented as:

[0029]

[0030] in, Represents the normalized point. Point cloud The center of mass, Point cloud The point furthest from the centroid, where i = 1, 2, ..., h, and h represents the point cloud. The number of midpoints;

[0031] Sparse 3D convolutional layers transform normalized point clouds into regular voxel meshes. Through sparse 3D convolution, the network captures the geometric and semantic features of the point cloud. For each voxel, its feature is the mean of the features of all points contained in the voxel grid, and the final feature value map of its c-th channel is obtained. Represented as:

[0032]

[0033]

[0034]

[0035]

[0036] where c is the channel value in the feature value four-dimensional tensor, i.e., the quantization result of the geometric feature or semantic feature extracted from the point cloud, denotes the voxel resolution; [·] is an indicator function, which is 1 when the point belongs to the voxel grid (u, v, w) and 0 otherwise; denotes the feature information of the cth channel of the point ; is the number of points contained in the voxel grid (u, v, w); ( , , ) denotes the voxel index coordinates of the point ; is a floor function, which discretizes the continuous coordinates to obtain integer voxel indexes; ( , , ) denotes the normalized coordinates of the point .

[0037] Finally, the up-sampling layer restores the scale of the features output by the sparse three-dimensional convolutional layer through transpose convolution and jump connection, restores the geometric features and semantic features of the point cloud point by point, and finally outputs a feature pyramid, any layer of the pyramid contains a feature value map, which saves the context information of the point cloud, including global semantic layout information, low-resolution geometric detail information, etc.

[0038] The generated laser radar scan point cloud input is input into the input layer of the point-based convolution branch, and then is subjected to nonlinear transformation by the hidden layer, specifically, feature dimension increase and basic mapping are performed in the shallow MLP to learn the normal vector direction and local curvature and other geometric patterns of the points from the original three-dimensional coordinates; the deep MLP combines the corner points and plane structures from the edge information of the point cloud to obtain the structural features of the point cloud; finally, high-dimensional feature mapping is performed through the output layer to obtain a set of high-dimensional feature mapping matrices corresponding to the input point cloud as the final output, and each row in the matrix represents the high-resolution, fine-grained local geometric features of a point in the point cloud. The detailed information in the original laser radar scan point cloud of the candidate position is modeled in the point-based convolution branch, and a multi-layer perceptron is directly applied to each point in the point cloud to extract the detailed features of the single point. Such point-based features are high-resolution and complementary to the coarse-grained information based on voxels.

[0039] The feature matrix output by the point-based convolution branch and the feature pyramid output by the voxel-based convolution branch are input into the fusion layer of the global descriptor generator, and the fusion layer adopts a U-Net-based encoder-decoder structure. First, the feature pyramid is mapped back to the point feature through trilinear interpolation to generate a feature matrix that saves the context information on the voxels, and then the feature matrix is spliced with the feature matrix output by the point-based convolution branch to obtain a fusion feature matrix. Finally, through the full aggregation layer NetVLAD, all the fusion feature matrices of the points are aggregated into a feature description vector of global information, that is, the global descriptor of the candidate position.

[0040] S3.3, fine positioning is completed based on the similarity of the global descriptor.

[0041] By using The norm is used to compare the similarity of the global descriptor of each candidate position and the global descriptor of the laser radar scan point cloud, representing the distance between the two global descriptors in the feature space. The smaller the distance, the higher the similarity of the two descriptors:

[0042]

[0043] wherein, represents the global descriptor of the current robot laser radar scan point cloud, represents the global descriptor of the simulated point cloud of any candidate position.

[0044] A global descriptor similarity threshold value is set, and candidate positions less than are considered to match the current robot laser radar scan, and the candidate position is the final positioning position of the robot. The similarity threshold value is set to 0-0.55.​

[0045] In some embodiments, in the S3, in order to reduce the amount of calculation for generating global descriptors subsequently, a DBSCAN clustering method based on density with noise application is used to cluster the coarse positioning candidate positions obtained in the S2, to obtain candidate position clusters, a minimum spherical region formed by several representative points in the candidate position cluster is used as the center position of the candidate position cluster, and the center position is used as the representative position of the corresponding candidate position cluster, wherein the number of center positions is set to 10 -2 ~10 -3 orders of magnitude of the total number of points in the candidate position. The point cloud global descriptor encoder network is used to generate the point cloud global descriptor of the representative position of each candidate position cluster, and the similarity between the point cloud global descriptor and the global descriptor of the laser radar scan point cloud is calculated to complete the fine positioning.

[0046] Advantages of the present application:

[0047] The positioning method of the present application fuses semantic information and geometric features, through multi-level and cross-modal information fusion, the robot performs scene recognition instead of simple shape recognition, realizes accurate positioning of the ground robot based on the aerial view map, and solves the fundamental weakness of the traditional method in dealing with large-scale, dramatic changes in view angle and scene repetition challenges. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 The method flowchart of the present application.

[0049] Figure 2 The semantic node matching relationship diagram, wherein, represents a certain instance object in the laser radar scan point cloud, represents a certain instance object at a certain candidate position, and represent the minimum edge radius and the maximum edge radius of the instance, respectively.

[0050] Figure 3 The positioning process diagram of the present embodiment, wherein (a) is the candidate position diagram after initialization, (b) is the coarse positioning candidate position region after the first iteration calculation, (c) is the coarse positioning candidate position region after the second iteration calculation, (d) is the coarse positioning candidate position region after the third iteration calculation, (e) is the coarse positioning candidate position region after the fourth iteration calculation, and (f) is the final positioning result after fine positioning. DETAILED DESCRIPTION

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] This embodiment takes STPLS3D dataset scene 01 as an example. The scene area is about 3000 square meters and includes a variety of urban landscapes such as streets, trees, buildings, parking lots, etc. It also contains a rich number of object instances. Taking road types as an example, it includes cement roads, dirt roads, sandy areas, asphalt roads, etc. The method of the present invention will be further described below with reference to the accompanying drawings.

[0053] A point cloud global localization method that integrates semantic information and geometric features, the process is as follows: Figure 1 As shown, it includes the following steps:

[0054] Step 1: Based on the semantic information of the prior map point cloud, filter and initialize candidate locations;

[0055] Obtain prior map point clouds and LiDAR scan point clouds, both of which have semantic tags. In addition, the LiDAR scan point cloud also includes the position information of points in the point cloud, 3D coordinates, and instance tags.

[0056] To ensure that candidate locations converge accurately and effectively to the true values, the number of candidate locations in the prior map point cloud needs to be increased. However, a significant increase in candidate locations leads to a surge in computational load, resulting in a sharp decline in efficiency. To avoid the large computational load caused by invalid candidate locations, this embodiment excludes areas where the ground robot is unlikely to appear, such as areas corresponding to buildings and trees, to reduce the number of candidate locations. Specifically, in the prior map point cloud, areas with the semantic label corresponding to "ground" are selected. Candidate locations are then uniformly initialized within the selected areas at certain intervals, where the interval is 3.0-3.5m. In this embodiment, an interval of 3 meters is selected. Figure 3 As shown in (a), the blue particles represent candidate positions after initialization, and the red particles represent the target area to be located by the ground robot.

[0057] Step 2: Construct a semantic topology graph of the prior map point cloud and the lidar scan point cloud, calculate the topological similarity between the candidate locations and the lidar scan point cloud, and coarsely localize the ground robot to several regions with high topological similarity.

[0058] Step 2.1: Construct semantic topology graphs of the prior map point cloud and the lidar scan point cloud respectively to save semantic topology information.

[0059] For instance objects in the LiDAR scan point cloud of a ground robot (like Figure 2 (As shown by the green dots in the middle), its original point cloud is represented as a semantic node containing semantic topological information, which includes the geometric location center of the point cloud. and its semantic category labels ,in, =1,2,...,n, where n is the number of instances included in the LiDAR scan, then the semantic topology graph of the LiDAR scan is represented as: ={ ... ,... }

[0060] For any candidate location instance in the prior map point cloud (like Figure 2 The blue nodes in the middle are also represented as semantic nodes containing semantic topological information, whose semantic topological information includes the geometric center of the point cloud. Semantic category tags In addition, it also includes its distribution range around the geometric center. (like Figure 2 (the blue ring-shaped area), in which, =1,2,...,m, where m is the number of instances contained in the corresponding candidate position. and Let represent the minimum edge radius and the maximum edge radius of the instance, respectively. Then the semantic topology graph of the prior map is: ={ ... ,..., }

[0061] Step 2.2: Calculate the topological similarity between the candidate locations and the LiDAR scan point cloud.

[0062] For instance objects in the LiDAR scan point cloud and instance objects located near candidate positions in the prior map point cloud If the instance object Geometric position center Located in the instance object Geometric position center The radius of the vicinity is If the two semantic nodes are within the same ring, then they are considered to have the same semantic label, that is, the two semantic nodes have a matching relationship.

[0063] Compare each candidate location with all semantic nodes in the semantic topology graph. The matching relationships between semantic nodes in the semantic topology graph of each candidate location are calculated, and the proportion of semantic nodes with matching relationships in the semantic topology graph of each candidate location to the total number of semantic nodes in the corresponding candidate location semantic topology graph is calculated. This proportion is the topological similarity between the candidate location and the LiDAR scan point cloud. The calculation formula is as follows:

[0064] , =1,2,...,n, =1,2,...,m

[0065] Step 2.3, set the semantic topological similarity threshold. Candidate positions with a topological similarity greater than a set threshold are selected as coarse-localization candidate positions. The threshold ranges from 0.5 to 0.8. Specifically, in this embodiment, a similarity threshold is selected. =0.55.

[0066] To reduce the computational load per iteration, this embodiment employs an iterative loop calculation method to complete the topological similarity calculation for all candidate locations and compare it with a similarity threshold, such as... Figure 3 As shown in (b) to (e), these are schematic diagrams of the candidate positions after the first to fourth rounds of iterative calculation and screening, respectively. Figure 3 As shown in (e), in this embodiment, after four rounds of screening, the final candidate position after coarse positioning is obtained.

[0067] Step 3: Construct a point cloud global descriptor encoder network, extract point cloud global descriptors at candidate locations after coarse localization, and calculate their similarity with the global descriptors of the point cloud scanned by the LiDAR to complete fine localization;

[0068] Step 3.1: Construct the point cloud global descriptor encoder network;

[0069] The point cloud global descriptor encoder network comprises two parallel local feature extraction branches and a global descriptor generator that fuses their outputs. Specifically, the two local feature extraction branches are a point-based convolutional branch and a voxel-based convolutional branch. The core of the point-based convolutional branch is a multilayer perceptron (MLP), including an input layer, hidden layers (including shallow and deep MLPs), and an output layer, used to extract fine-grained features of the point cloud. The voxel-based convolutional branch is responsible for aggregating information at different resolutions, including a preprocessing layer, a normalization layer, a sparse 3D convolutional layer, and an upsampling layer, used to extract macroscopic structural features of the point cloud. The global descriptor encoder includes a fusion layer and a full aggregation layer. These layers aggregate the local features generated by the two local feature extraction branches with the semantic and instance information of the point cloud to obtain the final point cloud global descriptor.

[0070] Step 3.2: Clustering yields candidate location clusters.

[0071] For the candidate locations obtained in step 2, the density-based noisy application (DBSCAN) clustering method is used to cluster them, resulting in candidate location clusters. Specifically, in this embodiment, a total of 3 candidate location clusters are obtained after clustering. The smallest spherical region formed by several representative points in the candidate location cluster is used as the center position of the candidate location cluster, and this center position is used as the representative position of the corresponding candidate location cluster to reduce the computational cost of generating the global descriptor. The number of midpoints at the center position is set to 10 of the total number of midpoints in the candidate locations. -2 ~10 -3 In terms of magnitude, specifically in this embodiment, the number of midpoints at the center position is set to 10 of the total number of midpoints at the candidate positions. -2 Magnitude.

[0072] Step 3.3: Generate global descriptors for candidate location clusters.

[0073] LiDAR scanning point clouds are generated using Gazebo Velodyne simulation at representative locations of each candidate location cluster, and the point-based convolutional branch and voxel-based convolutional branch in the point cloud global descriptor encoder network are input simultaneously.

[0074] After the generated LiDAR scan point cloud is input via a voxel-based convolutional branch, the preprocessing layer first transforms the LiDAR scan point cloud into a coordinate system with the point cloud centroid as the origin; the normalization layer then normalizes it to eliminate the influence of point cloud scale differences. For any point cloud... One point Its normalized point cloud can be represented as:

[0075]

[0076] Sparse 3D convolutional layers transform normalized point clouds into regular voxel meshes. Through sparse 3D convolution, the network captures the geometric and semantic features of the global scene structure of the point cloud. For each voxel, its feature is the mean of the features of all points contained in the voxel mesh, and the final feature value map of its c-th channel is obtained. Represented as:

[0077]

[0078]

[0079]

[0080]

[0081] Through sparse convolution, the point cloud global descriptor encoder network captures the geometric and semantic features of the global scene structure of the point cloud.

[0082] Finally, the upsampling layer performs scale recovery on the features output by the sparse 3D convolutional layer through transposed convolution and skip connections, recovering the geometric and semantic features of the point cloud point by point, and finally outputting a feature pyramid. Each layer of the pyramid contains a feature map that saves the contextual information of the point cloud, including global semantic layout information, low-resolution geometric details, etc.

[0083] The generated LiDAR scanning point cloud is input into a point-based convolutional branch. The input layer serves as the entry point for the LiDAR scanning point cloud, receiving the original point cloud. Then, a nonlinear transformation is performed by the hidden layers. Specifically, in the shallow MLP layer, feature upscaling and basic mapping are performed, learning geometric patterns such as the normal vector direction and local curvature of the points from the original 3D coordinates. The deep MLP layer combines corner points and planar structures from the point cloud edge information to obtain the structural features of the point cloud. Finally, high-dimensional feature mapping is performed through the output layers, resulting in a set of high-dimensional feature mapping matrices corresponding to the input point cloud as the final output. Each row in the matrix represents the high-resolution, fine-grained local geometric features of a point in the point cloud. The detailed information in the original LiDAR point cloud is modeled in the point-based convolutional branch, and the multilayer perceptron is directly applied to each point in the point cloud to extract the detailed features of individual points. This point-based feature is high-resolution and complements the coarse-grained information based on voxels.

[0084] Feature fusion is achieved by inputting the feature matrix output from the point-based convolutional branch and the feature pyramid output from the voxel-based convolutional branch into a fusion layer of the global descriptor generator. Since the output formats of the two branches differ significantly, a U-Net-based encoder-decoder structure is used for fusion. First, trilinear interpolation maps the feature pyramid back to point features, generating a feature matrix that preserves voxel context information. This matrix is ​​then concatenated with the feature matrix output from the point-based convolutional branch to obtain a fused feature matrix. Finally, a full aggregation layer, NetVLAD, aggregates all the fused feature matrices into a global feature description vector, which is the global descriptor of the candidate location cluster.

[0085] Step 3.4: Perform fine-grained localization based on global descriptor similarity.

[0086] use Norm comparison is used to assess the similarity between the global descriptors of each candidate location cluster and the global descriptors of the LiDAR scanned point cloud. This norm represents the distance between the two global descriptors in the feature space; the smaller the distance, the higher the similarity between the two descriptors.

[0087]

[0088] Set global descriptor similarity threshold Similarity less than Candidate location clusters are retained, considered to match the current scan, and these candidate location clusters become the robot's final localization locations. The aforementioned similarity threshold... The range is set to 0-0.55. Specifically, in this embodiment, a similarity threshold is set. =0.15.

[0089] Figure 3 The key process of localization in this embodiment is shown. (a) represents the candidate positions after initialization. The blue particles are all possible candidate positions, and the red particles represent the target area of ​​the ground robot for this localization. (b) to (e) show the key iterative process of coarse localization in one localization. Each iteration selects candidate areas that meet the requirements more. Figure (f) shows the result of fine localization of the candidate positions obtained after coarse localization, which is also the final localization result of this embodiment. In the candidate position cluster represented by the blue particles, all candidate positions are distributed in a small range around the target area of ​​this localization, proving that the localization is successful.

[0090] Multiple verifications have shown that the coarse positioning process of the method of the present invention has an actual iteration count of about 5 times and a positioning time of 3 to 5 seconds, which basically meets the positioning performance requirements of ground robots in large scene maps and can be put into practical use.

[0091] The above description represents a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A point cloud global localization method that integrates semantic information and geometric features, characterized in that, include: S1, Based on the semantic information of the prior map point cloud, filter and initialize candidate locations; S2, construct a semantic topology graph of the prior map point cloud and the lidar scan point cloud, calculate the topological similarity between the candidate location and the lidar scan point cloud, and coarsely localize the ground robot to several regions with high topological similarity. S3. Construct a point cloud global descriptor encoder network to extract point cloud global descriptors at candidate locations after coarse localization and calculate their similarity with the global descriptors of the point cloud scanned by the LiDAR to complete fine localization.

2. The point cloud global localization method fusing semantic information and geometric features according to claim 1, characterized in that, S1 specifically involves: acquiring a priori map point cloud and a LiDAR scan point cloud, both of which have semantic tags. The LiDAR scan point cloud also includes the position information of points in the point cloud, including three-dimensional coordinates and instance tags; in the priori map point cloud, filtering out areas with the semantic tag "ground"; and uniformly initializing candidate positions at certain intervals within the filtered areas.

3. The point cloud global localization method fusing semantic information and geometric features according to claim 2, characterized in that, The interval is 3.0-3.5m.

4. The point cloud global localization method integrating semantic information and geometric features according to claim 1, characterized in that, Specifically, S2 is: S2.1 Construct semantic topology graphs of the prior map point cloud and the lidar scan point cloud respectively to save semantic topology information; For instance objects in the LiDAR scan point cloud The original point cloud is represented as semantic nodes, whose semantic topological information includes the geometric location center of the point cloud. and its semantic category labels ,in, =1,2,...,n, where n is the number of instances in the LiDAR scanned point cloud. The semantic topology of the LiDAR scanned point cloud is then represented as: ={ ... ,... }; For any candidate location instance in the prior map point cloud It also represents its original point cloud as semantic nodes, whose semantic topological information includes the geometric location center of the point cloud. Semantic category tags Distribution range ,in, =1,2,...,m, where m is the number of instances contained in the corresponding candidate position. and Let represent the minimum edge radius and the maximum edge radius of the instance, respectively. Then, the semantic topological graph of any candidate location in the prior map point cloud is represented as follows: ={ ... ,..., }; S2.2, Calculate the topological similarity between the candidate locations and the lidar scan point cloud; if lie in The radius of the vicinity is If the two semantic nodes are within the ring-shaped area, then they are considered to have a matching relationship. Compare each candidate's position one by one All semantic nodes in The matching relationship of each semantic node in the text is determined, and each candidate position is calculated. The topological similarity between each candidate location and the LiDAR scanned point cloud is obtained by calculating the proportion of semantic nodes with matching relationships to the total number of semantic nodes. S2.3, Set the semantic topological similarity threshold Candidate positions with a topological similarity greater than the set threshold are selected as coarsely located candidate positions.

5. The point cloud global localization method fusing semantic information and geometric features according to claim 4, characterized in that, In S2.2, the formula for calculating topological similarity M is: , =1,2,...,n, =1,2,...,m in, ,express and The spacing, (·) indicates the number of elements in the set.

6. The point cloud global localization method fusing semantic information and geometric features according to claim 4, characterized in that, In S2.3, the topological similarity threshold ranges from 0.5 to 0.

8.

7. The point cloud global localization method fusing semantic information and geometric features according to claim 1, characterized in that, Specifically, S3 is: S3.1, Construct the point cloud global descriptor encoder network; The point cloud global descriptor encoder network comprises two parallel local feature extraction branches and a global descriptor generator. Specifically, the two local feature extraction branches are a point-based convolutional branch and a voxel-based convolutional branch, which are used to extract fine-grained features and macroscopic aggregate structure features of the point cloud, respectively. The global descriptor encoder is used to aggregate the local features generated by the two local feature extraction branches with the semantic and instance information of the point cloud to obtain the final point cloud global descriptor. S3.2, Generate global descriptors for candidate positions; At each candidate location, a LiDAR scanning point cloud is simulated and generated. The point-based convolutional branch and the voxel-based convolutional branch in the point cloud global descriptor encoder network are input simultaneously. The feature matrix and feature pyramid are output respectively. The two are input into the fusion layer of the global descriptor generator, and finally the global descriptor of the candidate location is output. S3.3, performs fine-grained localization based on global descriptor similarity; use Norm comparison of the similarity between the global descriptors of each candidate location and the global descriptors of the LiDAR scanned point cloud. Set a global descriptor similarity threshold. ,reserve Less than The candidate position is the final positioning position of the robot.

8. The point cloud global localization method fusing semantic information and geometric features according to claim 7, characterized in that, Specifically, S3.2 is as follows: After the generated LiDAR scan point cloud is input via a voxel-based convolutional branch, the preprocessing layer first transforms the LiDAR scan point cloud into a coordinate system with the point cloud centroid as the origin; the normalization layer then normalizes it for any point cloud. One point Its normalized point cloud can be represented as: in, Represents the normalized point. Point cloud The center of mass, Point cloud The point furthest from the centroid, where i = 1, 2, ..., h, and h represents the point cloud. The number of midpoints; Sparse 3D convolutional layers transform normalized point clouds into regular voxel meshes. For each voxel, the final eigenvalue map of its c-th channel. Represented as: Where c is the channel value in the eigenvalue four-dimensional tensor. Indicates voxel resolution; [·] is an indicator function, when the point... The value is 1 if it belongs to the voxel grid (u, v, w), and 0 otherwise. Point The feature information of the c-th channel; The number of points contained within the voxel grid (u, v, w); , , ) represents a point voxel index coordinates; The floor function discretizes continuous coordinates to obtain integer voxel indices; , , ) represents a point Normalized coordinates; Finally, the upsampling layer restores the scale of the features output by the sparse 3D convolutional layer through transposed convolution and skip connections, and finally outputs a feature pyramid. The generated LiDAR scanning point cloud is input into the input layer of the point-based convolutional branch, then undergoes nonlinear transformation by the hidden layer, and high-dimensional feature mapping is performed through the output layer to obtain a set of high-dimensional feature mapping matrices corresponding to the input point cloud as the final output. The feature matrix output from the point-based convolutional branch and the feature pyramid output from the voxel-based convolutional branch are input into the fusion layer of the global descriptor generator. The fusion layer adopts an encoder-decoder structure based on U-Net to obtain a fused feature matrix, which is then passed through a full aggregation layer to obtain the global descriptor for the candidate positions.

9. The point cloud global localization method fusing semantic information and geometric features according to claim 7, characterized in that, Similarity threshold The range is 0-0.

55.

10. The point cloud global localization method fusing semantic information and geometric features according to claim 1, characterized in that, In step S3, a density-based, noisy clustering method is used to cluster the coarsely located candidate locations obtained in step S2, resulting in candidate location clusters. The smallest spherical region formed by several representative points in each candidate location cluster is used as the center location of the candidate location cluster, and thus serves as the representative location of the corresponding candidate location cluster. The number of midpoints at the center location is set to 10 times the total number of midpoints in the candidate locations. -2 ~10 -3 The system utilizes a point cloud global descriptor encoder network to generate point cloud global descriptors representing the locations of each candidate location cluster, and calculates their similarity to the global descriptors of the LiDAR-scanned point cloud to achieve fine localization.

Images