A global positioning method based on multi-layer 3D point cloud bird's-eye view

Abstract

This invention discloses a global localization method based on multi-layer 3D point cloud bird's-eye view, belonging to the field of mobile robot localization and navigation technology. The method first divides the LiDAR point cloud into multiple overlapping slices along the height direction and projects them to generate a multi-layer bird's-eye view. Then, the multi-layer bird's-eye view is input into a global localization network containing a shared feature extraction module, a vertical slice attention module, a slice feature fusion module, and a NetVLAD aggregation module to extract global and local feature descriptors. Finally, a reference frame is obtained through global feature retrieval, and the pose of the query frame is calculated by combining keypoint matching and random sampling consensus algorithms. This method can preserve the vertical structure information of the point cloud, reduce localization ambiguity in complex scenes, and balance localization accuracy and computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of mobile robot positioning and navigation technology, and specifically relates to a global positioning method based on multi-layer 3D point cloud bird's-eye view. Background Technology

[0002] With the development of autonomous driving technology, global positioning, as the foundation of autonomous navigation systems, is of paramount importance in terms of accuracy. LiDAR, due to its ability to provide precise geometric information and its robustness to changes in lighting conditions, is widely used in global positioning.

[0003] Currently, global localization methods based on LiDAR are mainly divided into methods that directly process 3D point clouds and projection-based methods. In their paper "3D LiDAR-Based Global Localization Using Siamese Neural Network," authors Huan Yin et al. used point cloud feature learning based on Siamese networks for 3D global localization. However, because this method directly extracts and matches features in the original 3D point cloud space, its handling of the disorder and sparsity of point cloud data still relies heavily on manual design or dimensionality reduction representation. In large-scale or dynamic scenarios, the matching efficiency is low and the computational complexity is high, making it difficult to meet the requirements of real-time localization.

[0004] Existing projection-based methods (such as projecting point clouds into bird's-eye view (BEV) images) utilize 2D convolutional neural networks, resulting in high computational efficiency. In their paper "BEVPlace: Learning LiDAR-based Place Recognition using Bird's Eye View Images," authors Lun Luo et al. employed a single-layer BEV projection combined with convolutional networks and global descriptors for LiDAR-based location retrieval. However, their single-plane orthogonal projection compresses Z-axis height information, leading to a loss of vertical geometric details. In complex vertical structures (such as overpasses or tree canopy occlusion), this can cause feature aliasing, reducing positioning accuracy and robustness.

[0005] Therefore, how to effectively utilize vertical structure information to resolve positioning ambiguities in complex scenarios while maintaining the computational efficiency of projection methods is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] This invention proposes a global localization method based on multi-layer 3D point cloud bird's-eye view. Addressing the problem of lost vertical geometric details due to compression of Z-axis height information in existing single-layer bird's-eye view projection methods, this invention preserves vertical structural information through a multi-layer slicing strategy and designs a rotational vertical attention network to explicitly model semantic dependencies between slices. This generates local feature descriptors with rotation equivariance and global feature descriptors with rotation invariance for global localization. This invention significantly improves the robustness and accuracy of localization while maintaining real-time performance.

[0007] The technical solution for implementing this invention is: a global positioning method based on multi-layer 3D point cloud bird's-eye view, comprising the following steps:

[0008] Step 1: Download the KITTI odometry benchmark dataset. This dataset contains continuous 3D point cloud data acquired by an onboard 64-line LiDAR, covering urban roads, rural roads, and highways. Simultaneously, this dataset synchronously records the precise 6-DOF pose of each frame of 3D point cloud data in the global coordinate system using an onboard high-precision RTK-GPS and inertial navigation system (INS), serving as the true pose label for subsequent global localization model training and accuracy evaluation. Sequence 00 from this dataset is selected as the training set, and sequences 02, 05, 06, and 08 are selected as the test set for model training and testing evaluation, respectively. Proceed to Step 2.

[0009] Step 2: Preprocess the 3D point cloud data in the training and testing sets to generate multi-layer bird's-eye view. Considering that traditional bird's-eye view projection directly compresses the Z-axis, resulting in the loss of vertical geometric details, this invention adopts a multi-layer slicing strategy. The 3D point cloud data in the training and testing sets are divided into multi-layer slices with a certain overlap rate along the height direction. Each slice is discretized using the normalized point density formula and projected into a multi-layer bird's-eye view, which serves as the input data for subsequent network training and testing. Proceed to Step 3.

[0010] Step 3: Construct a global localization network based on multi-layer slice feature fusion and vertical attention. The global localization network includes a shared feature extraction module, a vertical slice attention module, a slice feature fusion module, and a NetVLAD aggregation module. Proceed to Step 4.

[0011] Step 4: Using the multi-layer bird's-eye view of the training set obtained in Step 2, and the real pose labels provided in Step 1, train the global localization model. The specific training process is as follows:

[0012] Step 4-1: Input the multi-layer bird's-eye view images from the training set into the shared feature extraction module. Considering that the vehicle's orientation changes arbitrarily during driving, a discrete rotation group sampling strategy is introduced to make the model possess rotational equivariance. First, perform multi-angle rigid body rotation transformations on the multi-layer bird's-eye view images to generate multi-layer bird's-eye view images with different rotation angles. Then, using a parameter-shared convolutional neural network, feature extraction is performed independently on each layer of the bird's-eye view image under all rotation angles, outputting the multi-layer deep semantic features of the multi-layer bird's-eye view image under different rotation angles, and proceeding to Step 4-2.

[0013] Step 4-2: Input the multi-layer deep semantic features into the vertical slice attention module. At the same rotation angle, stack the deep semantic features of each layer according to the slice dimension to form a multi-layer feature tensor, and rearrange them according to spatial position into a vertical feature sequence. After injecting learnable vertical position encoding into the vertical feature sequence, use the Transformer encoder to perform context modeling on the multi-layer slice features at each spatial position to capture the semantic dependencies between layers of different heights, outputting the enhanced multi-layer deep semantic features through vertical interaction, i.e., the enhanced multi-layer deep semantic features, and proceed to step 4-3.

[0014] Step 4-3: Input the enhanced multi-layer deep semantic features into the slice feature fusion module. First, at each rotation angle, the enhanced multi-layer deep semantic features are weighted and fused using learnable weights to obtain single-layer fused features at different rotation angles. Then, inverse rotation transformation is performed on the single-layer fused features corresponding to each rotation angle to align their space and restore them to the original coordinate system. Finally, max pooling is applied to the features at each rotation angle after they have been restored to the original coordinate system, aggregating the feature information and outputting local feature descriptors with rotational equivariance, then proceeding to step 4-4.

[0015] Step 4-4: Input the local feature descriptors into the NetVLAD aggregation module to generate rotation-invariant global feature descriptors, while retaining the local feature descriptors for subsequent refined pose calculations, and proceed to step 4-5.

[0016] Steps 4-5: At the beginning of each training round, input the multi-layer bird's-eye view of the training set obtained in step 2 into the global localization model, extract the corresponding global feature descriptors, and store the global feature descriptors in the training set feature cache pool, then proceed to step 4-6.

[0017] Steps 4-6: Mine and construct training triplet data containing hard samples for the current training round, as follows:

[0018] Step 4-6-1: Traverse the multi-layer bird's-eye view of the training set, select a frame of multi-layer bird's-eye view as anchor data, and proceed to step 4-6-2.

[0019] Step 4-6-2: Using the real pose labels provided by the training set, calculate and filter out multi-layer bird's-eye view images whose real physical distance from the anchor point data is within the set positive sample threshold as positive sample candidate sets, and multi-layer bird's-eye view images whose real physical distance is outside the set negative sample threshold as negative sample candidate sets, and proceed to step 4-6-3.

[0020] In steps 4-6-3 and 4-5, the global feature descriptors corresponding to the anchor data, positive sample candidate set, and negative sample candidate set are retrieved from the training set feature cache pool. The Euclidean distance between the anchor data and the positive and negative sample candidate sets in the feature space is calculated. The multi-layer bird's-eye view with the greatest feature distance in the positive sample candidate set is selected as the difficult positive sample data, and the multi-layer bird's-eye view with the closest feature distance in the negative sample candidate set is selected as the difficult negative sample data. This constructs a training triplet consisting of anchor data, difficult positive sample data, and difficult negative sample data. Proceed to step 4-7.

[0021] Steps 4-7: Input the anchor data, hard positive sample data, and hard negative sample data from the training triplet data into the global localization network, and re-extract the global feature descriptors of the anchor points, hard positive samples, and hard negative samples under the current network weights, then proceed to step 4-8.

[0022] Steps 4-8: Calculate the triplet loss using the three global feature descriptors described in Step 4-7. Execute the backpropagation algorithm based on this triplet loss to iteratively update the learnable parameters in the global localization network. Repeat steps 4-5 to 4-8 until the triplet loss converges or the maximum number of training rounds is reached, ultimately obtaining the trained global localization model. Proceed to Step 5.

[0023] Step 5: Using the trained global localization model, execute a two-stage localization strategy on the multi-layer bird's-eye view of the test set obtained in Step 2, and output the accurate global pose of the test set query frame in the global coordinate system to evaluate the model accuracy, as follows:

[0024] Step 5-1: Input the multi-layer bird's-eye view of the test set into the trained global localization model, extract the corresponding global feature descriptors and local feature descriptors, and bind the extracted global feature descriptors and local feature descriptors with the real pose labels obtained in Step 1 to construct the map database of the test scene, and then proceed to Step 5-2.

[0025] Step 5-2: Select a multi-layer bird's-eye view frame from the test set as the query frame. Extract the local and global feature descriptors of the query frame using the trained global localization model. Then, perform a similarity search between the global feature descriptors of the query frame and the global feature descriptors in the map database. Based on the Euclidean distance, select the reference frame that is most similar to the query frame. Then, obtain the true pose of the reference frame in the global coordinate system, the local feature descriptor of the reference frame, and the multi-layer bird's-eye view of the reference frame. Proceed to Step 5-3.

[0026] Step 5-3: Apply the FAST operator to extract key points on the multi-layer bird's-eye view of the query frame and the reference frame respectively, and obtain the feature values ​​of the key points at the corresponding points of the local feature descriptors through bilinear interpolation. Then, based on the extracted feature values, perform feature point matching between the key points of the query frame and the key points of the reference frame, and use the Random Sampling Consensus Algorithm (RANSAC) to remove mismatched point pairs, and then calculate the rigid body transformation matrix of the query frame relative to the reference frame, and proceed to step 5-4.

[0027] Step 5-4: Combining the true pose of the reference frame in the global coordinate system obtained in Step 5-2 and the rigid body transformation matrix obtained in Step 5-3, the precise pose of the query frame in the global coordinate system is calculated by matrix multiplication, thus completing high-precision global positioning.

[0028] Compared with the prior art, the significant advantages of this invention are:

[0029] 1) Since traditional bird's-eye view projection directly compresses the Z-axis, resulting in the loss of vertical information, this invention proposes a multi-layer slicing strategy that preserves the structural information of the environment in the vertical direction, effectively distinguishes complex vertical scenes such as overpasses and suspended vegetation, and significantly reduces positioning ambiguity caused by height information compression.

[0030] 2) Since the vehicle's orientation changes arbitrarily during driving, this invention introduces a sampling and inverse rotation alignment strategy based on discrete rotation groups to construct a rotation consistency network. The generated features have rotational equivariance, enabling the model to adapt to the positioning requirements of different vehicle orientations and improving the robustness of the positioning system.

[0031] 3) This invention innovatively combines the vertical attention mechanism with slice weighted fusion, explicitly models the semantic dependencies between slices, and automatically filters the height layer information that contributes most to localization through weighted fusion, achieving high-precision localization while maintaining the high-efficiency computational advantages of 2D convolutional architecture. Attached Figure Description

[0032] Figure 1 This is a flowchart of a global positioning method based on a multi-layer 3D point cloud bird's-eye view according to the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0034] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0035] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly and specifically defined.

[0036] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixing," etc., should be interpreted broadly. For example, "fixing" can mean a fixed connection, a detachable connection, or an integral part; "connection" can mean a mechanical connection or an electrical connection. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0037] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible to those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0038] The following section will further introduce the specific implementation method, as well as the technical difficulties and inventive points of this invention, using this design example as an example.

[0039] This invention provides a global localization method based on multi-layer 3D point cloud bird's-eye view, aiming to solve the problems of vertical information loss and localization ambiguity in complex scenes caused by commonly used single-layer bird's-eye view projection methods. By using a multi-layer slicing strategy and rotational vertical attention modeling, the accuracy and robustness of global localization are improved. The embodiments of this invention are described in further detail below.

[0040] Combination Figure 1The global positioning method based on multi-layer 3D point cloud bird's-eye view, as described in this invention, includes the following steps:

[0041] Step 1: Download the KITTI odometry benchmark dataset. This dataset contains continuous 3D point cloud data acquired by an onboard 64-line LiDAR, covering urban roads, rural roads, and highways. Simultaneously, this dataset synchronously records the precise 6-DOF pose of each frame of 3D point cloud data in the global coordinate system using an onboard high-precision RTK-GPS and inertial navigation system (INS), serving as the true pose label for subsequent global localization model training and accuracy evaluation. Sequence 00 from this dataset is selected as the training set, and sequences 02, 05, 06, and 08 are selected as the test set for model training and testing evaluation, respectively. Proceed to Step 2.

[0042] Step 2: Preprocess the 3D point cloud data from the training and test sets to generate multi-layer bird's-eye view images. First, crop the vertical extent of the point cloud. for:

[0043] ,

[0044] in, Indicates the minimum height of the vertical range. Indicates the highest height of the vertical range.

[0045] Next, the area along The axis is divided into A slice sequence with a certain overlap rate For the first The formula for calculating the height range of a slice is as follows:

[0046] ,

[0047] ,

[0048] in, Indicates the first The lower bound of the height of a slice Indicates the first The upper bound of the height of a slice Indicates the base thickness of the slice. This represents the overlap margin introduced to counteract vehicle bumps. In this embodiment, , , , , .

[0049] Then, each slice Discretize into a grid size of Aerial view , Indicates the height of the bird's-eye view. Describe the width of the bird's-eye view. It represents the set of real numbers.

[0050] Bird's-eye view grid The pixel value at that location is calculated using the normalized point density formula:

[0051] ,

[0052] in, This represents the normalized pixel value. Indicates falling into the first A slice and located in the grid The number of points inside, This indicates the maximum number of points contained in a single grid within the slice. In this embodiment, A horizontal area is captured with the center of the sensor as the origin. The generated size is The pixel-level bird's-eye view is output as a multi-layered bird's-eye view containing rich height information, and then proceeds to step 3.

[0053] Step 3: Construct a global localization network based on multi-layer slice feature fusion and vertical attention. The global localization network includes a shared feature extraction module, a vertical slice attention module, a slice feature fusion module, and a NetVLAD aggregation module. Proceed to Step 4.

[0054] Step 4: Using the multi-layer bird's-eye view of the training set obtained in Step 2, and the real pose labels provided in Step 1, train the global localization network to obtain the trained global localization model. The specific training process is as follows:

[0055] Step 4-1: Input the multi-layer bird's-eye view images from the training set into the shared feature extraction module. Considering that the vehicle's orientation changes arbitrarily during driving, a discrete rotation group sampling strategy is introduced to enable the network to have rotational equivariance. First, perform multi-angle rigid body rotation transformations on the multi-layer bird's-eye view images to generate multi-layer bird's-eye view images with different rotation angles. A discrete rotation group is predefined. The formula is as follows:

[0056] ,

[0057] in, Indicates the first A discrete rotation angle This represents the size of the rotation group. In this embodiment, This involves rotating and sampling the input sequence every 45°. Then, using a parameter-shared convolutional neural network, features are independently extracted from each layer of the bird's-eye view at all rotation angles, outputting multi-layer deep semantic features of the multi-layer bird's-eye view at different rotation angles.

[0058] ,

[0059] in, Indicates the first Layer slices at rotation angle The deep semantic features below Representing feature dimension, Indicates the feature map height. Indicates the width of the feature map. This represents the feature extraction operation in a convolutional neural network. This represents an image rotation transformation operation. Indicates the first Aerial view of the first floor.

[0060] In this embodiment, the convolutional neural network uses the first two layers of ResNet-34, and its first-layer convolutional kernel is adapted to single-channel input, in conjunction with step 2. The dimension is ,get Dimensions Proceed to step 4-2.

[0061] Step 4-2: Input the multi-layer deep semantic features into the vertical slice attention module. Considering the physical order of the bird's-eye view layers and the vertical dependence of the environmental structure, at the same rotation angle... Next, we will first analyze the multi-layer deep semantic features. Stacking slices by dimension yields a multi-layer deep semantic feature tensor. , The formula is as follows:

[0062] ,

[0063] in, This indicates a stacking operation.

[0064] To model the semantic dependencies between slices in the high-dimensional dimension, the multi-layer deep semantic feature tensor is... The sequence is rearranged according to spatial location. Specifically, for each spatial coordinate... Take this location The feature vectors corresponding to each slice are used to form a vertical feature sequence, and the vertical feature sequences corresponding to all spatial locations are rearranged:

[0065] ,

[0066] in, The first dimension corresponds to the spatial location index, the second dimension corresponds to the slice dimension, and the third dimension corresponds to the feature dimension. Thus, the multi-head self-attention mechanism independently acts on the vertical feature sequences under each spatial coordinate to characterize the contextual relationship between different height layers at the same spatial location.

[0067] To preserve the order information of each slice in the height direction, a learnable vertical position code shared across all spatial grids is introduced:

[0068] ,

[0069] Since the physical height range corresponding to each slice remains consistent across different spatial grids, the vertical position encoding... Shared across all spatial locations, and with the rearranged feature sequence. By adding each element, we obtain the sequence after fusing the positional information:

[0070] ,

[0071] Subsequently, the sequence after fusing the location information The Transformer encoder is used for context modeling to obtain the output sequence enhanced with inter-layer semantic interaction:

[0072] ,

[0073] Finally, output the sequence Restructured according to the original spatial layout into an enhanced multi-layer deep semantic feature tensor , :

[0074] ,

[0075] in, Indicates the rotation angle Below, multi-layer deep semantic features are enhanced by the vertical slice attention module; Indicates the enhanced first Layer slice features. Through the above method, the network can explicitly model the semantic dependencies between slices of different heights at each spatial location, thereby improving the feature representation ability in complex vertical structure scenes, and then proceed to step 4-3.

[0076] Step 4-3: Input the enhanced multi-layer deep semantic features into the slice feature fusion module. First, at each rotation angle, the enhanced multi-layer deep semantic features are weighted and fused using learnable weights to obtain single-layer fused features at different rotation angles. Specifically, a weight vector is first learned, and normalized weights are calculated using Softmax, as shown in the following formula:

[0077] ,

[0078] in, Indicates the first Normalized weights of layer slices, Indicates the first Learnable weight parameters for layer slices, Indicates the first Learnable weight parameters for layer slices. Then... Enhanced features of each slice Perform a weighted summation to obtain the result at the rotation angle. Single-layer fusion features , The formula is as follows:

[0079] ,

[0080] Subsequently, to eliminate the spatial misalignment caused by rotation sampling, inverse rotation transformations were performed on the single-layer fused features corresponding to each rotation angle to spatially align them and restore them to the original coordinate system, as shown in the following formula:

[0081] ,

[0082] in, This represents the feature after alignment to the original coordinate system. This indicates the inverse rotation transformation.

[0083] Finally, the rotation angle features after restoring to the original coordinate system are analyzed. Max pooling is applied to select the best feature response for each spatial location under all rotation angles, and the aggregated feature information outputs a single-layer local feature descriptor with rotational equivariance. , :

[0084] ,

[0085] Where G represents the discrete rotation group.

[0086] Proceed to step 4-4.

[0087] Step 4-4: Input the local feature descriptors into the NetVLAD aggregation module to generate rotation-invariant global feature descriptors. In this embodiment, the number of cluster centers in the NetVLAD aggregation module is set to 64, and the output global feature descriptor has a dimension of 8192, while retaining the local feature descriptors. Used for subsequent refined pose calculation, proceed to steps 4-5.

[0088] Steps 4-5: At the beginning of each training round, input the multi-layer bird's-eye view of the training set obtained in step 2 into the global localization network, extract the corresponding global feature descriptors, and store the global feature descriptors in the training set feature cache pool, then proceed to step 4-6.

[0089] Steps 4-6: Mine and construct training triplet data containing hard samples for the current training round, as follows:

[0090] Step 4-6-1: Traverse the multi-layer bird's-eye view of the training set, select a frame of multi-layer bird's-eye view as anchor data, and proceed to step 4-6-2.

[0091] Step 4-6-2: Using the real pose labels provided by the training set, calculate and filter multi-layer bird's-eye view images whose real physical distance from the anchor point data is within the set positive sample threshold as positive sample candidate sets; and multi-layer bird's-eye view images whose real physical distance is outside the set negative sample threshold as negative sample candidate sets. In this embodiment, the positive sample threshold is set to 5m and the negative sample threshold is set to 7m. Proceed to step 4-6-3.

[0092] In steps 4-6-3 and 4-5, the global feature descriptors corresponding to the anchor data, positive sample candidate set, and negative sample candidate set are retrieved from the training set feature cache pool. The Euclidean distance between the anchor data and the positive and negative sample candidate sets in the feature space is calculated. The multi-layer bird's-eye view with the greatest feature distance in the positive sample candidate set is selected as the difficult positive sample data, and the multi-layer bird's-eye view with the closest feature distance in the negative sample candidate set is selected as the difficult negative sample data. This constructs a training triplet consisting of anchor data, difficult positive sample data, and difficult negative sample data. Proceed to step 4-7.

[0093] Steps 4-7: Input the anchor data, hard positive sample data, and hard negative sample data from the training triplet data into the global localization network, and re-extract the global feature descriptors of the anchor points, hard positive samples, and hard negative samples under the current network weights, then proceed to step 4-8.

[0094] Step 4-8: Calculate the triplet loss using the three global feature descriptors described in Step 4-7. The purpose is to shorten the distance between the anchor point and positive samples in the feature space, while simultaneously increasing the distance between the anchor point and negative samples. The specific formula for calculating the triplet loss is as follows:

[0095] ,

[0096] in, Indicates the loss of the triplet. This represents the global feature descriptor of the anchor point. Represents the global feature descriptor for difficult-to-positive samples. Represents the global feature descriptor for hard-to-bear samples. Represents the Euclidean distance in the feature space. This indicates the set boundary threshold.

[0097] Based on the triplet loss, the backpropagation algorithm is executed to iteratively update the learnable parameters in the global localization network. The training process from steps 4-5 to 4-8 is repeated until the triplet loss converges or the set maximum number of training rounds is reached, finally obtaining the trained global localization model, and then proceeding to step 5.

[0098] Step 5: Using the trained global localization model, execute a two-stage localization strategy on the multi-layer bird's-eye view of the test set obtained in Step 2, and output the accurate global pose of the test set query frame in the global coordinate system to evaluate the model accuracy, as follows:

[0099] Step 5-1: Input the multi-layer bird's-eye view of the test set into the trained global localization model, extract the corresponding global feature descriptors and local feature descriptors, and bind the extracted global feature descriptors and local feature descriptors with the real pose labels obtained in Step 1 to construct the map database of the test scene, and then proceed to Step 5-2.

[0100] Step 5-2: Select a single frame from the multi-layer bird's-eye view in the test set as the query frame. Extract the local and global feature descriptors of the query frame using the trained global localization model. Then, perform a similarity search between the global feature descriptors of the query frame and the global feature descriptors in the pre-built map database. Based on Euclidean distance, select the reference frame most similar to the query frame, and then obtain the prior true pose of the reference frame in the global coordinate system. 1. Local feature descriptors of the reference frame and multi-layer bird's-eye view of the reference frame, then proceed to step 5-3.

[0101] Step 5-3: Apply the FAST operator to extract key points on the multi-layer bird's-eye view of the query frame and reference frame respectively, and obtain the feature values ​​of the key points at the corresponding points of the local feature descriptors through bilinear interpolation. Then, based on the extracted feature values, perform feature point matching between the key points of the query frame and the key points of the reference frame, and use the Random Sampling Consensus Algorithm (RANSAC) to remove mismatched point pairs, and calculate the rigid transformation matrix of the query frame relative to the reference frame. Proceed to step 5-4.

[0102] Step 5-4: Based on the prior pose of the reference frame obtained in Step 5-2 The relative rigid body transformation matrix obtained in step 5-3 The precise pose of the query frame in the global coordinate system is calculated using matrix multiplication. :

[0103] ,

[0104] in, This represents the pose of the reference frame in the global coordinate system. This represents the rigid body transformation matrix of the query frame relative to the reference frame. It represents the precise pose of the query frame in the global coordinate system, ultimately achieving high-precision global positioning.

[0105] The global localization method based on multi-layer 3D point cloud bird's-eye view described in this invention is implemented using Python and the PyTorch framework, and uses a single NVIDIA RTX 3090 GPU for model training and inference. During training, the boundary threshold of the triplet loss is... The parameters are set to 0.5, the batch size to 4, the Adam optimizer is used, and the initial learning rate is set to... A step decay strategy was adopted, and the number of training rounds was set to 200 to ensure model convergence.

[0106] To demonstrate the superior performance of this invention, the experiments focused on evaluating the location identification capability, pose estimation accuracy, and operational efficiency of the method.

[0107] Location identification is the primary step in global localization (GLAN) tasks, and its core lies in accurately retrieving historical frames from the database that are most similar to the current scene. This invention selects CVTNet based on distance images, LoGG3D-Net based on raw point clouds, RING++ based on bird's-eye view representation, and the end-to-end network LCDNet as comparison methods, and uses the Recall@1 (R@1) metric for quantitative evaluation.

[0108] As shown in Table 1, the present invention achieved the best results in all test sequences, with an average recall rate of 98.8%.

[0109]

[0110] Regarding pose estimation, since CVTNet and LoGG3D-Net lack pose regression capabilities, this invention only compares the geometric pose calculation accuracy with RING++ and LCDNet. As shown in Table 2, the average translation error of this method is as low as 0.20m, and the average rotation error is only 0.28°, which is significantly better than the baseline methods.

[0111]

[0112] Regarding operational efficiency, see Table 3.

[0113]

[0114] This invention achieves an optimal balance between efficiency and functionality. LCDNet relies on computationally intensive 3D voxel convolutions, taking over 500ms; RING++ uses Radon transform for rotation invariance, but frequency domain matching still requires 39.5ms. This method, based on an efficient 2D convolutional architecture, requires only 26.8ms for single-frame inference, 18 times faster than LCDNet. While CVTNet is fast, it lacks pose estimation and robustness. This method achieves a processing frequency of 37Hz while maintaining high positioning accuracy, far exceeding the frame rate of mainstream LiDAR acquisition, and possesses extremely high practical engineering deployment value.

Claims

1. A global positioning method based on multi-layer 3D point cloud bird's-eye view, characterized in that, Includes the following steps: Step 1: Download the KITTI odometry benchmark dataset. The dataset contains continuous 3D point cloud data collected by an onboard 64-line LiDAR, covering urban roads, rural roads, and highways. Simultaneously, the dataset records the precise 6-DOF pose of each frame of 3D point cloud data in the global coordinate system through onboard high-precision RTK-GPS and inertial navigation system INS, which serves as the real pose label for subsequent global positioning model training and accuracy evaluation. Select sequence 00 from the dataset as the training set, and sequences 02, 05, 06, and 08 as the test set, for model training and testing evaluation, respectively. Proceed to step 2. Step 2: Preprocess the 3D point cloud data in the training and test sets to generate multi-layer bird's-eye view; proceed to Step 3. Step 3: Construct a global localization network based on multi-layer slice feature fusion and vertical attention. The global localization network includes a shared feature extraction module, a vertical slice attention module, a slice feature fusion module, and a NetVLAD aggregation module. Proceed to Step 4. Step 4: Using the multi-layer bird's-eye view of the training set obtained in Step 2 and the real pose labels provided in Step 1, train the global localization model to obtain the trained global localization model, and then proceed to Step 5. Step 5: Using the trained global localization model, perform a two-stage localization strategy on the multi-layer bird's-eye view of the test set obtained in Step 2, and output the accurate global pose of the test set query frame in the global coordinate system to evaluate the model accuracy.

2. The global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 1, characterized in that, Step 2 is detailed as follows: A multi-layer slicing strategy is adopted, dividing the 3D point cloud data in the training and test sets into multi-layer slices with a certain overlap rate along the height direction. Each slice is discretized using the normalized point density formula and projected into a multi-layer bird's-eye view, i.e.: Extract the vertical range of the point cloud for: ， in, Indicates the minimum height of the vertical range. Indicates the highest height of the vertical range; Next, the area along The axis is divided into A slice sequence with a certain overlap rate For the first The formula for calculating the height range of a slice is as follows: ， ， in, Indicates the first The lower bound of the height of a slice Indicates the first The upper bound of the height of a slice Indicates the base thickness of the slice. This indicates the overlap margin introduced to resist vehicle bumps. Then, each slice Discretize into a grid size of Aerial view , Indicates the height of the bird's-eye view. Describe the width of the bird's-eye view. Represents the set of real numbers; Bird's-eye view grid The pixel value at that location is calculated using the normalized point density formula: ， in, This represents the normalized pixel value. Indicates falling into the first A slice and located in the grid The number of points inside, This indicates the maximum number of points contained in a single grid within the slice; A horizontal area is cropped with the center of the sensor as the origin to generate a bird's-eye view, and the output is a multi-layered bird's-eye view containing rich height information.

3. The global positioning method based on multi-layer 3D point cloud bird's-eye view according to claim 3, characterized in that, Step 4, as follows: Step 4-1: Input the multi-layer bird's-eye view of the training set into the shared feature extraction module; considering that the vehicle's orientation changes arbitrarily during driving, a discrete rotation group sampling strategy is introduced to make the model have rotational equivariance; first, perform multi-angle rigid body rotation transformation on the multi-layer bird's-eye view to generate multi-layer bird's-eye view at different rotation angles; then, using a parameter-shared convolutional neural network, extract features independently for each layer of bird's-eye view under all rotation angles, output the multi-layer deep semantic features of the multi-layer bird's-eye view under different rotation angles, and proceed to step 4-2; Step 4-2: Input the multi-layer deep semantic features into the vertical slice attention module; under the same rotation angle, stack the deep semantic features of each layer according to the slice dimension to form a multi-layer feature tensor, and rearrange them according to spatial position into a vertical feature sequence; after injecting learnable vertical position encoding into the vertical feature sequence, use the Transformer encoder to perform context modeling on the multi-layer slice features at each spatial position to capture the semantic dependencies between different height layers, and output the multi-layer deep semantic features enhanced by vertical interaction, i.e., the enhanced multi-layer deep semantic features, and proceed to step 4-3; Step 4-3: Input the enhanced multi-layer deep semantic features into the slice feature fusion module. First, at each rotation angle, the enhanced multi-layer deep semantic features are weighted and fused using learnable weights to obtain single-layer fused features at different rotation angles. Then, inverse rotation transformation is performed on the single-layer fused features corresponding to each rotation angle to align their spaces and restore them to the original coordinate system. Finally, max pooling is applied to the features at each rotation angle after they have been restored to the original coordinate system to aggregate feature information and output local feature descriptors with rotation equivariance, then proceed to step 4-4. Step 4-4: Input the local feature descriptors into the NetVLAD aggregation module to generate rotation-invariant global feature descriptors, while retaining the local feature descriptors for subsequent refined pose calculations, and proceed to step 4-5. Steps 4-5: At the beginning of each training round, input the multi-layer bird's-eye view of the training set obtained in step 2 into the global localization network, extract the corresponding global feature descriptors, and store the global feature descriptors in the training set feature cache pool, then proceed to step 4-6. Steps 4-6: Mine and construct training triplet data containing hard samples for the current training round, then proceed to step 4-7; Steps 4-7: Input the anchor data, hard positive sample data and hard negative sample data from the training triplet data into the global localization network, and re-extract the global feature descriptors of the anchor points, hard positive samples and hard negative samples under the current network weights, and then proceed to step 4-8. Steps 4-8: Calculate the triplet loss using the three global feature descriptors obtained in Step 4-7. Execute the backpropagation algorithm based on the triplet loss to iteratively update the learnable parameters in the global localization network. Repeat the training process from Step 4-5 to Step 4-8 until the triplet loss converges or the maximum number of training rounds is reached, finally obtaining the trained global localization model.

4. The global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 3, characterized in that, In step 4-1, the specific details are as follows: First, perform rigid body rotation transformations at multiple angles on the multi-layered bird's-eye view to generate multi-layered bird's-eye view images with different rotation angles; then, predefine a discrete rotation group. The formula is as follows: ， in, Indicates the first A discrete rotation angle Indicates the size of the rotation group; Subsequently, using a parameter-sharing convolutional neural network, features are extracted independently for each layer of the bird's-eye view at all rotation angles, outputting multi-layer deep semantic features of the multi-layer bird's-eye view at different rotation angles: ， in, Indicates the first Layer slices at rotation angle The deep semantic features below Representing feature dimension, Indicates the feature map height. Indicates the width of the feature map. This represents the feature extraction operation in a convolutional neural network. This represents an image rotation transformation operation. Indicates the first Aerial view of the layer.

5. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 4, characterized in that, In step 4-2, the specific details are as follows: Multi-layer deep semantic features are input into the vertical slice attention module. Considering the physical order of the bird's-eye view layers and the vertical dependence of the environmental structure, at the same rotation angle... Next, we will first analyze the multi-layer deep semantic features. Stacking slices by dimension yields a multi-layer deep semantic feature tensor. , The formula is as follows: ， in, Indicates a stacking operation; To model the semantic dependencies between slices in the high-dimensional dimension, multi-layer deep semantic feature tensors are used. Rearrange the sequence according to spatial location; For each spatial coordinate Take this location The feature vectors corresponding to each slice are used to form a vertical feature sequence, and the vertical feature sequences corresponding to all spatial locations are rearranged: ， in, The first dimension corresponds to the spatial location index, the second dimension corresponds to the slice dimension, and the third dimension corresponds to the feature dimension; Thus, the multi-head self-attention mechanism acts independently on the vertical feature sequences under each spatial coordinate to characterize the contextual relationship between different height layers at the same spatial location; To preserve the order information of each slice in the height direction, a learnable vertical position code shared across all spatial grids is introduced. : ， Since the physical height range corresponding to each slice remains consistent across different spatial grids, the vertical position encoding... Shared across all spatial locations, and with the rearranged feature sequence. By adding element by element, we obtain the sequence after fusing positional information. : ， Subsequently, the sequence after fusing the location information The input is a Transformer encoder for context modeling, resulting in an output sequence enhanced with inter-layer semantic interaction. : ， Finally, output the sequence Restructured according to the original spatial layout into an enhanced multi-layer deep semantic feature tensor , : ， in, Indicates the rotation angle Below, multi-layer deep semantic features are enhanced by the vertical slice attention module; Indicates the enhanced first Layer slice features: Through the above method, the network can explicitly model the semantic dependencies between slices of different heights at each spatial location, thereby improving the feature representation ability in complex vertical structure scenarios.

6. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 5, characterized in that, Step 4-3, as follows: First, at each rotation angle, the enhanced multi-layer deep semantic features are weighted and fused using learnable weights to obtain single-layer fused features at different rotation angles. Specifically, a weight vector is first learned, and the normalized weights are calculated using Softmax, as shown in the following formula: ， in, Indicates the first Normalized weights of layer slices, Indicates the first Learnable weight parameters for layer slices, Indicates the first Learnable weight parameters for layer slices; subsequently, for Enhanced features of each slice Perform a weighted summation to obtain the result at the rotation angle. Single-layer fusion features , The formula is as follows: ， Subsequently, to eliminate the spatial misalignment caused by rotation sampling, inverse rotation transformations were performed on the single-layer fused features corresponding to each rotation angle to spatially align them and restore them to the original coordinate system, as shown in the following formula: ， in, This represents the feature after alignment to the original coordinate system. Indicates inverse rotation transformation; Finally, the rotation angle features after restoring to the original coordinate system are analyzed. Max pooling is applied to select the best feature response for each spatial location under all rotation angles, and the aggregated feature information outputs a single-layer local feature descriptor with rotational equivariance. , : ； Where G represents the discrete rotation group.

7. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 6, characterized in that: In steps 4-6, training triplet data containing hard samples is mined and constructed for the current training round, as follows: Step 4-6-1: Traverse the multi-layer bird's-eye view of the training set, select a frame of multi-layer bird's-eye view as anchor data, and proceed to step 4-6-2. Step 4-6-2: Using the real pose labels provided by the training set, calculate and filter out multi-layer bird's-eye view images whose real physical distance from the anchor point data is within the set positive sample threshold as positive sample candidate sets, and multi-layer bird's-eye view images whose real physical distance is outside the set negative sample threshold as negative sample candidate sets, and proceed to step 4-6-3. In steps 4-6-3 and 4-5, the global feature descriptors corresponding to the anchor data, positive sample candidate set, and negative sample candidate set are retrieved from the training set feature cache pool. The Euclidean distance between the anchor data and the positive and negative sample candidate sets in the feature space is calculated. The multi-layer bird's-eye view with the farthest feature distance in the positive sample candidate set is selected as the difficult positive sample data, and the multi-layer bird's-eye view with the closest feature distance in the negative sample candidate set is selected as the difficult negative sample data. Thus, training triplet data composed of anchor data, difficult positive sample data, and difficult negative sample data is constructed.

8. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 7, characterized in that, Steps 4-8 are detailed below: The triplet loss is calculated using the three global feature descriptors described in steps 4-7. The specific formula for calculating the triplet loss is as follows: ， in, Indicates the loss of the triplet. This represents the global feature descriptor of the anchor point. Represents the global feature descriptor for difficult-to-positive samples. Represents the global feature descriptor for hard-to-bear samples. Represents the Euclidean distance in the feature space. This indicates the set boundary threshold. Based on the triplet loss, the backpropagation algorithm is executed to iteratively update the learnable parameters in the global localization network; the training process from steps 4-5 to 4-8 is repeated until the triplet loss converges or the set maximum number of training rounds is reached, and finally the trained global localization model is obtained.

9. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 8, characterized in that, Step 5 is detailed below: Step 5-1: Input the multi-layer bird's-eye view of the test set into the trained global localization model, extract the corresponding global feature descriptors and local feature descriptors, and bind the extracted global feature descriptors and local feature descriptors with the real pose labels obtained in Step 1 to build a map database of the test scene, and then proceed to Step 5-2. Step 5-2: Select a multi-layer bird's-eye view from the test set as the query frame. Extract the local and global feature descriptors of the query frame using the trained global localization model. Then, perform a similarity search between the global feature descriptor of the query frame and the global feature descriptor in the map database. Select the reference frame that is most similar to the query frame based on the Euclidean distance. Then, obtain the true pose of the reference frame in the global coordinate system, the local feature descriptor of the reference frame, and the multi-layer bird's-eye view of the reference frame. Proceed to Step 5-3. Step 5-3: Apply the FAST operator to extract key points on the multi-layer bird's-eye view of the query frame and the reference frame respectively, and obtain the feature values ​​of the key points at the corresponding points of the local feature descriptors through bilinear interpolation; then, based on the extracted feature values, perform feature point matching between the key points of the query frame and the key points of the reference frame, and use the random sampling consensus algorithm to remove mismatched point pairs, and then calculate the rigid body transformation matrix of the query frame relative to the reference frame, and proceed to step 5-4. Step 5-4: Combining the true pose of the reference frame in the global coordinate system obtained in Step 5-2 and the rigid body transformation matrix obtained in Step 5-3, the precise pose of the query frame in the global coordinate system is calculated by matrix multiplication, thus completing high-precision global positioning.

10. A global positioning method based on multi-layer 3D point cloud bird's-eye view as described in claim 9, characterized in that, In step 5-4, the specific details are as follows: Based on the prior pose of the reference frame obtained in step 5-2 The relative rigid body transformation matrix obtained in step 5-3 The precise pose of the query frame in the global coordinate system is calculated using matrix multiplication. : ， in, This represents the pose of the reference frame in the global coordinate system. This represents the rigid body transformation matrix of the query frame relative to the reference frame. It represents the precise pose of the query frame in the global coordinate system, ultimately achieving high-precision global positioning.

Images