Vector map registration positioning method and system of a graph attention neural network

Abstract

The application belongs to the technical field of automatic driving, and particularly relates to a vector map registration positioning method and system of a graph attention neural network, and comprises the following steps: acquiring a vector map, wherein the vector map comprises a global vector map and a local vector map; encoding the vector map through a vector map local encoding network and a vector map global encoding network connected in sequence to obtain a vector map local feature and a vector map global feature; finding a mapping relationship between a graph structure feature of the global vector map and a graph structure feature of the local vector map, and obtaining pose information after attention aggregation and attention pose decoding processing. The application can use the vector map to perform a registration positioning task.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a vector map registration and localization method and system using a graph attention neural network. Background Technology

[0002] With the development of intelligent driving and autonomous driving, the amount of data collected by vehicle sensors is increasing exponentially. When processing large-scale scenarios, the prior point cloud map of the vehicle computing unit will consume a lot of storage resources, and the computing resources for the registration and positioning process will also increase significantly. Moreover, traditional registration and positioning methods are difficult to integrate into the data closed loop.

[0003] Vector maps are maps built from vector data and are widely used due to their lossless scaling and lightweight nature. However, the use of vector maps for registration and localization tasks is still a research gap.

[0004] Therefore, it is necessary to develop a vector map registration and localization method and system based on graph attention neural networks. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a vector map registration and localization method and system based on graph attention neural networks, which can use vector maps to perform registration and localization tasks.

[0006] In a first aspect, the vector map registration and localization method of the graph attention neural network of the present invention includes the following steps:

[0007] Obtain a vector map: Obtain a vector map, which includes a global vector map and a local vector map;

[0008] Vector map encoding: The vector map is encoded by a vector map local encoding network and a vector map global encoding network connected in sequence to obtain the vector map local features and vector map global features;

[0009] Registration and Decoding: Find the mapping relationship between the graph structure features of the global vector map and the graph structure features of the local vector map, and obtain the pose information after attention aggregation and attention pose decoding.

[0010] Optionally, the step of obtaining the vector map includes:

[0011] The global vector map is a pre-constructed prior map;

[0012] The local vector map is constructed during vehicle movement. This construction method combines prior knowledge and real-time data, improving the accuracy and robustness of registration and positioning.

[0013] Optionally, the vector map includes elements such as poles, lane lines, curbs, and traffic signs;

[0014] The elements are represented by polylines consisting of multiple vectors and their attributes. This representation accurately describes the shape and attributes of map elements, providing a basis for encoding and registration.

[0015] Optionally, the vector map encoding step includes:

[0016] The steps of local encoding for each element in a vector map, where each element contains multiple vectors, are as follows: each element is encoded into a subgraph through a vector map local encoding network. The subgraph consists of a set of nodes and an adjacency matrix of the nodes, where each vector is a node. The nodes are defined by the vector's start coordinates, end coordinates, attributes, and polyline number.

[0017] The steps for updating node encoding involve using a multi-layer structure, including feature connection operations, multi-layer perceptron processing, max pooling operations, and nearest neighbor vector features.

[0018] The steps of globally encoding the entire vector map through a global encoding network for vector maps involve using global graph structure data to connect various polyline features, and then using a graph neural network with an attention mechanism to encode all sub-graphs to obtain a global representation after attention aggregation.

[0019] The above local and global encoding steps are performed on both the global and local vector maps to obtain their global and local features. Each element is locally encoded as a subgraph, consisting of a set of nodes and an adjacency matrix; this encoding method captures the vector relationships and attribute information within each element. Node encodings are updated using a multi-layered structure, including feature connections, multilayer perceptron processing, max pooling, and nearest-neighbor vector features; this process extracts more abstract, high-level node features, enhancing the expressive power of the encoding. The entire vector map is then globally encoded using a global encoding network, connecting various polyline features using global graph structure data, and encoding all subgraphs through a graph neural network with an attention mechanism; this global encoding method captures the map's global structure and the relationships between elements.

[0020] Optionally, the registration and decoding steps specifically include:

[0021] The global and local features of the vector map are merged to form a multi-level complex graph structure feature.

[0022] The complex graph structure features are updated using self-attention and cross-attention mechanisms to obtain matching features of all nodes in the global vector map and the local vector map.

[0023] The matching features of all nodes in the global vector map and the local vector map are combined into a high-dimensional feature matrix;

[0024] The high-dimensional feature matrix is ​​fused to obtain a fused feature matrix;

[0025] The fused feature matrix is ​​input into the spatial feature extraction module. After convolution and max pooling operations, pose information is obtained through fully connected layers and attention layers. Global and local features are merged to form a multi-level complex graph structure feature; this merging method can comprehensively utilize global and local information, improving registration accuracy. The complex graph structure feature is updated through self-attention and cross-attention mechanisms to obtain matching features of all nodes in the global and local vector maps; this attention mechanism can capture the correlation and differences between nodes, helping to find the correct matching relationship. The matching features are combined into a high-dimensional feature matrix and then fused to obtain a fused feature matrix; this process can extract richer matching features and improve the robustness of registration. The fused feature matrix is ​​input into the spatial feature extraction module. After convolution and max pooling operations, pose information is obtained through fully connected layers and attention layers; this decoding method can convert the feature matrix into pose information, achieving the goal of registration and localization.

[0026] Optionally, the complex graph structure features are updated using self-attention and cross-attention mechanisms, specifically as follows:

[0027] When the number of layers is odd, a self-attention mechanism is used for calculation. Attention scores are calculated by query coefficients, key coefficients, and value coefficients, and node features are updated.

[0028] When the number of layers is even, a cross-attention mechanism is used for calculation. Combined with information from another map, the attention score is calculated by changing the coefficients of the attention mechanism, and the node features are updated. This alternating use of self-attention and cross-attention can more effectively capture the correlations and differences between nodes, improving the accuracy of registration.

[0029] Optionally, the matching features of all nodes in the global vector map and the local vector map are obtained through multi-layer propagation. This multi-layer propagation method can progressively extract and refine the matching features, improving the accuracy and stability of registration.

[0030] Optionally, the dimension of the high-dimensional feature matrix is ​​determined by the number of elements in the map and the feature dimension. This dimensionality setting ensures that the feature matrix contains sufficient information while avoiding increased computational complexity due to excessively high dimensionality.

[0031] Optionally, the pose information includes longitudinal coordinates, lateral coordinates, and an angle around the Z-axis. This information can accurately describe the vehicle's position and attitude on the map, providing a basis for localization.

[0032] Secondly, the present invention provides a vector map registration and localization system based on a graph attention neural network, comprising a memory and a controller. The memory stores a computer-readable program, which, when invoked by the controller, can execute the steps of the vector map registration and localization method based on a graph attention neural network as described in the present invention.

[0033] The unexpected beneficial effects of this invention are as follows:

[0034] (1) Hierarchical coding improves feature extraction capability:

[0035] This invention employs a hierarchical encoding method for vector maps using sequentially connected local and global encoding networks. This encoding approach captures both local details and global structure of the map, effectively improving the ability to obtain spatial relationships and attribute features. The local encoding network performs fine-grained feature extraction for each element (such as poles, lane lines, etc.), capturing its internal vector relationships and attribute information. The global encoding network then integrates these local features to form a global understanding of the entire vector map. This hierarchical encoding not only improves the accuracy of feature extraction but also enhances the model's ability to handle complex map structures. This enables the model to more accurately understand map information when facing complex and ever-changing real-world environments, providing a solid foundation for subsequent registration and localization.

[0036] (2) Integrating attention mechanisms to improve the accuracy of pose information:

[0037] In the vector map registration step, this invention incorporates an attention mechanism. This mechanism automatically focuses on important features in the map and ignores irrelevant noise, resulting in more accurate pose information after vector map registration. The attention mechanism dynamically adjusts the influence of different features on the registration result by calculating attention scores, which helps the model more accurately find the mapping relationship between the global and local vector maps in complex environments. The introduction of the attention mechanism significantly improves the accuracy and robustness of registration. Even with incomplete map information or the presence of noise, the model can accurately estimate the vehicle's position and attitude by focusing on important features.

[0038] (3) Use vector maps to reduce system resource consumption:

[0039] This invention uses vector maps as the analysis object. Compared to traditional raster maps or point cloud maps, vector maps have a more compact data structure, significantly reducing memory usage and computational requirements. Vector maps define map elements through multiple vectors and their attributes. This representation is not only concise and efficient but also facilitates various graph operations and feature extraction. Using vector maps significantly reduces the overall memory and computational usage of the system, thereby improving system efficiency and real-time performance. This is particularly important for autonomous driving systems that require real-time processing of large amounts of map data, ensuring efficient and stable operation in complex environments.

[0040] In summary, this invention achieves efficient processing and accurate registration and positioning of vector maps through hierarchical encoding, fusion attention mechanisms, and the use of vector maps. These technologies not only improve system performance and accuracy but also reduce system resource consumption, providing strong support for practical applications such as autonomous driving. Attached Figure Description

[0041] Figure 1 This is a flowchart of the vector map registration and localization method using a graph attention neural network as described in the embodiments of this application.

[0042] Figure 2 This is a schematic diagram illustrating the principle of vector map encoding in the embodiments of this application.

[0043] Figure 3 This is a schematic diagram of node encoding update in an embodiment of this application.

[0044] Figure 4 This is a schematic diagram of vector map registration in an embodiment of this application.

[0045] Figure 5 This is a schematic diagram of attention pose decoding in an embodiment of this application.

[0046] Figure 6 This is a schematic diagram of the vector map registration and localization system based on the graph attention neural network described in the embodiments of this application.

[0047] Figure 7 This is a schematic diagram of the computer-readable program described in the embodiments of this application. Detailed Implementation Plan

[0048] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] This application provides a vector map registration and localization method using a graph attention neural network. This method can be executed by any computing device with computing, processing, and storage functions, such as a PC, mobile terminal, or server.

[0050] like Figure 1 As shown in the embodiments of this application, a vector map registration and localization method using a graph attention neural network includes the following steps:

[0051] S1, Get Vector Map: Get the vector map, which includes a global vector map and a local vector map.

[0052] S2, Vector Map Encoding: The vector map is encoded using a series of interconnected local and global encoding networks to obtain local and global features. For example... Figure 2 The diagram shown illustrates the encoding of a vector map.

[0053] S3, Registration and Decoding: Find the mapping relationship between the graph structure features of the global vector map and the graph structure features of the local vector map, and obtain the pose information by sequentially processing attention aggregation and attention pose decoding.

[0054] In this embodiment, the global vector map is a pre-constructed prior map. The local vector map is a local vector map constructed during vehicle movement.

[0055] In one possible embodiment, the specific method for acquiring the vector map is as follows: A LiDAR is deployed on the vehicle to collect point cloud signals during the vehicle's movement, which are then sent to a computing device to convert the point cloud signals into a vector map. No specific limitations are placed on the type of LiDAR or the conversion method from point cloud to vector map.

[0056] In one possible embodiment, the vector map mainly includes elements such as poles, lane lines, curbs, and traffic signs. Lane lines and curbs are represented as polylines with multiple control points, traffic signs are represented by the spatial position indicated by four corner points, and poles are represented by a combination of horizontal and vertical coordinates and their diameters. All of the above elements can be approximated by polylines composed of multiple vectors and their attributes.

[0057] In one possible embodiment, the step of encoding the vector map specifically includes:

[0058] S21 is the step of locally encoding each element in the vector map. Each element contains multiple vectors. Each element is encoded into a subgraph using a vector map local encoding network. The subgraph consists of a set of nodes and an adjacency matrix of the nodes. Each vector is treated as a node, and each node is defined by the vector's start and end coordinates, attributes, and polyline number. Specifically:

[0059] The steps for local encoding of each element in the vector map, where each element contains multiple vectors, are as follows: Each element is encoded into a sub-map through the vector map local encoding network, and the sub-map is as follows: (1)

[0060] P j =(V j E j (1)

[0061] In formula (1), V j Let E be the set of nodes in the subgraph. j Let P be the adjacency matrix of the nodes. j Let represent the subgraph after encoding the j-th element, where each vector in the subgraph is a node, and the nodes are defined by the following formula (2):

[0062]

[0063] In formula (2), v i v represents the i-th node (or vector) of the subgraph. i ∈V j , Let these be the starting coordinates of vector i. a represents the endpoint coordinates of vector i. i The attribute is represented by vector i, where i represents the i-th vector, such as lane lines, curbs, traffic signs, and poles, and j' represents the polyline number of the vector.

[0064] S22, the step of updating node encoding, uses a multi-layer structure to update node encoding, including feature connection operations, multi-layer perceptron processing, max pooling operation, and nearest neighbor vector features; specifically as follows:

[0065] With node set V q =(v1,v2,v3,…,v q ,…,v n For example, v q ∈V q ,q=1,2,3,…,n, the node encoding update has a multi-level structure, as shown in the diagram below. Figure 3 As shown, the node's encoding is updated according to the following formula (3):

[0066]

[0067] In formula (3), This represents the encoded feature of the q-th vector in the (l+1)-th layer. Let represent the encoded features of the q-th vector in the l-th layer, φ(·) denote a simple feature concatenation operation, g(·) denote a multilayer perceptron (MLP), and Φ(·) denote a max pooling operation. This represents the feature of the nearest neighbor vectors of the q-th vector in the l-th layer.

[0068] After iteratively updating the node set through a multi-layer network structure, the object-level feature representation is obtained; the object-level feature representation is as follows: (4)

[0069]

[0070] In formula (4), O q L represents the feature of the q-th vector after local encoding. n Indicates the total number of floors. This indicates that the q-th vector is in the L-th position. n The layer's encoded features, view(·) indicates changing the feature dimension (specifically, removing the dimension with a dimension of 1).

[0071] S23, the step of globally encoding the entire vector map using a global encoding network, connects the various polyline features using global graph structure data, and encodes all sub-graphs using a graph neural network with an attention mechanism to obtain the global representation after attention aggregation; specifically:

[0072] The global encoding of the entire map is obtained through a vector map global encoding network, and the various polyline features are linked together using global graph structure data. All element features in a vector map (i.e., elements encoded to obtain element features, i.e., subgraphs) are represented as {p1, p2, p3, ..., p...}. m The following formula (5) is represented using a multilayer network:

[0073]

[0074] In formula (5), Let k represent the set of all polyline features in the r-th layer, k = 1, 2, 3, ..., m, where m is the number of polyline features; This represents the polyline feature of the k-th element in the (r+1)-th layer. Representing the adjacency matrix, GNN sa (·) represents the graph neural network encoding with an attention mechanism, as shown in the following formula (6):

[0075] GNN sa(P) = softmax(W) Q P(W K P) T W V (6)

[0076] In formula (6), P represents the feature matrix of all subgraphs, and W Q W K and W V These represent the linear matrices of query, key, and value in the attention mechanism, respectively, and are all learnable parameters. softmax() represents the softmax activation function.

[0077] All subgraphs are encoded using formula (5) to obtain the global representation after attention aggregation (MAP), as shown in formula (7):

[0078]

[0079] In formula (7), Rm represents the number of times the polymerization process is performed.

[0080] Perform the above local encoding and global encoding steps on the global vector map and the local vector map respectively to obtain the global and local features of the vector map.

[0081] S3, find the mapping relationship between the graph structure features of the global vector map and the graph structure features of the local vector map, and obtain the pose information after sequential attention aggregation and attention pose decoding. A schematic diagram of vector map registration is shown below. Figure 4 As shown.

[0082] In one possible embodiment, the attention pose decoding process in S3 includes:

[0083] S31, the global and local features of the vector map are merged to form a multi-level complex graph structure feature; the complex graph structure feature is updated through self-attention and cross-attention mechanisms to obtain the matching features of all nodes in the global and local vector maps; specifically as follows:

[0084] Through the operation in S2, the global vector map and the local vector map in S1 have been encoded into global vector map features and local vector map features, respectively denoted as... and A represents global features, and B represents local features. By merging the global features and local features of the vector map, a multi-level complex graph structure feature is formed.

[0085] Attention aggregation processing includes updating through self-attention and cross-attention mechanisms, and the updating process is as follows (8):

[0086]

[0087] In formula (8), This is represented as the feature of the i'th node in the global features at the l'+1 layer. Represented as the feature of the i'th node in the global vector map at layer l', m' ε→i' The calculation process of the attention mechanism is shown below. With the graph unchanged, when l' is singular, the self-attention mechanism is used for calculation, as shown in formula (9):

[0088]

[0089] In formula (9), α i',j' This is represented as an attention score. q i' Let k represent the query coefficient of the i'-th node in the self-attention mechanism. j' Let v be the key coefficient of the j'-th node in the self-attention mechanism. j' Let N(i') represent the value of the j'th node in the self-attention mechanism, and let N(i') represent the set of nearest neighbors of the i'th node.

[0090] When l' is even, the cross-attention mechanism is used for calculation, which requires combining the information of local features of the vector map. The coefficients of the attention mechanism in formula (9) need to be changed as shown in formula (10).

[0091]

[0092] Where, q i” k represents the coefficient of the query for the i-th node in the local features of the vector map in the cross-attention mechanism. j” V represents the coefficient of the key of the j-th node in the global features of the vector map in the cross-attention mechanism. j” The coefficient representing the value of the j”-th node in the global features of the vector map in the cross-attention mechanism. The coefficient matrix representing the query of the cross-attention mechanism for the features of nodes in layer l'. The coefficient matrix of the keys representing the cross-attention mechanism for the features of nodes in layer l'. The coefficient matrix b represents the values ​​of the cross-attention mechanism for the features of the l'-th layer nodes. q b represents the bias coefficient of the query in the cross-attention mechanism. k b represents the bias coefficient of the key in the cross-attention mechanism. v The bias coefficient representing the value in the cross-attention mechanism. This is represented as the feature of the i”th node in the global features at the l’th layer. It is represented as the feature of the j”th node in the l' layer in the local features.

[0093] After propagation through layer L', the matching features of all nodes in the global vector map are as follows: (11)

[0094]

[0095] in Let f represent the matching feature of the i-th node in the global vector map features, and let f represent the matching features of all nodes in the global vector map features. A W represents the weight coefficient matrix of the matching features. Represented as the i”'th node in the global vector map features, b i”' It is represented as the bias coefficient of the i”'th node in the global vector map features.

[0096] The matching features of all nodes in the local vector map are calculated using the same process described above, resulting in the matching features f of all nodes in the local vector map. B .

[0097] S32, combine the matching features of all nodes in the global vector map and the local vector map into high-dimensional feature matrices respectively; perform feature fusion on the high-dimensional feature matrices to obtain a fused feature matrix; input the fused feature matrix into the spatial feature extraction module, and after convolution and max pooling operations, decode it through a fully connected layer and an attention layer to obtain the pose information; specifically:

[0098] The matching features of all nodes in the global vector map and local vector map obtained through the S31 operation are combined into a high-dimensional feature matrix, denoted as . and Where N and M represent the number of elements in the global vector map and the local vector map, respectively, and d represents its feature dimension. In the implementation of this application, N = M = N' is uniformly set.

[0099] right and Perform feature fusion and fuse the feature matrices. The following formula (12):

[0100]

[0101] in

[0102] The fused feature matrix is ​​input into a spatial feature extraction module with two layers of convolution and max pooling operations. The matrix after spatial feature extraction is then unfolded and further encoded through a fully connected layer. Finally, it is input into an attention layer to obtain the decoded pose information.

[0103] The final output is pose information, including: X (longitudinal) coordinate value, Y (lateral) coordinate value, and Yaw (around the Z-axis) angle.

[0104] In one possible embodiment, the vector map registration and localization method of graph attention neural network is implemented through a pre-trained vehicle registration and localization model. During the model training process, the mean squared error loss function (MSE Loss) is used as the iterative optimization objective, as shown in the following formula (13):

[0105]

[0106] In formula (13), y o This represents the actual pose information. This represents the predicted pose information calculated using the model described above.

[0107] like Figure 6 As shown in the embodiments of this application, a vector map registration and localization system based on a graph attention neural network includes a memory and a controller. The memory stores a computer-readable program. When the computer-readable program is invoked by the controller, it can execute the steps of the vector map registration and localization method based on a graph attention neural network as described in the embodiments of this application.

[0108] like Figure 7 As shown in the embodiments of this application, the computer-readable program is divided according to functional modules, which can be divided into an acquisition module, a vector map encoding module, a vector map registration module, and an output module connected in sequence.

[0109] The acquisition module is used to acquire vector maps.

[0110] The vector map encoding module is used to encode vector maps, converting them into local and global features. This module includes a local encoding network and a global encoding network.

[0111] The vector map registration module is used to find the mapping relationship between the graph structure features of the global vector map and the graph structure features of the local vector map. After attention aggregation and attention pose decoding, the pose information is obtained. The vector map registration module includes an attention aggregation submodule and an attention pose decoding submodule.

[0112] The output module is used to output pose information.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vector map registration and localization method using a graph attention neural network, characterized in that, Includes the following steps: Obtain a vector map: Obtain a vector map, which includes a global vector map and a local vector map; Vector map encoding: The vector map is encoded using a locally encoded network and a globally encoded network, which are connected sequentially, to obtain local and global features. Specifically, the process involves: Local encoding of each element in the vector map, where each element contains multiple vectors. The local encoding network encodes each element into a subgraph, which consists of a set of nodes and an adjacency matrix. Each vector is considered a node, defined by its origin, destination, attributes, and polyline number. The node encoding is then updated using a multi-layered structure, including feature concatenation, multilayer perceptron processing, max pooling, and nearest-neighbor vector features. Finally, the global encoding network performs global encoding on the entire vector map. Global graph structure data is used to connect the polyline features, and a graph neural network with an attention mechanism encodes all subgraphs, resulting in a globally represented representation after attention aggregation. These local and global encoding steps are performed on both the global and local vector maps to obtain the global and local features of the vector map. Registration and Decoding: Find the mapping relationship between the graph structure features of the global vector map and the graph structure features of the local vector map, and obtain the pose information after attention aggregation and attention pose decoding.

2. The vector map registration and localization method using a graph attention neural network according to claim 1, characterized in that, The global vector map is a pre-constructed prior map; The local vector map is a local vector map constructed during vehicle movement.

3. The vector map registration and localization method using a graph attention neural network according to claim 1, characterized in that, The vector map contains elements such as poles, lane lines, curbs, and traffic signs; The elements are all represented by polylines composed of multiple vectors and their attributes.

4. The vector map registration and localization method using a graph attention neural network according to claim 3, characterized in that, The registration and decoding steps specifically include: The global and local features of the vector map are merged to form a multi-level complex graph structure feature; the complex graph structure feature is updated through self-attention and cross-attention mechanisms to obtain the matching features of all nodes in the global and local vector maps. The matching features of all nodes in the global vector map and the local vector map are combined into high-dimensional feature matrices; the high-dimensional feature matrices are fused to obtain fused feature matrices; the fused feature matrices are input into the spatial feature extraction module, and after convolution and max pooling operations, the pose information is obtained by decoding through a fully connected layer and an attention layer.

5. The vector map registration and localization method using a graph attention neural network according to claim 4, characterized in that, The complex graph structure features are updated using self-attention and cross-attention mechanisms, specifically as follows: When the number of layers is odd, a self-attention mechanism is used for calculation. Attention scores are calculated by query coefficients, key coefficients, and value coefficients, and node features are updated. When the number of layers is even, a cross-attention mechanism is used for calculation. By combining information from another map, the attention score is calculated by changing the coefficients of the attention mechanism, and the node features are updated.

6. The vector map registration and localization method using a graph attention neural network according to claim 4, characterized in that, The matching features of all nodes in the global vector map and the local vector map are obtained through multi-layer propagation.

7. The vector map registration and localization method using a graph attention neural network according to claim 4, characterized in that, The dimension of the high-dimensional feature matrix is ​​determined by the number of elements in the map and the feature dimension.

8. The vector map registration and localization method using a graph attention neural network according to claim 1, characterized in that, The pose information includes longitudinal coordinate values, lateral coordinate values, and angles around the Z-axis.

9. A vector map registration and localization system based on a graph attention neural network, characterized in that, It includes a memory and a controller, wherein the memory stores a computer-readable program that, when invoked by the controller, can perform the steps of the vector map registration and localization method of the graph attention neural network as described in any one of claims 1 to 8.

Images