Positioning method and apparatus

Abstract

A positioning method, a positioning apparatus, a vehicle, a readable storage medium, and a computer program product, used for calculating a positioning offset on the basis of a difference between a vector element in an offline map and a sensing element, thereby correcting initial positioning in combination with environment sensing, and outputting a positioning result with higher precision. The method comprises: acquiring a first vector element set and a second vector element set (201), the first vector element set comprising at least one first vector element in a first map, and the second vector element set comprising at least one second vector element in data collected on the basis of a sensor; acquiring an offset on the basis of a difference between the first vector element set and the second vector element set (202); obtaining first positioning on the basis of the offset and initial positioning (203), the initial positioning being used for representing an initial positioning location of a target, and the offset being able to be used for correcting the initial positioning, so that the precision of the first positioning is higher than the precision of the initial positioning.

Description

A positioning method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202411225711.9, filed with the State Intellectual Property Office of China on August 30, 2024, entitled “A Positioning Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of positioning, and more particularly to a positioning method and apparatus. Background Technology

[0003] In intelligent scenarios, the demand for precise device positioning is increasing. In scenarios such as autonomous driving, robot navigation, or terminal positioning, accurate device positioning is crucial for the realization of device functions. For example, in autonomous driving scenarios, the accuracy of vehicle positioning can directly affect driving safety.

[0004] High-precision positioning of equipment heavily relies on initial positioning. Typically, laser positioning depends on iterative closest point (ICP) matching of laser point clouds, while vector map positioning relies on k-dimensional tree (KD-Tree) matching of vector topological maps. Achieving high-precision positioning heavily depends on small initial positioning errors, such as less than 5 meters. This initial positioning is usually achieved using a Global Navigation Satellite System (GNSS). However, in some scenarios, such as urban canyons or overpasses, multipath propagation or occlusion issues can lead to larger initial positioning errors, such as 20 meters. In these cases, neither ICP matching nor vector map matching can quickly converge from such large errors, thus failing to achieve rapid high-precision positioning.

[0005] Therefore, how to achieve more efficient and high-precision positioning has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a positioning method and apparatus for calculating the positioning offset based on the difference between vector elements and perceived elements in an offline map, thereby combining environmental perception to correct the initial positioning and output a more accurate positioning result.

[0007] In a first aspect, this application provides a positioning method, comprising: acquiring a first vector element set and a second vector element set, wherein the first vector element set includes at least one first vector element from a first map, and the second vector element set includes at least one second vector element from data collected by a sensor; subsequently acquiring an offset based on the difference between the first vector element set and the second vector element set; subsequently obtaining a first positioning based on the offset and an initial positioning, wherein the initial positioning is used to represent the initial positioning position of the target, and the offset can be used to correct the initial positioning so that the accuracy of the first positioning is higher than the accuracy of the initial positioning.

[0008] In this embodiment, the difference between vectors in the map and vectors in the perception is used to calculate the initial positioning offset, and this offset is used to correct the initial positioning, resulting in a more accurate positioning result. This is equivalent to combining environmental perception capabilities to correct the initial positioning, which can improve positioning accuracy even in scenarios with large initial positioning errors, using less computing power.

[0009] In one possible implementation, the aforementioned process of obtaining the offset based on the first set of vector elements and the second set of vector elements may include: obtaining first vector features based on the first set of vector elements and second vector features based on the second set of vector elements using a feature extraction network, wherein the feature extraction network may be a network that extracts features based on an attention mechanism; and then obtaining the offset based on the first vector features and the second vector features. In this embodiment, the offset can be calculated using the difference between the features extracted from the two vectors, thereby obtaining a more accurate offset with lower computational cost.

[0010] In one possible implementation, the aforementioned acquisition of a first vector feature based on a first set of vector elements and acquisition of a second vector feature based on a second set of vector elements may include: encoding the first set of vector elements to obtain a first encoding and encoding the second set of vector elements to obtain a second encoding, thereby encoding the vector elements as an input sequence of a feature extraction network, and then extracting features from the first encoding to obtain the first vector feature and extracting features from the second encoding to obtain the second vector feature through the feature extraction network.

[0011] In this embodiment of the application, vector elements can be encoded before features are extracted, thereby extracting more accurate vector features from the encoded data.

[0012] In one possible implementation, the aforementioned encoding of the first vector element set to obtain a first code, and encoding of the second vector element set to obtain a second code, includes: obtaining a first vector code and a first geometric code based on node information of the first vector elements in the first vector element set. The first code includes both a first vector code and a first geometric code. The first vector code represents at least one vector formed by adjacent nodes in the first vector element, and the first geometric code represents the distance and / or offset angle between adjacent nodes forming the vector in the first vector element. Furthermore, obtaining a second vector code and a second geometric code based on node information of the second vector elements in the second vector element set. The second code includes both a second vector code and a second geometric code. The second vector code represents at least one vector formed by adjacent nodes in the second vector element, and the second geometric code represents the distance and / or offset angle between adjacent nodes forming the vector in the second vector element. In this embodiment, encoding can be performed from either the vector dimension or the geometric dimension, so that the output encoded data can accurately represent the vector elements, enabling the subsequent extraction of accurate features.

[0013] In one possible implementation, the aforementioned acquisition of a first vector feature based on a first set of vector elements and acquisition of a second vector feature based on a second set of vector elements may include: extracting features from the first set of vector elements using a graph neural network to obtain a first initial feature, and extracting features from the second set of vector elements to obtain a second initial feature; and extracting features from the first initial feature using a feature extraction network to obtain a first vector feature, and extracting features from the second initial feature to obtain a second vector feature.

[0014] In this embodiment, graph neural networks can be used to directly extract features from vector elements, thereby extracting vector features from data very accurately and efficiently.

[0015] In one possible implementation, the aforementioned steps of obtaining a first vector feature based on a first set of vector elements and obtaining a second vector feature based on a second set of vector elements may include: obtaining a first image corresponding to the first set of vector elements and a second image corresponding to the second set of vector elements; then, through an image feature extraction network, extracting a first image feature from the first image and extracting a second image feature from the second image; then, through a feature extraction network, extracting features from the first image features to obtain a first vector feature and extracting features from the second image features to obtain a second vector feature.

[0016] In this embodiment, vectors can be converted into images or projected onto images, and then initial vector features can be extracted from the images using an image feature extraction network, which can achieve highly efficient vector feature extraction.

[0017] In one possible implementation, the aforementioned feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism. Therefore, in this embodiment, the feature extraction network can focus on the correlation between features or the correlation between features to extract features, thereby making the final output features more accurate.

[0018] In one possible implementation, the aforementioned process of obtaining the offset based on the first vector feature and the second vector feature may include: fusing the first vector feature and the second vector feature to obtain a fused feature, such as stitching or weighted fusion; inputting the fused feature into a first positioning model and outputting the offset. In this embodiment, the perception capability of the positioning model can be utilized to calculate the offset based on the difference between the map vector feature and the perceived vector feature, thereby calculating a more accurate offset with very low computing power.

[0019] In one possible implementation, the aforementioned method further includes: inputting the fused features into a second localization model and outputting the confidence level of the offset. In embodiments of this application, the confidence level of the offset may also be output so that the usability of the offset can be evaluated in the next stage based on this confidence level.

[0020] In one possible implementation, the aforementioned method may further include: obtaining a second map using an offset and a first map; sensing the result, for example, offsetting the first map using the offset to obtain the second map; then obtaining a verification index based on the difference between the second map and the first map, the verification index indicating the usability of the offset; and then, if the verification index meets preset verification conditions, updating the initial positioning based on the offset to obtain a first positioning. In this embodiment, the offset may also be substituted into the map to verify the offset, and if the verification passes, the initial positioning may be updated using the offset, i.e., updating the initial positioning when the accuracy of the offset is high, thereby improving the accuracy of the output first positioning.

[0021] In one possible implementation, the aforementioned process of obtaining a verification index based on the difference between the second map and the first map may include: obtaining a first mask corresponding to the first map and a second mask corresponding to the second map; determining a luminous flux index based on the first and second masks, the luminous flux index including the number of element points through which the same ray passes from both the first and second masks, and using this luminous flux index as a verification index or one of the verification indices, such as considering the offset usable when the luminous flux exceeds a first threshold. In this embodiment, the luminous flux between the original offline map and the corrected map can be used as a verification index, thereby accurately representing the difference between the two maps and more accurately identifying whether the offset is accurate.

[0022] In one possible implementation, when obtaining the confidence level of the offset, the verification metric also includes the confidence level of the offset. For example, when the confidence level is the variance of the offset, the offset can be considered usable if the variance is less than a second threshold. In this embodiment, the confidence level of the offset can also be used as a verification metric, thereby determining whether the offset is usable more efficiently.

[0023] In one possible implementation, the aforementioned method may further include: if the verification index does not meet preset conditions, the offset may be recalculated or the offset may be recalculated using the next frame of sensor data. In this embodiment, if the offset fails verification, the offset may be recalculated until a verified offset is obtained, thereby achieving more accurate initial positioning correction.

[0024] In one possible implementation, the aforementioned method may further include: using the first location as the initial location, calculating the offset again based on the first vector element set and the second vector element set, and then updating the first location to obtain the second location. In this embodiment, the corrected first location may also be used as the initial location to iteratively calculate the offset, thereby refining the first location with finer granularity through iteration and improving the accuracy of the output location result.

[0025] Secondly, this application provides a positioning device, comprising:

[0026] The acquisition module is used to acquire a first vector element set and a second vector element set. The first vector element set includes at least one first vector element in the first map, and the second vector element set includes at least one second vector element in the data collected by the sensor.

[0027] The offset calculation module is used to obtain the offset based on the first vector element set and the second vector element set;

[0028] The positioning module is used to obtain the first positioning based on the offset and the initial positioning. The initial positioning is used to represent the initial positioning position of the target. The accuracy of the first positioning is higher than that of the initial positioning.

[0029] The effects achieved by the second aspect and any optional implementation of the second aspect can be referred to the effects described in the first aspect or any optional implementation of the first aspect, and will not be repeated here.

[0030] In one possible implementation, the offset calculation module is specifically used to: obtain a first vector feature based on a first set of vector elements and a second vector feature based on a second set of vector elements through a feature extraction network; and obtain an offset based on the first vector feature and the second vector feature.

[0031] In one possible implementation, the offset calculation module is specifically used for: encoding a first set of vector elements to obtain a first code, and encoding a second set of vector elements to obtain a second code; extracting features from the first code to obtain a first vector feature through a feature extraction network, and extracting features from the second code to obtain a second vector feature.

[0032] In one possible implementation, the offset calculation module is specifically used for: obtaining a first vector code and a first geometric code based on the node information of the first vector elements in the first vector element set, wherein the first code includes the first vector code and the first geometric code, the first vector code representing at least one vector formed by adjacent nodes in the first vector element, and the first geometric code representing the distance or offset angle between adjacent nodes forming the vector in the first vector element; and obtaining a second vector code and a second geometric code based on the node information of the second vector elements in the second vector element set, wherein the second code includes the second vector code and the second geometric code, the second vector code representing at least one vector formed by adjacent nodes in the second vector element, and the second geometric code representing the distance or offset angle between adjacent nodes forming the vector in the second vector element.

[0033] In one possible implementation, the offset calculation module is specifically used for: extracting features from a first set of vector elements through a graph neural network to obtain a first initial feature, and extracting features from a second set of vector elements to obtain a second initial feature; and extracting features from the first initial feature through a feature extraction network to obtain a first vector feature, and extracting features from the second initial feature to obtain a second vector feature.

[0034] In one possible implementation, the offset calculation module is specifically used to: obtain a first image corresponding to a first set of vector elements and a second image corresponding to a second set of vector elements; extract first image features from the first image and extract second image features from the second image through an image feature extraction network; extract features from the first image features to obtain first vector features through a feature extraction network, and extract features from the second image features to obtain second vector features.

[0035] In one possible implementation, the feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism.

[0036] In one possible implementation, the offset calculation module is specifically used to: fuse the first vector feature and the second vector feature to obtain the fused feature; input the fused feature into the first positioning model and output the offset.

[0037] In one possible implementation, the offset calculation module is further configured to: input the fused features into the second localization model and output the confidence level of the offset.

[0038] In one possible implementation, the aforementioned apparatus further includes: a post-processing module, configured to: obtain a second map using the offset and a first map; and obtain a verification index based on the difference between the second map and the first map, the verification index being used to indicate the availability of the offset;

[0039] The positioning module is specifically used to update the initial positioning based on the offset to obtain the first positioning when the verification indicators meet the preset verification conditions.

[0040] In one possible implementation, the post-processing module is specifically used to: obtain a first mask corresponding to a first map and a second mask corresponding to a second map; determine a luminous flux index based on the first mask and the second mask, wherein the luminous flux index includes the number of element points through which the same ray passes from both the first mask and the second mask, and the verification index includes the luminous flux index.

[0041] In one possible implementation, when obtaining the confidence level of the offset, the verification metric also includes the confidence level of the offset.

[0042] In one possible implementation, the positioning module is further configured to use the first positioning as the initial positioning, and after calculating the offset again based on the first vector element set and the second vector element set, update the first positioning to obtain the second positioning.

[0043] Thirdly, embodiments of this application provide a positioning device, including a processor and a memory, wherein the processor and the memory are interconnected via a circuit, and the processor calls program code in the memory to execute processing-related functions in the positioning method shown in any of the first aspects above. Optionally, the positioning device may be a chip.

[0044] Fourthly, embodiments of this application provide an intelligent driving vehicle, which includes a processor and a memory, wherein the processor and the memory are interconnected by a circuit, and the processor calls program code in the memory to perform processing-related functions in the positioning method shown in any of the first aspects above.

[0045] Fifthly, embodiments of this application provide a digital processing chip or chip, the chip including a processing unit and a communication interface, the processing unit obtaining program instructions through the communication interface, the program instructions being executed by the processing unit, the processing unit being used to perform processing-related functions as described in the first aspect or any optional embodiment of the first aspect.

[0046] In a sixth aspect, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any optional implementation thereof.

[0047] In a seventh aspect, embodiments of this application provide a computer program product comprising a computer program / instructions, which, when executed by a processor, causes the processor to perform the method described in the first aspect or any optional implementation thereof. Attached Figure Description

[0048] Figure 1 is a structural schematic diagram of a vehicle provided in an embodiment of this application;

[0049] Figure 2 is a flowchart illustrating a positioning method provided in an embodiment of this application;

[0050] Figure 3 is a schematic diagram of an application architecture of the method provided in an embodiment of this application;

[0051] Figure 4 is a schematic diagram of another application architecture of the method provided in the embodiments of this application;

[0052] Figure 5 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0053] Figure 6 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0054] Figure 7 is a schematic diagram of a vector method provided in an embodiment of this application;

[0055] Figure 8 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0056] Figure 9 is a schematic diagram of a GNN structure provided in an embodiment of this application;

[0057] Figure 10 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0058] Figure 11 is a schematic diagram of a Transformer structure provided in an embodiment of this application;

[0059] Figure 12 is a schematic diagram of a self-attention module structure provided in an embodiment of this application;

[0060] Figure 13 is a schematic diagram of a cross-attention module structure provided in an embodiment of this application;

[0061] Figure 14 is a schematic diagram of a pose decoder structure provided in an embodiment of this application;

[0062] Figure 15 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0063] Figure 16 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0064] Figure 17 is a flowchart illustrating another positioning method provided in an embodiment of this application;

[0065] Figure 18 is a structural schematic diagram of a positioning device provided in an embodiment of this application;

[0066] Figure 19 is a schematic diagram of another positioning device provided in an embodiment of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0068] To facilitate understanding, some terms or concepts involved in the methods provided in the embodiments of this application will be introduced first.

[0069] (1) Deep Neural Networks

[0070] A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple intermediate layers. Based on the position of these layers, the internal neural network of a DNN can be divided into three categories: input layer, intermediate layers, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are considered intermediate layers, or hidden layers.

[0071] Although DNNs appear complex, each layer can be represented as a linear relational expression: in, It is the input vector. It is the output vector. is the offset vector, also known as the bias parameter; w is the weight matrix (also called coefficients); and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is also quite large. The definitions of these parameters in DNNs are as follows: Taking the coefficient w as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0072] In summary, the coefficient from the k-th neuron in layer L-1 to the j-th neuron in layer L is defined as...

[0073] It's important to note that the input layer does not have a W parameter. In deep neural networks, more intermediate layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0074] (2) Convolutional Neural Network

[0075] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of a series of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are called the convolutional kernel. Shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, the kernel can learn appropriate weights. Furthermore, the direct benefit of shared weights is that it reduces the connections between layers in the CNN, while also reducing the risk of overfitting.

[0076] (3) Graph Neural Networks (GNN)

[0077] A graph typically includes multiple nodes, and the features of a node can include the node's own features + the features of its neighboring nodes + the features of the neighboring nodes' neighboring nodes + ... GNN can be used to extract the features of each node in a graph.

[0078] GNN is a very broad category, including various graph-based neural network models, such as Graph Convolutional Network (GCN), GraphSAGE, and GAT.

[0079] (4) Graph Convolutional Network (GCN)

[0080] Graph convolutional neural networks are deep learning models for modeling and processing non-Euclidean spatial data (such as graph data). Their principle is based on pairwise message passing, allowing graph nodes to iteratively update their representations by exchanging information with their neighbors.

[0081] Similar to CNNs, GCNs differ in that CNNs typically take two-dimensional data as input, while GCNs typically take graph data as input. GCNs ingeniously design a method to extract features from graph data, which can then be used to perform node classification, graph classification, link prediction, and obtain graph embeddings.

[0082] (5) transformer

[0083] A transformer structure is a feature extraction network that includes both an encoder and a decoder (classified as a convolutional neural network). Of course, in some cases, a transformer structure may not include an encoder but may include a decoder.

[0084] Encoder: Learns features, such as pixel features, in the global receptive field through self-attention.

[0085] Decoder: Learns the features of the desired module, such as the features of the output box, through self-attention and cross-attention.

[0086] For example, a Transformer layer structure may include an attention network and a feedforward network module. Taking natural language processing as an example, the attention network obtains corresponding weight values ​​by calculating the relevance between words based on the attention mechanism, thus obtaining context-related word representations, which is the core part of the Transformer structure. The feedforward network further transforms the obtained representations to obtain the final output of the Transformer layer. In addition to these two important components, residual layers (ADD) and linear normalization (Norm) are also stacked on these two components to optimize the output of the Transformer layer.

[0087] (6) Attention mechanism

[0088] Attention mechanisms can quickly extract important features from sparse data. They provide an effective modeling approach for capturing global contextual information through QKV (Queries, Keys, Values). Assuming the input is Q(query), and the context is stored as key-value pairs (K, V), then the attention mechanism is essentially a mapping function from the query to a series of key-value pairs. The essence of the attention function can be described as a mapping from a query to a series of (key-value) pairs. Attention essentially assigns a weight coefficient to each element in the sequence, which can also be understood as soft addressing. If each element in the sequence is stored in (K, V) form, then attention performs addressing by calculating the similarity between Q and K. The similarity calculated between Q and K reflects the importance of the extracted V values, i.e., the weights, and then a weighted sum is obtained to obtain the final feature value.

[0089] Attention calculation mainly consists of three steps. The first step is to calculate the similarity between the query and each key to obtain weights. Common similarity functions include dot product, concatenation, and perceptron. The second step typically uses a softmax function (which can normalize the weights, resulting in a probability distribution where the sum of all weight coefficients is 1, and also highlights the weights of important elements) to normalize these weights. Finally, the weights and their corresponding key values ​​are weighted and summed to obtain the final feature value. The specific calculation formula is as follows:

[0090] Where d is the dimension of matrix Q,K.

[0091] Furthermore, attention includes self-attention and cross-attention. Self-attention can be understood as a special type of attention where the inputs to the QKV features are consistent. Cross-attention, on the other hand, involves inconsistent inputs to the QKV features. Attention integrates the queried features as updated values ​​for the current features using the similarity between features (e.g., inner product) as weights. Self-attention is attention extracted based on the attention drawn from the feature map itself.

[0092] For convolutional networks, the kernel size limits the receptive field, often requiring multiple layers to focus on the entire feature map. Self-attention, on the other hand, has the advantage of global focus; it can obtain global spatial information about the feature map through simple lookups and assignments.

[0093] (7) Mask

[0094] A mask can be understood as a label for data, also called a code. By fusing data with a corresponding mask, it allows for greater focus on specific parts of the data. Typically, masks are used to extract regions of interest (ROIs). For example, a pre-made ROI mask is fused with the image to be processed to obtain an ROI image, where image values ​​within the ROI remain unchanged, while values ​​outside the ROI are all 0. Masks can also serve a shielding function, such as using a mask to shield certain areas of an image, preventing them from participating in processing or parameter calculations, or processing or statistically analyzing only the unmasked areas.

[0095] Secondly, the method provided in this application can be applied to various device positioning scenarios, such as terminal positioning, vehicle positioning, or robot positioning. Correspondingly, the method provided in this application can also be deployed in devices such as terminals or servers.

[0096] In one possible implementation, the method provided in this application can be deployed in a terminal, such as a mobile phone, tablet personal computer (TPC), media player, smart TV, laptop computer (LC), augmented reality (AR) / virtual reality (VR), in-vehicle terminal, intelligent driving vehicle, personal digital assistant (PDA), personal computer (PC), camera, camcorder, smartwatch, wearable device (WD), etc., and this application does not limit it in this regard.

[0097] In one possible implementation, the method provided in this application embodiment can also be deployed on a server, such as a cloud server or a server connected to a terminal. For example, the method provided in this application embodiment can be deployed on a cloud server connected to a terminal. The cloud server can receive data such as images or point clouds from the terminal, and the cloud server uses the method provided in this application embodiment to locate the terminal and provide feedback to the terminal.

[0098] For example, high-precision vehicle positioning is crucial for intelligent driving functions, such as Lane Departure Warning (LDW) or Lane Keeping Assist (LKA) within Advanced Driving Assistance Systems (ADAS). Lane Departure Warning and Lane Keeping Assist require real-time detection of lane markings and determination of whether the vehicle is deviating from its current lane. If the vehicle deviates from its lane without a turn signal being given, a warning signal will be issued, the steering wheel will vibrate, or even the steering wheel will be actively pulled back to remind the driver to return to the lane. The method provided in this application supports precise vehicle positioning, accurately determining the vehicle's position in a real-world scenario. For instance, when high-precision positioning determines that the vehicle has deviated from its lane, the Lane Keeping Assist system can control the steering wheel to steer the vehicle back into its current lane.

[0099] For example, in robot applications, robots typically need to accurately locate themselves in order to plan their travel routes. Therefore, the method provided in this application can be used to perform high-precision positioning of the robot, thereby enabling the planning of a more suitable travel route based on high-precision positioning.

[0100] For example, in a terminal positioning scenario, taking a mobile phone as an example, users can enable location services on their mobile phones and deploy the methods provided in this application's embodiments on those phones, thereby providing users with more accurate positioning, displaying more precise positioning information, and improving user experience.

[0101] The following describes the device structure used in the method provided in this application embodiment, taking deployment in a vehicle as an example.

[0102] For example, taking the method provided in this application embodiment as an example of deployment in a vehicle, the structure of the vehicle can be as shown in Figure 1.

[0103] Referring to Figure 1, which is a schematic diagram of a vehicle structure provided in an embodiment of this application, the vehicle 100 can be configured in an intelligent driving mode. For example, the vehicle 100 can control itself while in intelligent driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine whether there are obstacles in the surrounding environment, and control the vehicle 100 based on the obstacle information. When the vehicle 100 is in intelligent driving mode, it can also be set to operate without human interaction.

[0104] Please refer to Figure 1 first. Figure 1 is a structural schematic diagram of a vehicle provided in an embodiment of this application. Figure 1 is a functional block diagram of a vehicle 100 provided in an embodiment of this application. The vehicle 100 can be configured to a full or partial intelligent driving mode. For example, the vehicle 100 can obtain environmental information about its surroundings through the perception system 120, and obtain an intelligent driving strategy based on the analysis of the surrounding environmental information to achieve full intelligent driving, or present the analysis results to the user to achieve partial intelligent driving.

[0105] Vehicle 100 may include various subsystems, such as an infotainment system 110, a perception system 120, a decision control system 130, a drive system 140, and a computing platform 150. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0106] In some embodiments, the infotainment system 110 may include a communication system 111, an entertainment system 112, and a navigation system 113.

[0107] Communication system 111 may include wireless communication system 111, which can communicate wirelessly with one or more devices directly or via a communication network. For example, wireless communication system 111 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. Wireless communication system 111 may communicate using WiFi and wireless local area network (WLAN). In some embodiments, wireless communication system 146 may communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Wireless communication system 111 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0108] The entertainment system 112 may include a central control screen, a microphone, and speakers. Users can listen to the radio and play music within the vehicle using the entertainment system 112; or connect their mobile phones to the vehicle and project their screens onto the central control screen, which may be touch-sensitive, allowing users to operate the system. In some cases, the microphone can capture the user's voice signal, and analysis of this signal can enable the user to control certain aspects of the vehicle 100, such as adjusting the interior temperature. In other cases, music can be played to the user through the speakers.

[0109] The navigation system 113 may include map services to provide navigation for the vehicle 100, and the navigation system 113 may be used in conjunction with the vehicle's global positioning system 121 and inertial measurement unit 122. The map may be a two-dimensional map, a high-precision map, or a map constructed based on data collected during the vehicle's operation.

[0110] The perception system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100. For example, the perception system 120 may include a global positioning system 121 (which may include a global navigation satellite system (GNSS), specifically GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU) 122, a lidar (LiDAR) 123, a millimeter-wave radar 124, an ultrasonic radar 125, and a camera device 126. The perception system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the vehicle 100. The data collected by sensors in the vehicle mentioned below in this application may include data collected by the various units in the perception system 120.

[0111] The Global Positioning System 121 can be used to determine the geographical location of vehicle 100.

[0112] The inertial measurement unit 122 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 may be a combination of an accelerometer and a gyroscope.

[0113] The lidar 123 can use lasers to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the lidar 123 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0114] The millimeter-wave radar 124 can use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the millimeter-wave radar 124 can also be used to sense the speed and / or direction of travel of objects.

[0115] The ultrasonic radar 125 can use ultrasonic signals to sense objects around the vehicle 100.

[0116] The camera device 126 can be used to capture image information of the surrounding environment of the vehicle 100. The camera device 126 may include a monocular camera, a binocular camera, a structured light camera, and a panoramic camera, etc. The image information acquired by the camera device 126 may include still image information or video stream information.

[0117] The decision control system 130 includes a computing system 131 that analyzes and makes decisions based on information acquired by the sensing system 120. The decision control system 130 also includes a vehicle controller 132 that controls the power system of the vehicle 100, and a steering system 133, a throttle 134, and a braking system 135 for controlling the vehicle 100.

[0118] The computing system 131 can process and analyze various information acquired by the perception system 120 to identify targets, objects, and / or features in the environment surrounding the vehicle 100. The targets may include pedestrians or animals, and the objects and / or features may include traffic signals, road boundaries, and obstacles. The computing system 131 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking, and other techniques. In some embodiments, the computing system 131 may be used to map the environment, track objects, estimate object speeds, etc. The computing system 131 can analyze the acquired information and derive a control strategy for the vehicle.

[0119] The vehicle controller 132 can be used to coordinate the control of the vehicle's power battery and engine 141 to improve the power performance of the vehicle 100.

[0120] The steering system 133 can be used to adjust the forward direction of the vehicle 100. For example, in one embodiment, it can be a steering wheel system.

[0121] The throttle 134 is used to control the operating speed of the engine 141 and thus the speed of the vehicle 100.

[0122] Braking system 135 is used to control the deceleration of vehicle 100. Braking system 135 can use friction to slow down the rotational speed of wheel 144. In some embodiments, braking system 135 can convert the kinetic energy of wheel 144 into electric current. Braking system 135 may also take other forms to slow down the rotational speed of wheel 144 to control the speed of vehicle 100.

[0123] The drive system 140 includes components that provide powered motion to the vehicle 100. In one embodiment, the drive system 140 may include an engine 141, an energy source 142, a transmission system 143, and wheels 144. The engine 141 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 141 converts the energy source 142 into mechanical energy.

[0124] Examples of energy sources 142 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 142 may also provide energy to other systems of vehicle 100.

[0125] The drivetrain 143 transmits mechanical power from the engine 141 to the wheels 144. The drivetrain 143 may include a gearbox, a differential, and a drive shaft. In one embodiment, the drivetrain 143 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 144.

[0126] Some or all of the functions of vehicle 100 are controlled by computing platform 150. Computing platform 150 may include at least one processor 151, which can execute instructions 153 stored in a non-transitory computer-readable medium such as memory 152. In some embodiments, computing platform 150 may also be multiple computing devices that control individual components or subsystems of vehicle 100 in a distributed manner.

[0127] Processor 151 can be any conventional processor, such as a commercially available CPU. Alternatively, processor 151 may also include a graphics processing unit (GPU), a field-programmable gate array (FPGA), a system-on-chip (SoC), an application-specific integrated circuit (ASIC), or a combination thereof. Processor 151 can be located on a device remote from the vehicle and can communicate wirelessly with the vehicle.

[0128] In some embodiments, memory 152 may contain instructions 153 (e.g., program logic) that can be executed by processor 151 to perform various functions of vehicle 100. Memory 152 may also contain additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the infotainment system 110, perception system 120, decision control system 130, and drive system 140.

[0129] The method provided in this application embodiment can be executed by a computing platform 150. For example, the processor 151 can read the program stored in the memory 152, perform high-precision positioning based on the data collected by the perception system 120, and send instructions to the decision control system 130 to control the vehicle to achieve intelligent driving function.

[0130] Of course, the method provided in this application embodiment can also be directly deployed in the perception system 120 to output the high-precision positioning of the vehicle, or it can be deployed in the decision control system 130. The decision control system 130 locates the current position of the vehicle based on the data from the perception system, and controls the vehicle to realize the intelligent driving function of the vehicle based on the perceived target.

[0131] In addition to instruction 153, memory 152 may also store data such as road maps, route information, vehicle position, direction, speed, and other similar vehicle data, as well as other information. This information can be used by vehicle 100 and computing platform 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0132] The computing platform 150 can control the functions of the vehicle 100 based on inputs received from various subsystems, such as the drive system 140, the perception system 120, and the decision control system 130. For example, the computing platform 150 can utilize inputs from the decision control system 130 to control the steering system 133 to avoid obstacles detected by the perception system 120. In some embodiments, the computing platform 150 is operable to provide control over many aspects of the vehicle 100 and its subsystems.

[0133] Alternatively, one or more of these components may be installed separately from or associated with vehicle 100. For example, memory 152 may exist partially or completely separately from vehicle 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0134] Optionally, the above components are just an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 1 should not be construed as a limitation on the embodiments of this application.

[0135] The aforementioned vehicle 100 can be any vehicle or vehicle-mounted terminal capable of intelligent driving, such as a car, truck, motorcycle, bus, ship, airplane, helicopter, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, or train. This application embodiment does not impose any special limitations on this type of vehicle.

[0136] Traditional high-precision positioning solutions typically rely on laser positioning, vectorized map positioning, and signals from sensors such as GNSS, IMU, and wheel speedometers to achieve high-precision positioning capabilities. However, they are highly dependent on high-precision initial positioning, such as an error of less than 5 meters. In scenarios such as urban canyons or under overpasses, multipath propagation and occlusion issues can lead to larger initial positioning errors, such as deviations of up to 20 meters.

[0137] For example, commonly used positioning algorithms may include ICP algorithms or vector matching algorithms. In scenarios with low initial positioning accuracy, the laser point cloud ICP algorithm depends on the accuracy of the initial positioning. When the error exceeds a certain distance threshold, it cannot converge quickly. Or, vector feature matching has errors in matching geometric shapes, and it cannot quickly iterate and converge from large errors, thus failing to quickly complete high-precision positioning.

[0138] For example, point cloud matching implemented through the network (GeoTransformer) can overcome the problem of large distance thresholds, but it requires feature encoding of laser point clouds, which consumes a lot of computing power and has high latency, and places very high demands on the algorithms of the equipment deployed in practice.

[0139] For example, the network-based matching localization algorithm (BEV-Locator) takes unprocessed raw images or point cloud information as input, which has a large difference in modality from high-precision map vector information, making it difficult to train. Furthermore, the relocalization convergence range is small (<1m), the prediction accuracy is poor, and it cannot meet the actual relocalization requirements of more than 10m.

[0140] Therefore, this application provides a positioning method that can achieve high-precision positioning without limitation on the accuracy of the initial positioning. Even if the accuracy of the initial positioning is low, high-precision positioning can still be achieved. For example, it can quickly converge errors below 1m, and through further refinement, it can achieve decimeter-level errors, thus achieving very high-precision positioning.

[0141] The method flow provided in this application embodiment will be described below in conjunction with the aforementioned vehicle structure and application scenarios. Referring to Figure 2, a flowchart of a positioning method provided in this application embodiment is shown below.

[0142] 201. Obtain the first set of vector elements and the second set of vector elements.

[0143] The first set of vector elements includes at least one vector element from the first map. A vector element is a vector representation of a geometric figure, such as a vector used to represent line segments in the map. For ease of distinction, vector elements from the first map are referred to as first vector elements or map vector elements. The first map may specifically include a vectorized map, a high-precision map, a standard-precision map, or a vectorized map that falls between the two.

[0144] In one possible scenario, when the first map does not contain vector elements—for example, if the first map is a high-precision map, a standard-precision map, or a vectorized map that does not directly include unit vector elements—then vector elements can be extracted from the first map using a pre-set model or algorithm. For instance, a pre-trained model for vector recognition can be used to identify vector elements from the map.

[0145] The second set of vector elements includes at least one vector element from the data collected by the sensor. For ease of distinction, the vector elements from the data collected by the sensor are referred to as second vector elements or perception vector elements. Specifically, after acquiring the data collected by the sensor, vectors can be extracted from the data to obtain one or more second vector elements. For example, a pre-trained online map generation network can be set up, taking the data collected by the sensor as input and outputting a topology map constructed from vectors.

[0146] In addition, an initial location can be obtained, which represents the position of the target, i.e., the target that needs to be located. For example, if the method provided in this application embodiment is deployed in a vehicle, the target can be the vehicle; if the method provided in this application embodiment is deployed in a terminal, the target can be the terminal, and so on. Specifically, the initial location can be obtained using GNSS signals or using laser point clouds, etc.

[0147] 202. Obtain the offset based on the first vector element set and the second vector element set.

[0148] The offset can be used to represent the offset between the perceived map and the first map. The perceived map is a map formed by perceived vector elements and can be used to correct the initial positioning. Specifically, the offset can be the offset value of parameters in each dimension of the positioning. For example, if the initial positioning includes (x, y, θ), where (x, y) represents the target coordinates and θ represents the target angle, such as the heading angle or the angle with the axis, the corresponding offset can be expressed as (Δx, Δy, Δθ), where Δx represents the offset value of x, Δy represents the offset value of y, and Δθ represents the offset value of θ.

[0149] Specifically, the difference between the first vector set and the second vector set can be calculated, and the offset can be calculated based on this difference. This difference can typically represent the matching relationship between the map constructed from the sensing vector elements and the offline map. Based on this matching relationship, the error between the initial observed pose and the actual pose of the target can be calculated, so that the error can be used to correct the initial observed pose and improve the positioning accuracy.

[0150] In one possible implementation, a pre-trained feature extraction network can be used to obtain first vector features based on a first set of vector elements and second vector features based on a second set of vector elements; an offset can then be obtained based on the first and second vector features. In this embodiment, features can be extracted from the first and second sets of vector elements respectively, thereby calculating the difference between the first and second sets of vector elements to obtain a more accurate offset.

[0151] The methods for obtaining the first vector feature based on the first set of vector elements and the second vector feature based on the second set of vector elements can include various approaches. For example, features can be extracted from the encoding after encoding the vector elements, or features can be extracted directly from the vector elements using a GNN and then extracted from the features output by the GNN using a feature extraction network. Alternatively, vector elements can be converted into images and then features can be extracted using an image feature extraction network and then extracted from the features output by the image feature extraction network using a feature extraction network. The different feature extraction methods will be introduced below.

[0152] In one possible implementation, vector elements in a first set of vector elements can be encoded to obtain a first code, and vector elements in a second set of vector elements can be encoded to obtain a second code. Specifically, the encoding method can be to convert the vector elements into encoded data of a preset format. Then, through the aforementioned feature extraction network, features are extracted from the first code to obtain first vector features, and features are extracted from the second code to obtain second vector features. In this embodiment, vector elements can be encoded to convert them into more standardized encoded data, facilitating subsequent feature extraction.

[0153] In one possible implementation, the specific encoding method may include: obtaining a first vector code and a first geometric code based on the node information of the first vector elements in the first vector element set, wherein the first code includes the first vector code and the first geometric code, the first vector code represents at least one vector formed by adjacent nodes in the first vector element, and the first geometric code represents the distance or offset angle between adjacent nodes forming the vector in the first vector element; similarly, a second vector code and a second geometric code may be obtained based on the node information of the second vector elements in the second vector element set, wherein the second code includes the second vector code and the second geometric code, the second vector code represents at least one vector formed by adjacent nodes in the second vector element, and the second geometric code represents the distance or offset angle between adjacent nodes forming the vector in the second vector element.

[0154] In this embodiment, the encoding of vector elements can be divided into vector encoding and geometric encoding. Vector encoding can be used to represent individual nodes or line segments in a vector element, while geometric encoding can be used to represent the relationships between nodes in a vector element, so that the nodes form geometric relationships. This facilitates the extraction of vector features and geometric features from the vector dimension and geometric dimension during subsequent feature extraction, thereby enabling the features to more accurately represent the vector dimension and geometric dimension of the vector element.

[0155] In one possible implementation, features can be extracted from a first set of vector elements using a graph neural network to obtain first initial features, and features can be extracted from a second set of vector elements to obtain second initial features. A feature extraction network can then extract features from the first initial features to obtain first vector features, and extract features from the second initial features to obtain second vector features. In this embodiment, GNNs can be used to directly extract features from vectors, allowing downstream feature extraction networks to directly perform further vector feature extraction based on the features output by the GNN.

[0156] In one possible implementation, a first image corresponding to a first set of vector elements and a second image corresponding to a second set of vector elements can be obtained. For example, the first vector elements can be fused into the same image to obtain the first image, or the second vector elements can be fused into the same image to obtain the second image. Subsequently, an image feature extraction network is used to extract first image features from the first image and second image features from the second image. A feature extraction network is then used to extract features from the first image features to obtain first vector features, and features are extracted from the second image features to obtain second vector features. In this embodiment, vector elements can be converted into images and image features can be extracted, and further vector features can be extracted from the image features, thereby providing a method for extracting vector features.

[0157] Optionally, the aforementioned feature extraction network can specifically be a network that extracts features based on an attention mechanism.

[0158] In one possible implementation, the feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism.

[0159] For example, in one possible scenario, a self-attention module and a cross-attention module can be set up. The aforementioned first vector code, first initial feature, or first image feature is used as input to the self-attention module to extract the first feature based on the self-attention mechanism. Similarly, the aforementioned second vector code, second initial feature, or second image feature is used as input to the self-attention module to extract the second feature based on the self-attention mechanism, thereby extracting features within the vector using the self-attention mechanism. After inputting the first and second features into the cross-attention mechanism, the first and second vector features are further extracted based on the correlation between external features. Therefore, in this embodiment, features can be extracted based on an attention mechanism, focusing on the correlation between vectors and their internal structures, thereby extracting more accurate vector features.

[0160] In one possible implementation, after obtaining the first vector feature and the second vector feature, the first vector feature and the second vector feature can be fused to obtain a fused feature, such as by concatenating them into a fused feature; then the fused feature is input into the first localization model, and the offset is output. In this embodiment of the application, the first vector feature and the second vector feature can be fused and then input into the localization model, and the localization model can use the difference between the two features to identify the offset.

[0161] In one possible implementation, the fused features can also be input into a second localization model, and the confidence level of the offset can be output, such as the variance of the offset or other indicators representing the confidence level. In this embodiment of the application, the confidence level of the offset can be further output so that the accuracy of the offset can be determined by referring to the confidence level during subsequent localization.

[0162] 203. Obtain the first position based on the offset and the initial position.

[0163] The initial positioning can be corrected based on the offset to obtain the corrected positioning, which is called the first positioning for easy distinction. The accuracy of the first positioning is higher than that of the initial positioning.

[0164] In this embodiment, the difference between map vectors and perception vectors can be used to calculate the offset that measures the initial positioning deviation. This offset can then be used to correct the initial positioning, resulting in a more accurate positioning result. Furthermore, the method provided in this embodiment requires minimal computing power, achieving very high positioning efficiency.

[0165] Optionally, when correcting the initial positioning using the offset, the offset can be verified. If the verification passes, the initial positioning can be corrected using the offset. If the verification fails, the offset can be recalculated using the data of the next frame until a verified offset is calculated.

[0166] In one possible implementation, a second map can be obtained using the offset and a first map. For example, the first map can be corrected using the offset to obtain a corrected first map, i.e., the second map. Then, based on the difference between the second map and the first map, a verification index is obtained. This verification index can be used to indicate the usability of the offset. If the verification index meets preset verification conditions, the initial positioning can be updated based on the offset to obtain the first positioning. For example, if the difference between the first map and the second map is large, it may indicate that the accuracy of the offset is low, and the offset is unusable. If the difference between the first map and the second map is small, it may indicate that the accuracy of the offset is high, and the offset is usable. Therefore, in this embodiment, the calculated offset can be verified to check its accuracy. A more accurate offset can then be used to correct the initial positioning, resulting in a more accurate first positioning, such as quickly converging the error to below 1m.

[0167] Optionally, the verification indicators may include, but are not limited to, one or more of the following: confidence level of light flux or offset determined based on the first map and the second map.

[0168] For example, in one possible implementation, a first mask can be generated based on vector information in a first map, and a second mask can be generated based on vector information in a second map. For example, elements in the map that contain vectors and elements that do not contain vectors can be set to different values, such as 0 and 1, or positive and negative numbers, respectively. Subsequently, a light flux index is determined based on the first mask and the second mask. This light flux index includes the number of elements that the same ray passes through in both the first mask and the second mask.

[0169] For example, if the confidence level of the offset is output using the second positioning model, the confidence level can also be used as a verification index or one of the verification indices.

[0170] Optionally, preset conditions can be set to determine whether the offset passes the verification. For example, the preset conditions can be that the luminous flux exceeds a first threshold and / or the variance is less than a second threshold. Accordingly, if the calculated luminous flux index corresponding to the offset is greater than the threshold and / or the variance is less than the second threshold, the verification can be considered successful.

[0171] Furthermore, to further improve positioning accuracy, the first positioning can be used as the initial positioning. The offset is then calculated again based on the first and second vector element sets to update the first positioning and obtain the second positioning. This is iterative positioning, which allows for rapid convergence of positioning errors and further refinement to achieve higher positioning accuracy. For example, the positioning error can be quickly converged to below 1m through the first positioning correction, and further refined to 0.1m or lower. Generally, the more iterations of positioning correction, the higher the achievable accuracy, and the lower the error, until convergence. The specific number of iterations can be determined based on the actual application scenario.

[0172] Optionally, trigger conditions can be set for recalculating the offset to correct the first positioning. For example, if the accuracy of the first positioning is less than the third threshold, the error is greater than the fourth threshold, the variance is greater than the fifth threshold, or the light flux is less than the sixth threshold, or one or more of these conditions can be set to refine the first positioning and obtain a more granular positioning result.

[0173] The foregoing has described the method flow provided in this application. The following section provides a more detailed description of the method provided in the embodiments of this application, using specific application scenarios.

[0174] First, the method provided in this application can be applied to positioning scenarios, especially high-precision positioning scenarios. In the following embodiments, the method provided in this application will be described exemplarily using deployment on a vehicle as an example.

[0175] Typically, in a high-precision positioning system, multiple positioning sources can be set up, as shown in Figure 3. Positioning methods may include using GNSS signals for precise positioning, using point cloud maps and point cloud sensing data for laser positioning, and combining IMU or wheel speed recording for assisted positioning. The method of the system in this application embodiment can be deployed in the vector topology positioning module shown in Figure 3. The inputs include vector maps and vector sensing, etc. At the same time, in the high-precision positioning module, the offset output in the method provided in this application embodiment is also used to correct the initial positioning and output high-precision positioning.

[0176] The vector topology positioning module can run continuously after the vehicle is started, or it can run based on trigger conditions. For example, when the vehicle is started, the method provided in this application embodiment can be started for high-precision positioning; or, when trigger conditions are met, such as when the positioning error is greater than a preset error value, the initial positioning is within a preset area, or the initial positioning is in a preset type of scenario, etc., the specifics can be determined according to the actual application scenario, and this application does not limit it.

[0177] This application provides a vector topology map localization method based on neural networks. It constructs a matching relationship between online and offline vector maps, outputs the error between the initial observed pose and the actual pose of the vehicle based on the matching relationship, i.e., the offset, and uses the offset to correct the initial positioning to obtain a more accurate positioning result.

[0178] Taking the application on the vehicle as an example, as shown in Figure 4, the input of the vector topology positioning module can include vector features of the offline map and vector features perceived by the vehicle, and the output is the initial positioning offset and the confidence level of the offset.

[0179] Specifically, referring to Figure 5, the architecture of the method provided in this embodiment can be divided into multiple modules, such as data preprocessing 41, feature extraction 42, offset calculation 43, post-processing 44, or output 45. Optionally, the processing result can be corrected again. That is, after performing data preprocessing 41, feature extraction 42, offset calculation 43, and post-processing 44 once, the output can be used as the initial positioning to perform data preprocessing 41, feature extraction 42, offset calculation 43, post-processing 44, and output 45 again.

[0180] The detailed steps for each module are described below.

[0181] Phase 1: Data Preprocessing

[0182] The input data in this application embodiment can be divided into map vector elements and perception vector elements. The offline vector, namely the first vector mentioned above, usually comes from offline maps or maps constructed using perception data; the perception vector usually comes from data collected by sensors.

[0183] Map vector elements and perception vector elements can be represented as polylines, each of which includes multiple nodes (vertex), and adjacent nodes are connected to form a vector.

[0184] Optionally, a threshold P can be set for each map vector element. map Set a threshold P for the perceptual vector elements perception When the number of line segments in a map vector element exceeds P mapThen, some line segments can be discarded. This can be done randomly or according to pre-set conditions, such as discarding shorter line segments or other line segments. When the number of line segments in the perceptual vector element exceeds P... perception Alternatively, some line segments can be discarded, such as randomly discarding or discarding line segments that are far from the vehicle. Therefore, in this embodiment, the number of line segments in the input map vector elements and perception vector elements can be set, thereby reducing the amount of input data and the computing power required for subsequent feature extraction.

[0185] Optionally, a threshold V can be set for each map vector element. map Set a threshold V for the sensing vector elements. perception When the number of vectors in a map vector element is less than V map When new nodes are interpolated into map vector elements, the number of vectors in the map vector elements can be increased; when the number of vectors in the perceived vector elements is less than V... perception In this case, new nodes can be interpolated into the perceptual vector elements, thereby expanding the number of vectors in the perceptual vector elements. Therefore, when the amount of input data is insufficient, the vectors can be expanded to increase the amount of data and improve the accuracy of subsequent feature extraction.

[0186] In addition, to facilitate subsequent extraction of vector features, offline vectors and perceptual vectors can be preprocessed to convert them into input data that can be used by the subsequent feature extraction network.

[0187] In this application, various preprocessing methods can be used, such as vector encoding, vector feature extraction, or vector-to-image conversion followed by feature extraction. The different preprocessing methods are described below.

[0188] 1. Vector encoding

[0189] Vector coding is the process of encoding vectors and outputting encoded data in a preset format, such as map vector coding (the first type of coding mentioned above) and perceptual vector coding (the second type of coding mentioned above).

[0190] For example, as shown in Figure 6, for map vector elements and perception vector elements, the map vector elements and perception vector elements are encoded from the vector dimension and geometric dimension respectively, and the corresponding vector codes and geometric codes are output respectively.

[0191] For example, taking one type of vector as an example, and specifically a two-dimensional vector, the structures of vector encoding and geometric encoding can be as shown in Figure 7. This application embodiment only uses two-dimensional coordinates for illustrative purposes. In practical applications, the following two-dimensional coordinates can be replaced with multi-dimensional coordinates, and the included angle can be replaced with the corresponding angle in multi-dimensional space.

[0192] In this context, each Vector in the vector encoding can include [x] start ,y start ,x end ,y end [,type], representing [starting coordinates, ending coordinates, category], where categories include, but are not limited to, road elements such as curbs, lane lines, lights, poles, signs, road markings, and sidewalks. In multidimensional space, the coordinates of vectors can also have additional dimensions. For example, in three-dimensional space, each Vector can have an additional one-dimensional coordinate, such as [x...]. start ,y start ,z start ,x end ,y end ,z end [,type] means that the coordinate values ​​of the z-dimensional of the starting point and the ending point have been added.

[0193] In geometric encoding, the distance and angle between the current node and its previous neighbor node can be included, such as [distance, angle], where distance represents the aforementioned distance and angle represents the aforementioned angle. For example, in vehicle positioning, angle can represent the heading angle. In three-dimensional space, the angle can also include pitch and roll angles, etc. The specific angle representation can be determined according to the actual application scenario. This application only provides an illustrative example and does not constitute a limitation on the name or number of angles.

[0194] After performing vector encoding and geometric encoding, the vector and geometric codes can be concatenated to output the final vector-encoded data. For example, for [x... start ,y start ,x end ,y end The [,type] and [distance,angle] are concatenated to output encoded data with dimensions [P,V,7], where P represents the number of polylines and V represents the number of vectors in each polyline. The P and V values ​​for vector maps and vector-aware maps may not be equal.

[0195] Therefore, in this embodiment of the application, the vector can be encoded into encoded data that can represent the shape of the vector element, so that the features of the vector element can be extracted more accurately from the encoded data in the future.

[0196] 2. Vector Feature Extraction

[0197] In this embodiment of the application, GNN can be used to directly extract features from map vector elements and perception vector elements, and output the corresponding map vector features (i.e., the aforementioned first initial features) and perception vector features (i.e., the aforementioned second initial features). For example, as shown in Figure 8, map vector elements and perception vector elements are input into GNN, and map vector features and perception vector features are output to the feature extraction network.

[0198] In addition, the aforementioned vector encoding part can be combined. First, the map vector elements and perception vector elements are encoded and the encoded data is output. Then, the encoded data is input into the GNN to output map vector features and perception vector features.

[0199] For example, the structure of a GNN can be as shown in Figure 9. After encoding the vector input through an MLP layer, a pooling operation is performed, and then the input before pooling is concatenated to output the vector features.

[0200] Therefore, in the embodiments of this application, GNN can be used to extract the features of each node in the vector, and the features output by GNN can be used to more accurately represent the features of each node.

[0201] 3. Feature extraction is performed after vector-to-image conversion.

[0202] In this embodiment of the application, map vector elements and perception vector elements can be converted into images respectively, and the resulting images are referred to as the first image and the second image respectively.

[0203] For example, as shown in Figure 10, after rasterizing the map vector elements and the perception vector elements, the element values ​​in each vector element are used as pixel values ​​and fused into the initial image, resulting in a first image containing map vector elements and a second image containing perception vector elements.

[0204] The first image and the second image are used as inputs to the image feature extraction network, and the corresponding map vector features (i.e., the aforementioned first image features) and perceptual vector features (i.e., the aforementioned second image features) are output.

[0205] The image feature extraction network can be a network that includes convolutional layers, such as the mainstream CNN architecture ResNet-18.

[0206] Therefore, in this embodiment, map vector elements and perception vector elements can be converted into images and features can be extracted, thereby extracting features from the image dimension and outputting features that express vector elements from the image dimension.

[0207] Phase 2: Feature Extraction

[0208] In the aforementioned Phase 1, the output data can include various scenarios, such as map vector codes and perceptual vector codes, or map vector features and perceptual vector features. Further feature extraction can be performed on the map vector codes and perceptual vector codes or map vector features and perceptual vector features output from Phase 1 using an attention mechanism, outputting first vector features and second vector features.

[0209] For example, the structure of the feature extraction network used in this embodiment can be as shown in Figure 11. This feature extraction network can include a self-attention module and a cross-attention module. The self-attention module can be used to extract features based on a self-attention mechanism. The number of self-attention modules can be set to two, or one self-attention module can be set to perform feature extraction twice. Here, we take setting two self-attention modules as an example for illustrative purposes. Map vector encoded data or map vector features are used as input to one self-attention module, and perceptual vector encoded data or perceptual vector features are used as input to the other self-attention module. The outputs of the two self-attention modules are respectively connected to the input of the cross-attention module. The cross-attention module can be used to extract features from the two input features based on a cross-attention mechanism, outputting a first vector feature and a second vector feature.

[0210] For ease of understanding, the self-attention module can be understood as the encoder in a Transformer, and the cross-attention module can be understood as the decoder in a Transformer.

[0211] The encoder structure is shown in Figure 12. It uses a self-attention mechanism to extract features from both the vector map encoding and the vector perception encoding, providing input for the next step of information interaction.

[0212] The Decoder structure can be shown in Figure 13. It utilizes a cross-attention mechanism to associate the feature elements obtained by the encoder. Specifically, map features are used as the query, and perceptual features as the key and value of the query object, interacting with each other; simultaneously, perceptual features are used as the query, and map features as the key and value of the query object, interacting with each other. Ultimately, two feature results are obtained. The Transformer Decoder module can iterate N times, with the result of the previous iteration serving as the input for the next. For example, N=3 means repeating 3 times.

[0213] Therefore, in the embodiments of this application, the features of map vector elements and perception vector elements can be extracted by Transformer based on the attention mechanism, thereby expressing the map vector elements and perception vector elements more accurately.

[0214] Phase 3, Offset Calculation

[0215] After obtaining the features of map vector elements and perception vector elements, these two features can be used to calculate the offset.

[0216] Specifically, a pose decoder can be set up to identify the target's offset and the corresponding confidence level.

[0217] For example, the structure of a pose decoder can be shown in Figure 14. First, the two input features can be concatenated. The concatenated features are then input into two MLPs, which output offsets (Δx, Δy, Δθ) and variances [var_x, var_y, var_θ], respectively. The variance can represent the confidence level of the offset.

[0218] Phase 4: Post-processing

[0219] In some scenarios, the data collected by the sensors may be inaccurate or have large errors due to sensor obstruction or malfunction. Therefore, the calculated offset can be verified. If the verification is successful, it can be used to correct the initial positioning. If the verification fails, the offset can be calculated using the next frame of sensing data, so that the final output positioning result is more accurate.

[0220] For example, the corrected positioning result can be determined by combining the model's output and the initial positioning, such as (x+Δx,y+Δy,θ+Δθ). Alternatively, the initial positioning can be corrected by performing a matrix transformation on the offsets (Δx, Δy, Δθ) corresponding to each node in the vector elements. The specific method can be adjusted according to the actual application scenario.

[0221] As shown in Figure 15, the vector map is corrected by combining the corrected positioning results. The perceptual map formed by the perceptual vector elements is aligned with the corrected vector map, and both maps are rasterized for subsequent processing. Masks are generated for the rasterized vector map and the perceptual map respectively. Typically, raster areas containing vector elements in both maps are set to non-zero, while raster areas without vector elements are set to zero. The same light ray passing through the non-zero mask area is simulated; the raster that passes through can be considered as luminous flux. The luminous flux that can pass through both the perceptual map mask and the vector map mask is calculated. This can be used to represent the overlapping area between the two maps or the differences between them. Generally, the larger the luminous flux, the smaller the difference between the two maps; the smaller the luminous flux, the larger the difference between the two maps.

[0222] Then, the light flux index is used for verification. For example, if the light flux index exceeds the threshold p, the verification is considered to be successful, meaning that the accuracy of the offset output in the previous stage three is considered to be high and can be transmitted to downstream tasks. If the light flux index does not exceed the threshold p, the verification is considered to be unsuccessful, meaning that the accuracy of the offset output in the previous stage three is considered to be low. The offset calculation is then performed using the sensor's perception data from the next frame until a verified offset is output.

[0223] In addition, the variance of the offset, i.e. the confidence level, can be used to verify the offset. For example, if the variance is less than the threshold v and the luminous flux exceeds the threshold p, the verification can be considered passed. If the variance is not less than the threshold v and the luminous flux does not exceed the threshold p, the verification can be considered failed. The specific verification conditions can be set according to the actual application scenario. The embodiments of this application are only examples and are not intended to limit the scope.

[0224] Therefore, in the embodiments of this application, light flux can be used as a performance indicator of offset, thereby improving the reliability of the final output result and outputting a more accurate positioning result.

[0225] Phase 5, Output

[0226] After obtaining the corrected positioning result in the aforementioned post-processing stage, the positioning result can be output directly, or it can be used as the initial positioning to iteratively execute stages one to four. The number of iterations can be two or more, depending on the actual application scenario.

[0227] For example, in one possible scenario, the corrected positioning result after the verification in the aforementioned stage four can be directly output, or the corrected positioning result can be output after iterating through stages one to four a preset number of times.

[0228] In one possible scenario, the iteration can also terminate when output conditions are met. For example, these output conditions may specifically include conditions based on the aforementioned confidence level and / or luminous flux settings. For instance, these output conditions may include a variance less than a preset variance and / or luminous flux exceeding a preset luminous flux value, thus outputting a positioning result. Therefore, in this embodiment, the final positioning result can be output when the output conditions are met, resulting in a more accurate positioning result.

[0229] For example, the process of executing stages one through four once can be shown in Figure 16, and the process of two iterations can be shown in Figure 17. It is understandable that, since the matching range of a single stage is relatively large, such as a map area of ​​200m x 200m, the matching accuracy will be affected by inaccurate perception of distant maps. After obtaining the pose error through a coarse matching correction, a local map based on the corrected pose, such as 40m x 40m, is re-acquired for further fine matching to obtain a finer-grained pose offset, thereby achieving higher pose accuracy.

[0230] In this embodiment, the positioning offset can be determined based on the matching of vector features in map data and perceived data. Therefore, even in scenarios with large positioning errors, such as parking lot positioning or indoor scenes, the offset can be more accurately identified based on sensor perceived data, thereby correcting and obtaining a more accurate positioning result. Furthermore, the vector encoder based on a self-attention mechanism can achieve more effective vector feature extraction, and the interaction between perceived and map information based on a cross-attention mechanism can enable mutual querying and interaction of features from different data, improving pose regression performance. Moreover, through multiple iterations, a coarse-to-fine positioning method can be achieved, improving positioning accuracy.

[0231] The foregoing has described the method flow provided in the embodiments of this application. The following describes the structure of the apparatus for executing the foregoing method flow.

[0232] Referring to Figure 18, a schematic diagram of a positioning device provided in an embodiment of this application includes:

[0233] The acquisition module 1801 is used to acquire a first vector element set and a second vector element set, wherein the first vector element set includes at least one first vector element in the first map, and the second vector element set includes at least one second vector element in the data collected by the sensor.

[0234] Offset calculation module 1802 is used to obtain the offset based on the first vector element set and the second vector element set;

[0235] The positioning module 1803 is used to obtain the first positioning based on the offset and the initial positioning. The initial positioning is used to represent the initial positioning position of the target. The accuracy of the first positioning is higher than that of the initial positioning.

[0236] In one possible implementation, the offset calculation module 1802 is specifically used to: obtain a first vector feature based on a first set of vector elements and a second vector feature based on a second set of vector elements through a feature extraction network; and obtain an offset based on the first vector feature and the second vector feature.

[0237] In one possible implementation, the offset calculation module 1802 is specifically used for: encoding a first set of vector elements to obtain a first code, and encoding a second set of vector elements to obtain a second code; extracting features from the first code to obtain a first vector feature through a feature extraction network, and extracting features from the second code to obtain a second vector feature.

[0238] In one possible implementation, the offset calculation module 1802 is specifically used for: obtaining a first vector code and a first geometric code based on the node information of the first vector elements in the first vector element set, wherein the first code includes the first vector code and the first geometric code, the first vector code represents at least one vector formed by adjacent nodes in the first vector element, and the first geometric code represents the distance or offset angle between adjacent nodes forming the vector in the first vector element; and obtaining a second vector code and a second geometric code based on the node information of the second vector elements in the second vector element set, wherein the second code includes the second vector code and the second geometric code, the second vector code represents at least one vector formed by adjacent nodes in the second vector element, and the second geometric code represents the distance or offset angle between adjacent nodes forming the vector in the second vector element.

[0239] In one possible implementation, the offset calculation module 1802 is specifically used to: extract features from a first set of vector elements through a graph neural network to obtain a first initial feature, and extract features from a second set of vector elements to obtain a second initial feature; and extract features from the first initial feature through a feature extraction network to obtain a first vector feature, and extract features from the second initial feature to obtain a second vector feature.

[0240] In one possible implementation, the offset calculation module 1802 is specifically used to: obtain a first image corresponding to a first set of vector elements and a second image corresponding to a second set of vector elements; extract first image features from the first image and extract second image features from the second image through an image feature extraction network; extract features from the first image features to obtain a first vector feature through a feature extraction network, and extract features from the second image features to obtain a second vector feature.

[0241] In one possible implementation, the feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism.

[0242] In one possible implementation, the offset calculation module 1802 is specifically used to: fuse the first vector feature and the second vector feature to obtain the fused feature; input the fused feature into the first positioning model and output the offset.

[0243] In one possible implementation, the offset calculation module 1802 is further configured to: input the fused features into the second localization model and output the confidence level of the offset.

[0244] In one possible implementation, the aforementioned apparatus further includes: a post-processing module 1804, configured to: obtain a second map using the offset and a first map; and obtain a verification index based on the difference between the second map and the first map, the verification index being used to indicate the availability of the offset;

[0245] The positioning module 1803 is specifically used to update the initial positioning based on the offset to obtain the first positioning when the verification index meets the preset verification conditions.

[0246] In one possible implementation, the post-processing module 1804 is specifically used to: obtain a first mask corresponding to a first map and a second mask corresponding to a second map; determine a light flux index based on the first mask and the second mask, wherein the light flux index includes the number of element points through which the same ray passes from both the first mask and the second mask, and the verification index includes the light flux index.

[0247] In one possible implementation, when obtaining the confidence level of the offset, the verification metric also includes the confidence level of the offset.

[0248] In one possible implementation, the positioning module 1803 is further configured to input the first positioning as an initial positioning input to the acquisition module 1801, so as to update the first positioning to obtain the second positioning after calculating the offset again based on the first vector element set and the second vector element set.

[0249] Figure 19 shows a schematic diagram of the hardware structure of a positioning device 190 provided in an embodiment of this application. This positioning device 190 can be used to implement the steps of the methods shown in Figures 2 to 17.

[0250] The positioning device 190 shown in Figure 19 may include a processor 1901, a memory 1902, a communication interface 1903, and a bus 1904. The processor 1901, the memory 1902, and the communication interface 1903 can be connected via the bus 1904.

[0251] The processor 1901 is the control center of the positioning device 190. It can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor, such as a GPU or NPU, and can be adapted to the actual application scenario.

[0252] As an example, processor 1901 may include one or more CPUs, and may also include other processors, such as the CPU, NPU or GPU shown in Figure 19.

[0253] The memory 1902 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0254] In one possible implementation, the memory 1902 may exist independently of the processor 1901. The memory 1902 can be connected to the processor 1901 via a bus 1904 and is used to store data, instructions, or program code. When the processor 1901 calls and executes the instructions or program code stored in the memory 1902, it can implement the methods provided in the embodiments of this application, such as the methods shown in Figures 2 to 17.

[0255] In another possible implementation, the memory 1902 can also be integrated with the processor 1901.

[0256] The communication interface 1903 is used for the positioning device 190 to connect with other devices via a communication network, which may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 1903 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0257] Bus 1904 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 19, but this does not indicate that there is only one bus or one type of bus.

[0258] It should be noted that the structure shown in FIG19 does not constitute a limitation on the positioning device 190. In addition to the components shown in FIG19, the positioning device 190 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0259] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0260] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0261] This application also provides a computer-readable storage medium storing a program for training a model or performing inference tasks, which, when run on a computer, causes the computer to perform all or part of the steps in the methods described in the embodiments shown in Figures 2 to 17 above.

[0262] This application also provides a digital processing chip. This digital processing chip integrates circuitry for implementing the aforementioned processor or processor functions, and one or more interfaces. When the digital processing chip integrates a memory, it can perform the method steps of any one or more of the foregoing embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the method steps of any one or more of the foregoing embodiments based on the program code stored in the external memory.

[0263] This application also provides a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0264] The positioning device provided in this application embodiment can be a chip, which includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip in the positioning device to execute the methods described in the embodiments shown in Figures 2 to 17 above. Optionally, the storage unit is a storage unit within the chip, such as a register or cache. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, such as random access memory (RAM).

[0265] Specifically, the aforementioned processing unit or processor can be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0266] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0267] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0268] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0269] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0270] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. The term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. The naming or numbering of steps in this application does not imply that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved. The division of modules in this application is a logical division. In actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the modules shown or discussed may be through some ports, and the indirect coupling or communication connection between modules may be electrical or other similar forms, which are not limited in this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules. Some or all of the modules can be selected to achieve the purpose of the solution in this application according to actual needs.

Claims

1. A positioning method, characterized in that, include: Obtain a first set of vector elements and a second set of vector elements, wherein the first set of vector elements includes at least one first vector element from the first map, and the second set of vector elements includes at least one second vector element from the data collected by the sensor; The offset is obtained based on the first vector element set and the second vector element set; A first positioning is obtained based on the offset and the initial positioning, wherein the initial positioning is used to represent the initial positioning position of the target, and the accuracy of the first positioning is higher than the accuracy of the initial positioning.

2. The method according to claim 1, characterized in that, The step of obtaining the offset based on the first vector element set and the second vector element set includes: The feature extraction network obtains a first vector feature based on the first vector element set and a second vector feature based on the second vector element set. The offset is obtained based on the first vector feature and the second vector feature.

3. The method according to claim 2, wherein obtaining the first vector feature based on the first set of vector elements and obtaining the second vector feature based on the second set of vector elements comprises: Encode the first set of vector elements to obtain a first code, and encode the second set of vector elements to obtain a second code; The feature extraction network extracts features from the first encoding to obtain the first vector feature, and extracts features from the second encoding to obtain the second vector feature.

4. The method according to claim 3, characterized in that, The process of encoding the first set of vector elements to obtain a first code, and encoding the second set of vector elements to obtain a second code, includes: Based on the node information in the first vector element in the first vector element set, a first vector code and a first geometric code are obtained. The first code includes the first vector code and the first geometric code. The first vector code represents at least one vector formed by adjacent nodes in the first vector element, and the first geometric code represents the distance or offset angle between adjacent nodes forming the vector in the first vector element. Based on the node information in the second vector element in the second vector element set, a second vector code and a second geometric code are obtained. The second code includes the second vector code and the second geometric code. The second vector code represents at least one vector formed by adjacent nodes in the second vector element, and the second geometric code represents the distance or offset angle between adjacent nodes forming the vector in the second vector element.

5. The method according to claim 2, characterized in that, The steps of obtaining the first vector feature based on the first set of vector elements and obtaining the second vector feature based on the second set of vector elements include: Features are extracted from the first set of vector elements using a graph neural network to obtain first initial features, and features are extracted from the second set of vector elements to obtain second initial features; The feature extraction network extracts features from the first initial features to obtain the first vector feature, and extracts features from the second initial features to obtain the second vector feature.

6. The method according to claim 2, characterized in that, The steps of obtaining the first vector feature based on the first set of vector elements and obtaining the second vector feature based on the second set of vector elements include: Obtain the first image corresponding to the first set of vector elements, and the second image corresponding to the second set of vector elements; A first image feature is extracted from the first image and a second image feature is extracted from the second image using an image feature extraction network. The feature extraction network extracts features from the first image features to obtain the first vector feature, and extracts features from the second image features to obtain the second vector feature.

7. The method according to any one of claims 2-6, characterized in that, The feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism.

8. The method according to any one of claims 2-7, characterized in that, The step of obtaining the offset based on the first vector feature and the second vector feature includes: The first vector feature and the second vector feature are fused to obtain the fused feature; The fused features are input into the first localization model, and the offset is output.

9. The method according to claim 8, characterized in that, The method also includes: The fused features are input into the second localization model, and the confidence level of the offset is output.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The second map is obtained using the offset and the first map; Based on the differences between the second map and the first map, a verification index is obtained, which is used to indicate the availability of the offset; The step of obtaining the first positioning based on the offset and the initial positioning includes: If the verification index meets the preset verification conditions, the initial positioning is updated according to the offset to obtain the first positioning.

11. The method according to claim 10, characterized in that, The step of obtaining verification indicators based on the differences between the second map and the first map includes: Obtain the first mask corresponding to the first map and the second mask corresponding to the second map; A light flux index is determined based on the first mask and the second mask. The light flux index includes the number of element points through which the same ray passes from both the first mask and the second mask. The verification index includes the light flux index.

12. The method according to claim 11, characterized in that, When obtaining the confidence level of the offset, the verification metric also includes the confidence level of the offset.

13. The method according to any one of claims 1-12, characterized in that, The method further includes: Using the first location as the initial location, the offset is calculated again based on the first vector element set and the second vector element set, and then the first location is updated to obtain the second location.

14. A positioning device, characterized in that, include: The acquisition module is used to acquire a first vector element set and a second vector element set, wherein the first vector element set includes at least one first vector element in the first map, and the second vector element set includes at least one second vector element in the data collected by the sensor. An offset calculation module is used to obtain the offset based on the first vector element set and the second vector element set; A positioning module is used to obtain a first positioning based on the offset and an initial positioning, wherein the initial positioning represents the initial positioning position of the target, and the accuracy of the first positioning is higher than the accuracy of the initial positioning.

15. The apparatus according to claim 14, characterized in that, The offset calculation module is specifically used for: The feature extraction network obtains a first vector feature based on the first vector element set and a second vector feature based on the second vector element set. The offset is obtained based on the first vector feature and the second vector feature.

16. The apparatus according to claim 15, wherein the offset calculation module is specifically used for: Encode the first set of vector elements to obtain a first code, and encode the second set of vector elements to obtain a second code; The feature extraction network extracts features from the first encoding to obtain the first vector feature, and extracts features from the second encoding to obtain the second vector feature.

17. The apparatus according to claim 16, characterized in that, The offset calculation module is specifically used for: Based on the node information in the first vector element in the first vector element set, a first vector code and a first geometric code are obtained. The first code includes the first vector code and the first geometric code. The first vector code represents at least one vector formed by adjacent nodes in the first vector element, and the first geometric code represents the distance or offset angle between adjacent nodes forming the vector in the first vector element. Based on the node information in the second vector element in the second vector element set, a second vector code and a second geometric code are obtained. The second code includes the second vector code and the second geometric code. The second vector code represents at least one vector formed by adjacent nodes in the second vector element, and the second geometric code represents the distance or offset angle between adjacent nodes forming the vector in the second vector element.

18. The apparatus according to claim 15, characterized in that, The offset calculation module is specifically used for: Features are extracted from the first set of vector elements using a graph neural network to obtain first initial features, and features are extracted from the second set of vector elements to obtain second initial features; The feature extraction network extracts features from the first initial features to obtain the first vector feature, and extracts features from the second initial features to obtain the second vector feature.

19. The apparatus according to claim 15, characterized in that, The offset calculation module is specifically used for: Obtain the first image corresponding to the first set of vector elements, and the second image corresponding to the second set of vector elements; A first image feature is extracted from the first image and a second image feature is extracted from the second image using an image feature extraction network. The feature extraction network extracts features from the first image features to obtain the first vector feature, and extracts features from the second image features to obtain the second vector feature.

20. The apparatus according to any one of claims 15-19, characterized in that, The feature extraction network includes at least one of a self-attention module or a cross-attention module. The self-attention module is used to extract features from the input data based on an attention mechanism, and the cross-attention module is used to extract features from at least two sets of input data based on a cross-attention mechanism.

21. The apparatus according to any one of claims 15-20, characterized in that, The offset calculation module is specifically used for: The first vector feature and the second vector feature are fused to obtain the fused feature; The fused features are input into the first localization model, and the offset is output.

22. The apparatus according to claim 21, characterized in that, The offset calculation module is also used for: The fused features are input into the second localization model, and the confidence level of the offset is output.

23. The apparatus according to any one of claims 14-22, characterized in that, The device further includes: a post-processing module, used for: The second map is obtained using the offset and the first map; Based on the differences between the second map and the first map, a verification index is obtained, which is used to indicate the availability of the offset; The positioning module is specifically used for: If the verification index meets the preset verification conditions, the initial positioning is updated according to the offset to obtain the first positioning.

24. The apparatus according to claim 23, characterized in that, The post-processing module is specifically used for: Obtain the first mask corresponding to the first map and the second mask corresponding to the second map; A light flux index is determined based on the first mask and the second mask. The light flux index includes the number of element points through which the same ray passes from both the first mask and the second mask. The verification index includes the light flux index.

25. The apparatus according to claim 24, characterized in that, When obtaining the confidence level of the offset, the verification metric also includes the confidence level of the offset.

26. The apparatus according to any one of claims 14-25, characterized in that, The device further includes: The positioning module is further configured to input the first positioning as an initial positioning into the acquisition module, so as to update the first positioning to obtain the second positioning after calculating the offset again based on the first vector element set and the second vector element set.

27. A positioning device, characterized in that, The method includes a processor coupled to a memory storing a program, wherein when the program instructions stored in the memory are executed by the processor, the method steps of any one of claims 1-13 are implemented.

28. A vehicle, characterized in that, It includes a processor and a memory, the memory storing a program that, when executed by the processor, implements the steps of the method according to any one of claims 1-13.

29. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the steps of the method as claimed in any one of claims 1-13.

30. A computer program product, characterized in that, The computer program product includes software code for performing the steps of the method as described in any one of claims 1-13.

Images