Online vector map construction method and device, storage medium and program product

Abstract

The invention relates to the technical field of automatic driving perception, and provides an online vector map construction method and device, a storage medium and a program product. The map construction method comprises the steps of performing reliability analysis on a vehicle laser radar point cloud and a multi-view image, and fusing the laser radar point cloud and the multi-view image into a fused BEV feature based on a reliability analysis result; generating point column data for representing map elements based on the historical high-precision map; performing cross attention mechanism processing on the fused BEV features and the point column data to obtain current map elements; and processing the historical high-precision map based on the current map element to obtain a high-precision vector map. According to the technical scheme, the point cloud and the image can be fused based on the reliability analysis result of the laser radar point cloud and the multi-view image, so that the influence of unreliable data points on the whole data can be automatically reduced when the visual field is limited or the sensor data is abnormal, and the accuracy of vector map construction can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving perception technology, and in particular to an online vector map construction method, device, storage medium, and program product. Background Art

[0002] With the rapid development of artificial intelligence (AI), autonomous driving technology is facing increasingly stringent requirements in terms of perception accuracy and path planning. Reliable 3D object detection is essential for autonomous driving. Fusion of multiple sensors facilitates precise environmental perception. For example, LiDAR provides precise 3D spatial information, while cameras provide rich semantic information. Fusion of these two types of data can enhance detection accuracy through complementary information. Furthermore, autonomous driving systems require precise decision-making capabilities. High-precision maps not only provide rich semantic information but also accurately reflect the road environment, providing strong data support for autonomous vehicles' path planning and decision-making.

[0003] In terms of 3D target detection, the existing LiDAR-camera fusion model is very sensitive to sensor failures. When one modality is damaged, it will severely degrade the performance of the other modality, thereby compromising the overall target detection accuracy and robustness. In terms of high-precision map construction, traditional methods usually rely on LiDAR point cloud data and use SLAM technology to build maps. After that, a large amount of human resources are required for semantic annotation and maintenance. In addition, this method has poor scalability and slow data update speed, making it difficult to reflect changes in the road environment in real time. Although end-to-end high-definition map construction technology has gradually received attention in recent years, there are still problems such as inaccurate coordinate prediction of the initial reference point and high computational complexity. It is difficult to meet the needs of large-scale queries and therefore needs improvement. Summary of the Invention

[0004] In order to help improve the accuracy of three-dimensional target detection, the present application provides an online vector map construction method, device, storage medium and program product.

[0005] In a first aspect, the present application provides an online vector map construction method, which adopts the following technical solutions: An online vector map construction method, the method comprising: Performing reliability analysis on the lidar point cloud collected by the on-board lidar and the multi-view images collected by the on-board camera, and fusing the lidar point cloud and the multi-view images into a fused BEV feature based on the reliability analysis result; Generate point array data for representing map elements based on historical high-precision maps; Fusing the fused BEV features with the point array data to obtain a current map element; The historical high-precision map is processed based on the current map elements to obtain a high-precision vector map.

[0006] By adopting the above technical solution, the lidar point cloud and multi-view images can be fused based on the reliability analysis results of the lidar point cloud and multi-view images. This can automatically reduce the impact of unreliable data points on the overall data when the field of view is limited or the sensor data is abnormal, thereby improving the accuracy of the data, which in turn can help improve the accuracy of vector map construction.

[0007] Optionally, the performing reliability analysis on the lidar point cloud collected by the on-board lidar and the multi-view image collected by the on-board camera, and fusing the lidar point cloud and the multi-view image into a fused BEV feature based on the reliability analysis result, includes: Projecting the point cloud features corresponding to the lidar point cloud and the image features corresponding to the multi-view images into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features; Analyzing the quality of the point cloud embedding feature and the image embedding feature by using a contrast loss function to obtain a point cloud confidence score corresponding to the point cloud embedding feature and an image confidence score corresponding to the image embedding feature; weighting the image embedding features based on the point cloud confidence score to obtain weighted image embedding features; weighting the point cloud embedding features based on the image confidence score to obtain weighted point cloud embedding features; The weighted image embedding feature and the weighted point cloud embedding feature are fused to obtain the fused BEV feature.

[0008] By adopting the above technical solution, the quality of features of different modalities can be taken into consideration during the feature fusion process, which can help improve the accuracy of the final fused BEV features.

[0009] Optionally, generating point sequence data for representing map elements based on historical high-precision maps includes: Initializing a central anchor point and an anchor neighborhood corresponding to the central anchor point based on the historical high-precision map, generating a non-central anchor point corresponding to the central anchor point in the anchor neighborhood, and constructing an anchor query unit based on the central anchor point and the non-central anchor point; grouping the anchor query units based on instances of map elements in the historical high-precision map, and performing intra-group feature interaction and inter-group feature interaction on the anchor query unit queries to update the anchor query units; The point sequence data is generated based on the updated anchor query unit.

[0010] By adopting the above technical solution, the physical location information can be explicitly combined by introducing the anchor neighborhood, which can help improve the accuracy of coordinate prediction, enhance the network's understanding of the location prior information, and thus help improve the accuracy of the final generated point column data.

[0011] Optionally, grouping the anchor query units and performing intra-group feature interaction and component feature interaction on the queries includes: For each query unit, fitting the query center anchor point to the target point and fitting the non-center anchor points to random points in the ground truth neighborhood; For each query unit group, splitting the local query of the query unit group to obtain local features; Performing inter-group feature interaction on the local features of each query unit group to obtain local features after inter-group interaction; The elements in the local features after the inter-group interaction are distributed to the corresponding query unit groups to achieve intra-group feature interaction.

[0012] By adopting the above technical solution, the ground truth neighborhood is introduced in the process of feature interaction. Since the ground truth neighborhood is a neighborhood representation based on the real target position, it provides the model with a clearer fitting target by combining each real target with the surrounding information. This not only helps to alleviate the ambiguity problem in the fitting process, but also enhances the robustness of the model, enabling the model to more accurately identify and locate targets when facing complex scenes.

[0013] Optionally, performing a cross-attention mechanism on the fused BEV features and the point sequence data to obtain the current map element includes: Using a cross attention mechanism, the fused BEV features and the point sequence data interact to obtain an interaction result; performing feature fusion on the interaction result and the point sequence data to obtain fused data; The fused data is input into a multi-layer perceptron to obtain the current map element.

[0014] By adopting the above technical solution, the correlation relationship between data of different modalities can be captured before data fusion and taken into consideration during the data fusion process, which can help improve the accuracy of the current map elements finally obtained.

[0015] Optionally, the processing the historical high-precision map based on the current map element to obtain a high-precision vector map includes: Aligning the current map element with the historical high-precision map; Optimizing the current map elements by optimizing the loss function so that the optimized map elements are more consistent with the historical high-precision map; The historical high-precision map is corrected based on the optimized map elements to obtain the high-precision vector map.

[0016] By adopting the above technical solution, the current map elements can be optimized by optimizing the loss function, so that the map elements used to update the historical high-precision map can be more accurate, which can help to further improve the accuracy of the high-precision vector map.

[0017] Optionally, the optimization loss function includes at least one of a center point loss term, a non-center point loss term, a segmentation loss term, and a depth loss term; The center point loss term is used to supervise the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-precision map; The non-center point loss term is used to supervise the alignment between the non-center point of the current map element and the non-center point of the corresponding map element in the historical high-precision map; The segmentation loss is used to enhance the semantic segmentation accuracy of the area corresponding to the current map element; The depth estimation loss is used to enhance the depth information representation of the current map element.

[0018] In a second aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute any one of the line vector map construction methods provided in the first aspect.

[0019] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed in a computer, the computer is caused to execute any one of the online vector map construction methods provided in the first aspect.

[0020] In a fourth aspect, the present application provides a computer program product that adopts the following technical solution: A computer program product includes computer instructions, characterized in that when the computer program product is run on a computer device, the computer device is caused to execute any one of the online vector map construction methods provided in the first aspect.

[0021] In summary, this application includes at least one of the following beneficial technical effects: 1. The LiDAR point cloud and multi-view image can be fused based on their reliability analysis results. This can automatically reduce the impact of unreliable data points on the overall data when the field of view is limited or the sensor data is abnormal, thereby improving the accuracy of the data and, in turn, helping to improve the accuracy of vector map construction.

[0022] 2. By introducing the anchor neighborhood to explicitly combine physical location information, it can help improve the accuracy of coordinate prediction and enhance the network's understanding of location prior information, which in turn can help improve the accuracy of the final generated point sequence data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of an online vector map construction method provided in an embodiment of the present application; Figure 2 This is a flow chart of a method for generating fused BEV features provided in an embodiment of the present application; Figure 3 This is a flow chart of a method for generating point array data provided by an embodiment of the present application; Figure 4 This is a flow chart of an anchor query unit update method provided in an embodiment of the present application; Figure 5 This is a flowchart of a method for determining a current map element provided by an embodiment of the present application; Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-6 It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0025] First, several terms involved in the embodiments of the present application are introduced.

[0026] Bird's-Eye-View (BEV) is a sensor data representation method that looks down at the ground from a high altitude and is widely used in the fields of autonomous driving and intelligent transportation.

[0027] The Cross-Attention Mechanism is a deep learning technique for sequence modeling that allows the model to simultaneously focus on different parts of the input and query sequences when processing sequential data. In image fusion tasks, the Cross-Attention Mechanism allows the model to learn the relationships between different input sequences or feature maps, helping to capture the dependencies between global and local information.

[0028] An embodiment of the present application discloses an online vector map construction method for use in an electronic device. The electronic device may be a controller provided on a vehicle, or may be a server in communication with the vehicle controller, as long as data calculation and processing can be achieved.

[0029] Reference Figure 1 ,The online vector map construction method includes the following steps: Step 100: Perform reliability analysis on the lidar point cloud collected by the on-board lidar and the multi-view image collected by the on-board camera, and fuse the lidar point cloud and the multi-view image into a fused BEV feature based on the reliability analysis result.

[0030] The fused BEV features refer to the features represented by the fused data on the bird's-eye view. Specifically, due to the differences in spatial representation between LiDAR point clouds and multi-view images, to facilitate the fusion of data from different modalities, both LiDAR point cloud data and multi-view images can be mapped into the bird's-eye view to obtain unified fused BEV features.

[0031] During the research process, we considered that traditional multimodal data fusion models are very sensitive to sensor failures. When the data of one modality is damaged, it will seriously affect the performance of other modalities, thereby compromising the accuracy and robustness of overall target detection. Based on this, in this embodiment, before fusing the data of different modalities, a reliability analysis is first performed on the data of different modalities. Then, based on the reliability analysis results, the data of different modalities are fused. This can automatically reduce the impact of unreliable data points on the overall data when the field of view is limited or the sensor data is abnormal, thereby improving data accuracy.

[0032] In one example, reliability analysis is achieved by quantifying the reliability scores of data of different modalities through cross-modal comparison and confidence-weighted cross-attention. Accordingly, the reliability analysis results include reliability scores (e.g., confidence levels) corresponding to data of different modalities. In this way, in the process of feature fusion, the weights of different modal information can be dynamically characterized based on the reliability scores.

[0033] Furthermore, before performing reliability analysis on the lidar point cloud collected by the on-board lidar and the multi-view images collected by the on-board camera, it is also necessary to aggregate the lidar point cloud and multi-view images in both time and space dimensions to improve the spatiotemporal consistency of features, which can help improve the accuracy of processing moving objects in dynamic scenes.

[0034] Step 200: Generate point sequence data for representing map elements based on historical high-precision maps.

[0035] Among them, historical high-precision maps are built based on pre-collected information, including information related to vehicle driving.

[0036] Map elements are used for elements related to vehicles and vehicle driving, such as road information, road components, traffic signs, etc.

[0037] Point array data is used for the geographic location information of map elements.

[0038] In one example, point sequence data is obtained by fitting map elements on a historical high-precision map through anchor points.

[0039] In step 300 , the fused BEV features and point column data are processed using a cross-attention mechanism to obtain the current map element.

[0040] Among them, the current map elements are map elements determined by combining the real-time collected lidar point cloud and multi-view images. Since the dynamic characteristics of the current environment are taken into account, the current map elements may differ from the map elements in the historical high-precision map.

[0041] In one example, the current map element is represented by a vector.

[0042] Step 400: Process the historical high-precision map based on the current map elements to obtain a high-precision vector map.

[0043] In one example, a historical high-precision map is processed based on current map elements to obtain a high-precision vector map, including: aligning the current map elements with the historical high-precision map; optimizing the current map elements by optimizing the loss function so that the optimized map elements are more consistent with the historical high-precision map; and correcting the historical high-precision map based on the optimized map elements to obtain a high-precision vector map.

[0044] Among them, the purpose of aligning the current map elements with the historical high-precision map is to ensure the consistency of the map elements. If the current map element cannot find a matching element in the historical high-precision map, it will be marked as a new element.

[0045] The process of optimizing the current map element specifically includes: after using the optimization loss function to calculate the loss value, optimizing the loss function through the back propagation algorithm and updating the network parameters to continuously improve the accuracy of the optimized map element.

[0046] The method of correcting the historical high-precision map specifically includes: adding new map elements that are not in the historical high-precision map to the historical high-precision map, correcting erroneous map elements that are not in the historical high-precision map to obtain an updated map, and converting the updated map into vector form to obtain a high-precision vector map.

[0047] In one example, the optimization loss function includes at least one of a center point loss term, a non-center point loss term, a segmentation loss term, and a depth loss term.

[0048] Among them, the center point loss term is used to supervise the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-precision map.

[0049] The non-center point loss term is used to supervise the alignment between the non-center points of the current map element and the non-center points of the corresponding map elements in the historical high-precision map to reduce fitting ambiguity.

[0050] The segmentation loss is used to enhance the semantic segmentation accuracy of the area corresponding to the current map element to enhance the representation of the map element area in the BEV feature.

[0051] The depth estimation loss is used to enhance the depth information representation of the current map element to improve the accuracy of the depth information.

[0052] In actual implementation, parameters are dynamically adjusted according to the map element category (such as lane lines have higher weights) or scene dynamics (such as increasing the timing loss weight when vehicles are dense).

[0053] The implementation principle of an online vector map construction method in an embodiment of the present application is as follows: reliability analysis is performed on the lidar point cloud collected by the on-board lidar and the multi-view images collected by the on-board camera, and based on the reliability analysis results, the lidar point cloud and the multi-view images are fused into fused BEV features; based on the historical high-precision map, point column data for representing map elements is generated; the fused BEV features and the point column data are processed by a cross-attention mechanism to obtain the current map elements; based on the current map elements, the historical high-precision map is processed to obtain a high-precision vector map. By adopting the above technical solution, the lidar point cloud and the multi-view images can be fused based on the reliability analysis results of the lidar point cloud and the multi-view images. In this way, the impact of unreliable data points on the overall data can be automatically reduced when the field of view is limited or the sensor data is abnormal, thereby improving the accuracy of the data, which in turn can help improve the accuracy of vector map construction.

[0054] In some embodiments, reference Figure 2 , step 100, performing reliability analysis on the lidar point cloud collected by the vehicle-mounted lidar and the multi-view image collected by the vehicle-mounted camera, and fusing the lidar point cloud and the multi-view image into a fused BEV feature based on the reliability analysis result, including: In step 110 , the point cloud features corresponding to the lidar point cloud and the image features corresponding to the multi-view image are projected into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features.

[0055] In one example, the point cloud feature z LiDAR It is the BEV feature generated by processing the lidar point cloud using a 3D backbone network.

[0056] In another example, the image feature z Camera After processing multi-view images using a 2D convolutional backbone to obtain initial image features, the initial image is converted into features in the BEV space through a "lift-splash-shoot" operation. Specifically, the conversion method involves inferring the position of each pixel in the initial image in 3D space, projecting the initial image onto a top-down plane, and integrating the optimized projection into the BEV space.

[0057] Furthermore, before projecting the point cloud features corresponding to the LiDAR point cloud and the image features corresponding to the multi-view images into a shared embedding space to obtain the point cloud embedding features corresponding to the point cloud features and the image embedding features corresponding to the image features, the method also includes: processing the image features using a temporal attention mechanism and a spatial attention mechanism to increase the spatial consistency and cross-frame temporal dependency of the multi-view features. Accordingly, in subsequent steps, the processed image features are used instead of the original image features for fusion calculation.

[0058] Specifically, for each time step t, the image features are flattened and projected into the embedding space to generate the embedded features Through the spatial self-attention mechanism, the features of each view are interacted with the features of other views, the attention weights are calculated, and the spatial enhanced features S are generated. t At different time steps, the attention weights of different time steps are calculated through the temporal attention mechanism, capturing temporal dependencies and generating spatiotemporal aggregate features to achieve spatiotemporal aggregation of image features. Furthermore, to stabilize training, each attention mechanism is followed by layer normalization and residual connections.

[0059] In one example, a multi-view image refers to an image captured by six camera perspectives of a vehicle-mounted camera. First, for each view (as a query vector Q k), calculate its value with all other views j (as key vector K j Sum value vector V j )’s similarity: in, (W1, W2, W3 are learnable parameters, are the embedding features of different perspectives k and j at the same time step t), and d represents the embedding dimension.

[0060] Then, the weighted sum of the values V of all views is j , we get the spatial enhancement features of view k: Finally, the features of all views are merged to obtain the global spatial features at time step t: In step 120 , the quality of the point cloud embedding features and the image embedding features is analyzed by using a comparative loss function to obtain a point cloud confidence score corresponding to the point cloud embedding features and an image confidence score corresponding to the image embedding features.

[0061] Specifically, during the quality analysis process, undamaged data pairs are encouraged to move closer in the future space, while damaged data pairs are pushed further away. In this way, by analyzing the quality of the embedding, a multi-layer perceptron and Sigmoid function are used to generate a point cloud confidence score C. LiDAR and image confidence score C Camera .

[0062] The specific process is: prepare positive sample pairs (undamaged LiDAR and camera features in the same frame) and negative sample pairs (artificially simulated damaged data (such as occlusion, noise injection, etc.) and other modal features), and use the contrast loss function: Among them, sim is the cosine similarity, τ is the temperature hyperparameter, K is the number of negative samples, and N is the number of positive samples in the input data.

[0063] Step 130: weighting the image embedding features based on the point cloud confidence score to obtain weighted image embedding features; and weighting the point cloud embedding features based on the image confidence score to obtain weighted point cloud embedding features.

[0064] In one example, the confidence weighting is calculated as follows: Among them, F L-C is the weighted image embedding feature; F C-L is the weighted point cloud embedding feature; C LiDARScore the radar confidence level; C Camera Score the image confidence; Q C is the query vector of image features, K L and V L are the key vector and value vector of radar features respectively; Q L is the query vector of radar features, K C and V C are the key vectors of image features respectively.

[0065] In the above technical solution, combining confidence with weighted cross-attention is conducive to focusing on key and accurate information more accurately in complex scenes, and can demonstrate higher target detection accuracy and stronger robustness.

[0066] Step 140: Fuse the weighted image embedding features and the weighted point cloud embedding features to obtain fused BEV features.

[0067] Specifically, the weighted image embedding features and the weighted point cloud embedding features are added together to obtain the fused BEV features.

[0068] In the above implementation, since the point cloud features and image features can be projected into a shared embedding space, and then the embedding quality of the two can be analyzed by comparing the loss function, and the features of different modalities can be weighted based on the analysis results, and finally the weighted features can be fused, the quality of the features of different modalities can be taken into consideration during the feature fusion process, which can help improve the accuracy of the final fused BEV features.

[0069] In some embodiments, reference Figure 3 The above step 200, generating point sequence data for representing map elements based on the historical high-precision map, includes the following steps: Step 210: Initialize the central anchor point and the anchor neighborhood corresponding to the central anchor point based on the historical high-precision map, generate a non-central anchor point corresponding to the central anchor point in the anchor neighborhood, and construct an anchor query unit based on the central anchor point and the non-central anchor point.

[0070] The central anchor point corresponds to a potential target point in the historical HD map. In one example, initializing the central anchor point includes: randomly initializing a set of anchor points on the HD map of the historical state, represented as Where N is the number of central anchor points, and each central anchor point p j Represents a potential destination point in the map.

[0071] The anchor neighborhood is generated based on the central anchor point. In one example, the anchor neighborhood is a square area with a fixed side length defined based on the central anchor point. The non-central anchor point is used to assist the fitting of the central anchor point. In one example, the method of randomly generating the non-central anchor point in the anchor neighborhood includes: randomly generating the non-central anchor point in the neighborhood Where N is the number of non-central anchor points.

[0072] In one example, the offset Δx of the non-central anchor point relative to the central anchor point ij and Δy ij Generated by random numbers. The offset calculation formula is: Where Δx ij Δy is the offset of the non-center anchor point relative to the center anchor point in the X-axis direction; ij is the offset of the non-center anchor point relative to the center anchor point in the Y-axis direction; a is the side length of the square anchor area, and β1 and β2 are random numbers in the interval (-1,1).

[0073] In one example, constructing an anchor query unit based on the central anchor point and the non-central anchor point includes: constructing a neighborhood center anchor query q for each anchor point and the non-central anchor point respectively. j and a non-neighborhood center anchor query q′ j , and the two together constitute an anchor query. Specifically, each query unit can be decomposed into a location part and a content part. i is the coordinate of the anchor point in the BEV plane (center or non-center), which provides an explicit physical location prior for the query and enables the model to directly utilize the geometric relationship of the anchor points in the BEV space. i It is the content shared by different query units and is a d-dimensional learnable parameter (similar to the weight in a neural network). It has two functions: (1) generating the offset from the anchor point to the target point, helping to adjust the anchor point position to match the real map element; (2) providing the category information of the anchor point (such as lane line, crosswalk, etc.) to distinguish different types of map elements.

[0074] In step 220 , anchor query units are grouped based on instances of map elements in the historical high-precision map, and intra-group feature interaction and inter-group feature interaction are performed on the anchor query units to update the anchor query units.

[0075] The query unit groups obtained by grouping anchor query units correspond to instances of map elements (such as lane lines or sidewalks). For example, each query unit group corresponds to an instance of a map element. Grouping can enhance feature interaction between anchor points within the same group while reducing redundant computation across groups (via the GL-SA mechanism).

[0076] Intra-group feature interaction refers to feature interaction between anchor query units within the same query unit group, while inter-group feature interaction refers to feature interaction between anchor query units between different query unit groups. Both intra-group and inter-group feature interaction enable full interaction between features between query units, helping to capture the correlations between different map elements. They also strengthen the dependencies between different features within the same map element, aiding in understanding the information in the image.

[0077] Step 230: Generate point sequence data based on the updated anchor query unit.

[0078] In the above technical solution, the physical location information can be explicitly combined by introducing the anchor neighborhood, which can help improve the accuracy of coordinate prediction and enhance the network's understanding of the location prior information, which in turn can help improve the accuracy of the final generated point sequence data.

[0079] For further reference, Figure 4 In step 220, anchor query units are grouped based on instances of map elements in the historical high-precision map, and intra-group feature interaction and inter-group feature interaction are performed on the anchor query units to update the anchor query units, including the following steps: In step 221 , for each query unit, the query center anchor point is fitted to the target point, and the non-center anchor points are fitted to random points in the ground truth neighborhood.

[0080] The ground truth neighborhood is a local spatial range centered on the annotated real object. In one example, the ground truth neighborhood is a circular area. In one embodiment, the radius of the ground truth neighborhood is determined by the distance D between the vertices, which is calculated as follows: where r is the radius of the ground truth neighborhood; ω is a scaling factor; and D is the distance between vertices.

[0081] It should be noted that in an HD map, each map element can be represented as a polyline consisting of ordered vertices (geometric key points of the HD map element), that is, the vertices determine its geometric shape. In this embodiment, the vertices on the polyline corresponding to the ground truth are uniformly sampled, that is, the distance D between adjacent vertices is fixed. The neighborhood of each vertex is defined as a circular area with a radius of r, where the upper limit of r is half the distance between vertices.

[0082] During the fitting process, the offset range of the non-central anchor point is generated by random numbers in the ground truth neighborhood, which is expressed as follows: Where, Δm i and Δni It is used to identify the offset range, β1 and β2 are random numbers in the interval (-1,1); r is the radius of the ground truth neighborhood.

[0083] Furthermore, the fitting process is optimized through the loss function to ensure that the central anchor point can accurately fit the target point and the non-central anchor points can accurately fit the random points in the ground truth neighborhood.

[0084] Specifically, the loss function is expressed as follows: L=λ1L center +λ2L non-center +λ3L seg +λ4L seg Where: L center The loss for fitting the target point (vertices in the ground truth neighborhood) to the supervised center anchor; L non-center The loss of random points in the neighborhood of the supervised non-central anchor fitting ground truth (reducing fitting ambiguity); L seg is the segmentation loss, which is used to enhance the representation of map element areas in BEV features; L depth This is the depth estimation loss, which is used to improve the accuracy of depth information. The introduction of the ground truth neighborhood and the design of the loss function can effectively improve the fitting accuracy.

[0085] Step 222: For each query unit group, split the local query of the query unit group to obtain local features.

[0086] Among them, the local features correspond to the split local queries.

[0087] Specifically, in the above technical solution, the process of splitting the local query is the process of doubling the dimension of the local query. Doubling the dimension enhances feature aggregation capabilities because the number of anchor queries is doubled (including non-central anchor points), and higher dimensions can accommodate more information.

[0088] In one example, we define a set of learnable parameters Among them L i Is the local query of group i. The local query is divided into two parts and In actual implementation, local queries can also be further doubled.

[0089] In one example, the local features are determined by: As an example) through a linear layer (linear transformation matrix W a ) Generate query vector Anchor query generates keys through linear layer Sum Based on the query vector q L and key Generate weight matrix The weight matrix and value Identify local features.

[0090] Specifically, based on the query vector q L and key Generate weight matrix It is expressed by the following formula: Will and value Multiply to get local features Step 223 , performing inter-group feature interaction on the local features of each query unit group to obtain the local features after the inter-group interaction.

[0091] Specifically, for each set of anchor queries Q i The position part P (including the central anchor point and the non-central anchor point) i Add sinusoidal position encoding (sinPE) and combine it with content part C i Add and average all groups to get the global feature Q m Each group of extracted local features and After that, take the mean and add the global feature Q m Enhance the context and obtain the preliminary fusion feature L 2′ : L 2′ Input the standard self-attention mechanism to enable local queries of different groups to exchange information with each other and output the local features L after the interaction between groups 2 .

[0092] Step 224 : Allocate the elements in the local features after the inter-group interaction to the corresponding query unit groups to achieve intra-group feature interaction.

[0093] Specifically, after assigning the elements of the local features after the inter-group interaction to the corresponding query unit groups, in each group, the anchor query queries all queries including the local query, thereby realizing the interaction of intra-group features.

[0094] Specifically, first, the anchor query Q i and local queries Generate keys through linear layers respectively Sum After merging, we get: Then, the anchor query Q i Generate query vector and Calculate the attention weight matrix Finally, the weighted aggregate value Output the updated anchor query That is, anchor query after intra-group interaction. In the above implementation, the ground truth neighborhood is introduced in the process of feature interaction. Since the ground truth neighborhood is a neighborhood representation based on the real target position, it provides the model with a clearer fitting target by combining each real target with the surrounding information. This not only helps to alleviate the ambiguity problem in the fitting process, but also enhances the robustness of the model, enabling the model to more accurately identify and locate targets when facing complex scenarios.

[0095] In some embodiments, reference Figure 5 Step 300 is to fuse the fused BEV features with the point array data to obtain the current map element, including the following steps: In step 310, a cross-attention mechanism is used to interact the fused BEV features and the point column data to obtain an interaction result.

[0096] Specifically, through the interaction between point sequence data and the fused BEV features, local contextual information related to the point sequence can be extracted from the fused BEV features, and finally an interaction result with enhanced environment can be obtained.

[0097] In one example, a cross-coarse-grained mechanism is used to interact with the fused BEV features and point column data, including: generating a query vector Q from the point column data, generating a key vector K and a value vector from the fused BEV features; calculating the attention weight matrix A based on the query vector Q and the key vector K; and using the attention weight matrix A to perform weighted summation on the value vector V to obtain the output of the cross-attention mechanism, that is, the interaction result.

[0098] Among them, the attention weight matrix A is calculated based on the query vector Q and the key vector K, which is expressed as follows: Where d is the dimension of the vector; Softmax is used to normalize the attention weights.

[0099] The value vector V is weighted and summed using the attention weight matrix A to obtain the output of the cross attention mechanism, which is expressed as follows: Output=A·V Step 320 : Fusing the interaction result with the point sequence data to obtain fused data.

[0100] Specifically, during the feature fusion process, the interaction results are concatenated with the point sequence to generate fused data. This allows us to further concatenate the context-enhanced features with the original point sequence after adding context to the features using the cross-attention mechanism. This improves the robustness of the vector representation by integrating context information with the geometric priors of the point sequence (such as initial position and order).

[0101] Step 330: Input the fused data into a multi-layer perceptron to obtain the current map element.

[0102] In the above implementation, the interactive attention mechanism can be used to first interact the fused BEV features and point column data, and then the interactive data can be fused with the point column data. In this way, the correlation between data of different modalities can be captured before data fusion and taken into consideration during the data fusion process, which can help improve the accuracy of the current map elements finally obtained.

[0103] The present application also provides an electronic device, such as Figure 6 As shown, Figure 6 The electronic device 500 shown includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 500 may further include a transceiver 504. It should be noted that in actual applications, the number of transceivers 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0104] The processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0105] The bus 502 may include a path for transmitting information between the above components. The bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 502 may be divided into an address bus, a data bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0106] The memory 503 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0107] The memory 503 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the application code stored in the memory 503 to implement the content shown in the above method embodiment.

[0108] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, PDAs (personal digital assistants), and PADs (tablet computers), and fixed terminals such as digital TVs and desktop computers, etc. They may also be server terminals. Figure 6 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0109] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the online vector map construction method provided in the above embodiment.

[0110] The present application embodiment also discloses a computer program product. Specifically, the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function of the embodiment of the present application is realized in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training equipment or data center to another website, computer, training equipment or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, data center, etc. that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)), etc.

[0111] The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features. The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A method for constructing an online vector map, characterized in that: The method comprises: Performing reliability analysis on the lidar point cloud collected by the on-board lidar and the multi-view images collected by the on-board camera, and fusing the lidar point cloud and the multi-view images into a fused BEV feature based on the reliability analysis result; Generate point array data for representing map elements based on historical high-precision maps; Fusing the fused BEV features with the point array data to obtain a current map element; The historical high-precision map is processed based on the current map elements to obtain a high-precision vector map.

2. The method according to claim 1, characterized in that The reliability analysis of the lidar point cloud collected by the vehicle-mounted lidar and the multi-view image collected by the vehicle-mounted camera, and fusing the lidar point cloud and the multi-view image into a fused BEV feature based on the reliability analysis result, includes: Projecting the point cloud features corresponding to the lidar point cloud and the image features corresponding to the multi-view images into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features; Analyzing the quality of the point cloud embedding feature and the image embedding feature by using a contrast loss function to obtain a point cloud confidence score corresponding to the point cloud embedding feature and an image confidence score corresponding to the image embedding feature; weighting the image embedding features based on the point cloud confidence score to obtain weighted image embedding features; weighting the point cloud embedding features based on the image confidence score to obtain weighted point cloud embedding features; The weighted image embedding feature and the weighted point cloud embedding feature are fused to obtain the fused BEV feature.

3. The method according to claim 1, characterized in that The generating of point sequence data for representing map elements based on historical high-precision maps includes: Initializing a central anchor point and an anchor neighborhood corresponding to the central anchor point based on the historical high-precision map, generating a non-central anchor point corresponding to the central anchor point in the anchor neighborhood, and constructing an anchor query unit based on the central anchor point and the non-central anchor point; Grouping the anchor query units based on instances of map elements in the historical high-precision map, and performing intra-group feature interaction and inter-group feature interaction on the anchor query unit queries to update the anchor query units; The point sequence data is generated based on the updated anchor query unit.

4. The method according to claim 3, characterized in that The grouping of the anchor query units and performing intra-group feature interaction and component feature interaction on the queries includes: For each query unit, fitting the query center anchor point to the target point and fitting the non-center anchor points to random points in the ground truth neighborhood; For each query unit group, splitting the local query of the query unit group to obtain local features; Performing inter-group feature interaction on the local features of each query unit group to obtain local features after inter-group interaction; The elements in the local features after the inter-group interaction are distributed to the corresponding query unit groups to achieve intra-group feature interaction.

5. The method according to claim 1, wherein The cross-attention mechanism processing of the fused BEV features and the point sequence data to obtain the current map element includes: Use a cross attention mechanism to interact the fused BEV features and the point column data to obtain an interaction result; Performing feature fusion on the interaction result and the point sequence data to obtain fused data; The fused data is input into a multi-layer perceptron to obtain the current map element.

6. The method according to claim 1, characterized in that The processing of the historical high-precision map based on the current map element to obtain a high-precision vector map includes: Aligning the current map element with the historical high-precision map; Optimizing the current map elements by optimizing the loss function so that the optimized map elements are more consistent with the historical high-precision map; The historical high-precision map is corrected based on the optimized map elements to obtain the high-precision vector map.

7. The method according to claim 6, characterized in that The optimization loss function includes at least one of a center point loss term, a non-center point loss term, a segmentation loss term, and a depth loss term; The center point loss term is used to supervise the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-precision map; The non-center point loss term is used to supervise the alignment between the non-center point of the current map element and the non-center point of the corresponding map element in the historical high-precision map; The segmentation loss is used to enhance the semantic segmentation accuracy of the area corresponding to the current map element; The depth estimation loss is used to enhance the depth information representation of the current map element.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the online vector map construction method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the online vector map construction method according to any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that When the computer program product is run on a computer device, the computer device is enabled to execute the online vector map construction method according to any one of claims 1 to 7.

Images