An online vector map construction method, device, storage medium and program product

Abstract

The application relates to the technical field of automatic driving perception, and provides an online vector map construction method, equipment, a storage medium and a program product. The map construction method comprises the following steps: performing reliability analysis on vehicle laser radar point cloud and multi-view images, and fusing the laser radar point cloud and the multi-view images into fused BEV features based on a reliability analysis result; generating point column data for representing map elements based on a historical high-precision map; performing cross-attention mechanism processing on the fused BEV features and the point column data to obtain current map elements; and processing the historical high-precision map based on the current map elements to obtain a high-precision vector map. Through the technical solution, the point cloud and the images can be fused based on the reliability analysis result of the laser radar point cloud and the multi-view images, so that the influence of unreliable data points on the overall data can be automatically reduced when the field of view is limited or the sensor data is abnormal, and the accuracy of vector map construction can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving perception, and in particular to an online vector map construction method, device, storage medium and program product. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, the requirements of automatic driving technology in perception accuracy, path planning and the like are increasingly stringent. In automatic driving, reliable 3D target detection is essential, and the fusion of multiple sensors is conducive to achieving accurate perception of the environment. For example, LiDAR can provide accurate three-dimensional spatial information, and a camera can provide rich semantic information. The fusion of these two types of data can improve detection accuracy through information complementation. In addition, the automatic driving system also needs accurate decision-making capability. A high-precision map can not only provide rich semantic information, but also accurately reflect the road environment, thereby providing strong data support for path planning and decision-making of an automatic driving vehicle.

[0003] In terms of 3D target detection, existing LiDAR-camera fusion models are very sensitive to sensor failure. When one modality is damaged, it will severely reduce the performance of the other modality, thereby impairing the overall target detection accuracy and robustness. In terms of high-precision map construction, traditional methods usually rely on LiDAR point cloud data and construct a map through SLAM technology, and then a large amount of human resources are needed for semantic annotation and maintenance. Moreover, the method has poor scalability, slow data update speed, and is difficult to reflect changes in the road environment in real time. Although end-to-end high-definition map construction technology has gradually attracted attention in recent years, there are still problems such as inaccurate coordinate prediction of initial reference points and high computational complexity, which makes it difficult to meet the demand for large-scale queries, and thus needs to be improved. SUMMARY

[0004] To help improve the accuracy of three-dimensional target detection, the present application provides an online vector map construction method, device, storage medium and program product.

[0005] In a first aspect, the present application provides an online vector map construction method, which adopts the following technical solution:

[0006] An online vector map construction method, the method comprising:

[0007] Performing reliability analysis on laser radar point clouds collected by a vehicle-mounted laser radar and multi-view images collected by a vehicle-mounted camera, and fusing the laser radar point clouds and the multi-view images into fused BEV features based on the reliability analysis results;

[0008] Generating point column data for representing map elements based on a historical high-precision map;

[0009] fuse the post-fusion BEV feature with the point line data to obtain a current map element;

[0010] process the historical high-precision map based on the current map element to obtain a high-precision vector map.

[0011] By adopting the technical solution, the laser radar point cloud and the multi-view image can be fused based on the reliability analysis results of the laser radar point cloud and the multi-view image, so that the influence of unreliable data points on the overall data can be automatically reduced when the field of view is limited or the sensor data is abnormal, thereby improving the accuracy of the data, and further helping to improve the accuracy of vector map construction.

[0012] Optionally, the reliability analysis on the laser radar point cloud collected by the vehicle-mounted laser radar and the multi-view image collected by the vehicle-mounted camera is performed, and the laser radar point cloud and the multi-view image are fused into a post-fusion BEV feature based on the reliability analysis results, including:

[0013] project the point cloud feature corresponding to the laser radar point cloud and the image feature corresponding to the multi-view image into a shared embedding space to obtain a point cloud embedding feature corresponding to the point cloud feature and an image embedding feature corresponding to the image feature;

[0014] analyze the quality of the point cloud embedding feature and the image embedding feature through a contrast loss function to obtain a point cloud confidence score corresponding to the point cloud embedding feature and an image confidence score corresponding to the image embedding feature;

[0015] weight the image embedding feature based on the point cloud confidence score to obtain a weighted image embedding feature, and weight the point cloud embedding feature based on the image confidence score to obtain a weighted point cloud embedding feature;

[0016] fuse the weighted image embedding feature and the weighted point cloud embedding feature to obtain the post-fusion BEV feature.

[0017] By adopting the technical solution, the quality of features of different modalities can be considered in the process of feature fusion, thereby helping to improve the accuracy of the post-fusion BEV feature obtained finally.

[0018] Optionally, the point line data for representing a map element is generated based on a historical high-precision map, including:

[0019] initialize a center anchor point and an anchor neighborhood corresponding to the center anchor point based on the historical high-definition map, generate a non-center anchor point corresponding to the center anchor point in the anchor neighborhood, and construct an anchor query unit based on the center anchor point and the non-center anchor point;

[0020] generate the point column data based on the updated anchor query unit.

[0021] By adopting the technical solutions, the anchor neighborhood is introduced to explicitly combine the physical position information, which can help improve the accuracy of coordinate prediction, enhance the understanding of the network for the position prior information, and further help improve the accuracy of the finally generated point column data.

[0022] Optionally, the grouping of the anchor query unit and the intra-group feature interaction and the inter-group feature interaction of the query include:

[0023] for each query unit, fitting the query center anchor point to a target point and fitting the non-center anchor point to a random point in a ground truth neighborhood;

[0024] for each query unit group, splitting the local query of the query unit group to obtain local features;

[0025] interacting the local features of each query unit group to obtain local features after inter-group feature interaction;

[0026] assigning elements in the local features after inter-group feature interaction to corresponding query unit groups to realize intra-group feature interaction.

[0027] By adopting the technical solutions, the ground truth neighborhood is introduced in the process of feature interaction. Since the ground truth neighborhood is a neighborhood representation based on a real target position, it provides a clearer fitting target for the model by combining each real target with the surrounding information. In this way, it not only helps to alleviate the ambiguity problem in the fitting process, but also enhances the robustness of the model, so that the model can more accurately identify and locate the target when facing complex scenes.

[0028] Optionally, the cross-attention mechanism processing of the fused BEV feature and the point column data to obtain a current map element includes:

[0029] interacting the fused BEV feature and the point column data using a cross-attention mechanism to obtain an interaction result; and performing feature fusion on the interaction result and the point column data to obtain fused data.

[0030] inputting the fused data into a multi-layer perception machine to obtain a current map element.

[0031] By adopting the technical solution, the association between data of different modalities can be captured before data fusion and taken into account in the process of data fusion, which can help improve the accuracy of the obtained current map element.

[0032] Optionally, the processing of the historical high-definition map based on the current map element to obtain a high-definition vector map comprises:

[0033] aligning the current map element with the historical high-definition map;

[0034] optimizing the current map element by optimizing a loss function, so that the optimized map element is more consistent with the historical high-definition map;

[0035] correcting the historical high-definition map based on the optimized map element to obtain the high-definition vector map.

[0036] By adopting the technical solution, the current map element can be optimized by optimizing a loss function, so that the map element used to update the historical high-definition map is more accurate, which can further help improve the accuracy of the high-definition vector map.

[0037] Optionally, the optimization loss function comprises at least one of a center point loss term, a non-center point loss term, a segmentation loss term, and a depth loss term.

[0038] The center point loss term is used to supervise the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-definition map.

[0039] The non-center point loss term is used to supervise the alignment between the non-center point of the current map element and the non-center point of the corresponding map element in the historical high-definition map.

[0040] The segmentation loss is used to enhance the semantic segmentation accuracy of the region corresponding to the current map element.

[0041] The depth estimation loss is used to enhance the depth information representation of the current map element.

[0042] In a second aspect, the present application provides an electronic device, which adopts the following technical solution:

[0043] An electronic device, the electronic device comprising:

[0044] at least one processor;

[0045] a memory;

[0046] at least one application program, wherein the at least one application program is stored in the memory and is configured to be executed by the at least one processor, and the at least one application program is configured to implement any one of the line vector map construction methods provided in the first aspect.

[0047] In a third aspect, the present application provides an electronic device, which adopts the technical scheme as follows:

[0048] A computer readable storage medium, the computer readable storage medium has a computer program stored thereon, and the computer program is characterized in that when the computer program is executed in a computer, the computer is caused to execute any one of the online vector map construction methods provided in the first aspect.

[0049] In a fourth aspect, the present application provides a computer program product, which adopts the technical scheme as follows:

[0050] A computer program product, comprising computer instructions, characterized in that when the computer program product is run on a computer device, the computer device is caused to execute any one of the online vector map construction methods provided in the first aspect.

[0051] In summary, the present application includes at least one of the following beneficial technical effects:

[0052] 1. Laser radar point cloud and multi-view image can be fused based on the reliability analysis results of the laser radar point cloud and the multi-view image, so that the influence of unreliable data points on the overall data can be automatically reduced when the field of view is limited or the sensor data is abnormal, thereby the accuracy of the data can be improved, and the accuracy of vector map construction can be improved.

[0053] 2. By introducing anchor neighborhood to explicitly combine physical location information, the accuracy of coordinate prediction can be improved, the understanding of location prior information by the network can be enhanced, and the accuracy of the finally generated point line data can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 is a flowchart of an online vector map construction method provided by an embodiment of the present application;

[0055] Figure 2 is a flowchart of a fusion BEV feature generation method provided by an embodiment of the present application;

[0056] Figure 3 is a flowchart of a point line data generation method provided by an embodiment of the present application;

[0057] Figure 4is a flowchart of an anchor query unit updating method provided by an embodiment of the present application.

[0058] Figure 5 is a flowchart of a current map element determination method provided by an embodiment of the present application.

[0059] Figure 6 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Figures 1-6 The specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0061] Firstly, a number of terms related to the embodiments of the present application are introduced.

[0062] Bird's-Eye-View (BEV) is a sensor data representation method from a high altitude overlooking the ground, which is widely used in autonomous driving and intelligent transportation fields.

[0063] Cross-Attention Mechanism is a deep learning technique for sequence modeling, which allows the model to focus on different parts of the input sequence and query sequence simultaneously when processing sequence data. In the image fusion task, Cross-Attention Mechanism allows the model to learn the mutual relationship between different input sequences or feature maps, which helps to capture the dependency between global and local information.

[0064] The embodiments of the present application disclose an online vector map construction method, which is used in an electronic device. The electronic device can be a controller arranged on a vehicle, or can also be a server in communication connection with the vehicle controller, as long as it can realize data calculation and processing.

[0065] Referring to Figure 1 The online vector map construction method comprises the following steps:

[0066] In step 100, the reliability of the laser radar point cloud collected by the vehicle-mounted laser radar and the multi-view image collected by the vehicle-mounted camera is analyzed, and the laser radar point cloud and the multi-view image are fused into fused BEV features based on the reliability analysis result.

[0067] The fused BEV feature refers to a feature represented on a bird's eye view after fusion of data. Specifically, due to the difference in spatial representation between the laser radar point cloud and the multi-view image, in order to facilitate the fusion of different modal data, the laser radar point cloud data and the multi-view image can be mapped into the bird's eye view to obtain unified fused BEV features.

[0068] In the research process, it is considered that the traditional multi-modal data fusion model is very sensitive to sensor failure. When the data of one modality is damaged, the performance of other modalities will be seriously affected, thereby damaging the accuracy and robustness of the overall target detection. Based on this, in the embodiment, before the data of different modalities is fused, the reliability of the data of different modalities is analyzed, and then the data of different modalities is fused based on the reliability analysis result. In this way, when the field of view is limited or the sensor data is abnormal, the influence of unreliable data points on the overall data can be automatically reduced, thereby improving the accuracy of the data.

[0069] In one example, the reliability analysis quantifies the reliability score of the data of different modalities through cross-modal comparison and confidence weighted mutual cross-attention, and the reliability analysis result includes the reliability score (such as confidence) corresponding to the data of different modalities. In this way, the weight of different modal information can be dynamically adjusted based on the reliability score during feature fusion.

[0070] Further, before the reliability analysis of the laser radar point cloud collected by the vehicle-mounted laser radar and the multi-view image collected by the vehicle-mounted camera, the laser radar point cloud and the multi-view image need to be aggregated in time and space dimensions to improve the spatio-temporal consistency of the features, thereby helping to improve the accuracy of processing moving objects in dynamic scenes.

[0071] Step 200, generating point column data for representing map elements based on a historical high-definition map.

[0072] The historical high-definition map is constructed based on pre-collected information, including information related to vehicle driving.

[0073] The map elements are elements related to vehicle driving, such as road information, road components, traffic signs, etc.

[0074] The point column data is used for geographical location information of the map elements.

[0075] In one example, the point column data is obtained by fitting the map elements on the historical high-definition map through anchor points.

[0076] Step 300, cross-attention mechanism processing is performed on the fused BEV feature and the point column data to obtain the current map elements.

[0077] The current map element is determined in combination with the laser radar point cloud and the multi-view image collected in real time. Since the dynamic characteristics of the current environment are considered, the current map element can be different from the map element in the historical high-precision map.

[0078] In one example, the current map element is represented by a vector.

[0079] At step 400, the historical high-precision map is processed based on the current map element to obtain a high-precision vector map.

[0080] In one example, the historical high-precision map is processed based on the current map element to obtain a high-precision vector map, including: aligning the current map element with the historical high-precision map; optimizing the current map element by optimizing a loss function to make the optimized map element more consistent with the historical high-precision map; and correcting the historical high-precision map based on the optimized map element to obtain a high-precision vector map.

[0081] The purpose of aligning the current map element with the historical high-precision map is to ensure the consistency of the map element. If the current map element cannot find a matching element in the historical high-precision map, it is marked as a new element.

[0082] The optimization process of the current map element specifically includes: after the loss value is calculated using the optimization loss function, the loss function is optimized by a back propagation algorithm, and the network parameters are updated to continuously improve the accuracy of the optimized map element.

[0083] The correction method of the historical high-precision map specifically includes: adding new map elements that are not in the historical high-precision map to the historical high-precision map, correcting map elements that are not in the historical high-precision map, obtaining an updated map, and converting the updated map to a vector form to obtain a high-precision vector map.

[0084] In one example, the optimization loss function includes at least one of a center point loss term, a non-center point loss term, a segmentation loss term, and a depth loss term.

[0085] The center point loss term is used to supervise the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-precision map.

[0086] The non-center point loss term is used to supervise the alignment between the non-center point of the current map element and the non-center point of the corresponding map element in the historical high-precision map to reduce fitting ambiguity.

[0087] The segmentation loss is used to enhance the semantic segmentation accuracy of the region corresponding to the current map element to enhance the representation of the map element region in the BEV feature.

[0088] The depth estimation loss is used to enhance the depth information representation of the current map element to improve the accuracy of the depth information.

[0089] In actual implementation, the parameters are dynamically adjusted according to the map element category (such as higher lane line weight) or scene dynamics (such as increasing the temporal loss weight when the vehicle is dense).

[0090] The implementation principle of the online vector map construction method according to an embodiment of the application is that: reliability analysis is performed on laser radar point clouds collected by a vehicle-mounted laser radar and multi-view images collected by a vehicle-mounted camera, and the laser radar point clouds and the multi-view images are fused into fused BEV features based on the reliability analysis results; point column data used to represent map elements is generated based on a historical high-precision map; the fused BEV features and the point column data are processed through a cross-attention mechanism to obtain a current map element; the historical high-precision map is processed based on the current map element to obtain a high-precision vector map. By using the above technical solution, the laser radar point clouds and the multi-view images can be fused based on the reliability analysis results of the laser radar point clouds and the multi-view images, so that the influence of unreliable data points on the overall data can be automatically reduced when the field of view is limited or the sensor data is abnormal, thereby improving the accuracy of the data, and further helping to improve the accuracy of vector map construction.

[0091] In some embodiments, referring to Figure 2 , step 100, performing reliability analysis on laser radar point clouds collected by a vehicle-mounted laser radar and multi-view images collected by a vehicle-mounted camera, and fusing the laser radar point clouds and the multi-view images into fused BEV features based on the reliability analysis results, comprises:

[0092] Step 110, projecting point cloud features corresponding to the laser radar point clouds and image features corresponding to the multi-view images into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features.

[0093] In one example, the point cloud features z LiDAR are BEV features generated after processing the laser radar point clouds using a 3D backbone network.

[0094] In another example, the image features z Camera are obtained by converting an initial image into BEV space features through a "boost-sputter-shoot" operation after processing the multi-view images using a 2D convolution backbone network to obtain initial image features. The specific conversion method is: the position of each pixel point of the initial image in the 3D space is inferred to project the initial image onto the top-down plane, and the projection onto the BEV space is integrated and optimized.

[0095] Further, before projecting the point cloud features corresponding to the laser radar point cloud and the image features corresponding to the multi-view image into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features, the method further comprises: processing the image features by using a temporal attention mechanism and a spatial attention mechanism to increase the spatial consistency of the multi-view features and the temporal dependence across frames. Correspondingly, in subsequent steps, the processed image features are used to replace the original image features for fusion calculation.

[0096] Specifically, for each time step t, the image features are flattened and projected into an embedding space to generate embedding features Through the spatial self-attention mechanism, the features of each view are interacted with the features of other views to calculate attention weights and generate spatial enhanced features S t Between different time steps, the attention weights of different time steps are calculated by the temporal attention mechanism to capture the temporal dependence relationship and generate spatio-temporal aggregation features to realize the spatio-temporal aggregation of the image features. Further, in order to stabilize the training, layer normalization and residual connection are performed after each attention mechanism.

[0097] In one example, the multi-view image refers to the images captured by six camera views of a vehicle-mounted camera. First, for each view (as a query vector Q k ), the similarity between the view and all other views j (as a key vector K j and a value vector V j ) is calculated:

[0098]

[0099] wherein, (W1, W2, W3 are learnable parameters, is the embedding feature of different views k, j at the same time step t), and d represents the embedding dimension.

[0100] Then, the value V j of all views is weighted and summed to obtain the spatial enhanced feature of the view k:

[0101]

[0102] Finally, the features of all views are combined to obtain the global spatial feature of the time step t:

[0103]

[0104] In step 120, the quality of the point cloud embedding features and the image embedding features is analyzed by using a contrast loss function to obtain a point cloud confidence score corresponding to the point cloud embedding features and an image confidence score corresponding to the image embedding features.

[0105] Specifically, in the process of quality analysis, the undamaged data is encouraged to be close in the future space, while the damaged data is pushed away. In this way, by analyzing the embedded quality, the point cloud confidence score C can be generated using the multi-layer perceptron and the Sigmoid function LiDAR and the image confidence score C Camera .

[0106] The specific process is: preparing positive sample pairs (undamaged LiDAR and camera features in the same frame) and negative sample pairs (artificially simulated damaged data (such as occlusion, noise injection, etc.) and other modal features), using a contrastive loss function:

[0107]

[0108] Where sim is the cosine similarity, τ is the temperature hyperparameter, K is the number of negative samples, and N is the number of positive samples in the input data.

[0109] Step 130, weighting the image embedding features based on the point cloud confidence score to obtain weighted image embedding features; weighting the point cloud embedding features based on the image confidence score to obtain weighted point cloud embedding features.

[0110] In one example, the confidence weighting calculation is represented by the following formula:

[0111]

[0112] Where F L-C is the weighted image embedding feature; F C-L is the weighted point cloud embedding feature; C LiDAR is the radar confidence score; C Camera is the image confidence score; Q C is the query vector of the image feature, K L and V L are the key vector and value vector of the radar feature, respectively; Q L is the query vector of the radar feature, K C and V C are the key vector of the image feature.

[0113] In the above technical solution, the combination of confidence and weighted cross-attention is beneficial to more accurately focus on key and accurate information in complex scenes, and can exhibit higher target detection accuracy and stronger robustness.

[0114] Step 140, fusing the weighted image embedding features and the weighted point cloud embedding features to obtain fused BEV features.

[0115] Specifically, the weighted image embedding feature and the weighted point cloud embedding feature are added to obtain the fused BEV feature.

[0116] In the above embodiments, the point cloud feature and the image feature can be projected into a shared embedding space, and then the embedding quality of the two is analyzed through a comparison loss function, and the features of different modalities are weighted based on the analysis result, and finally the weighted features are fused. In this way, the quality of the features of different modalities can be considered in the process of feature fusion, which can help to improve the accuracy of the fused BEV feature obtained finally.

[0117] In some embodiments, with reference to Figure 3 The step 200 of generating the point column data for representing the map elements based on the historical high-definition map comprises the following steps.

[0118] The step 210 of initializing the center anchor point and the anchor neighborhood corresponding to the center anchor point based on the historical high-definition map, generating non-center anchor points corresponding to the center anchor point in the anchor neighborhood, and constructing an anchor query unit based on the center anchor point and the non-center anchor point.

[0119] The center anchor point corresponds to a potential target point in the historical high-definition map. In one example, initializing the center anchor point comprises: randomly initializing a group of anchor points on the high-definition map of the historical state, denoted as Where N is the number of center anchor points, and each center anchor point p j represents a potential target point in the map.

[0120] The anchor neighborhood is generated based on the center anchor point. In one example, the anchor neighborhood is a square domain with a fixed side length defined based on the center anchor point;

[0121] The non-center anchor point is used to assist the fitting of the center anchor point. In one example, the way of randomly generating the non-center anchor point in the anchor neighborhood comprises: randomly generating the non-center anchor point in the neighborhood Where N is the number of non-center anchor points.

[0122] In one example, the offset of the non-center anchor point relative to the center anchor point is Δx ij and Δy ij generated by a random number. The calculation formula of the offset is:

[0123]

[0124] Where Δx ij is the offset of the non-center anchor point relative to the center anchor point in the X-axis direction; and Δy ijis the offset of the non-central anchor point relative to the central anchor point in the Y-axis direction; a is the side length of the square anchor field, and β1 and β2 are random numbers in the interval (-1, 1).

[0125] In one example, constructing the anchor query unit based on the central anchor point and the non-central anchor point includes: constructing a neighborhood central anchor query q j and a non-neighborhood central anchor query q' j for each anchor point and non-central anchor point, respectively, which together constitute an anchor query.

[0126] Specifically, each query unit can be decomposed into a position part and a content part. The position part P i is the coordinates of the anchor point (central or non-central) in the BEV plane, providing an explicit physical location prior for the query, enabling the model to directly utilize the geometric relationships of the anchor points in the BEV space. The content part C i is a shared content part in different query units, which is a d-dimensional learnable parameter (similar to weights in neural networks). Its role has two aspects: (1) generating the offset of the anchor point to the target point, helping to adjust the anchor point position to match the real map elements; (2) providing the category information of the anchor point (such as lane lines, pedestrian crossings, etc.), which is used to distinguish different types of map elements.

[0127] Step 220, grouping the anchor query units based on instances of map elements in the historical high-precision map, and performing intra-group feature interaction and inter-group feature interaction on the anchor query units to update the anchor query units.

[0128] Among them, the query unit group obtained by grouping the anchor query units corresponds to the instances of the map elements (such as different lane lines, sidewalks). For example: each query unit group corresponds to an instance of a map element. Through grouping, the feature interaction of anchor points within the same group can be strengthened, while reducing cross-group redundant calculations (through the GL-SA mechanism).

[0129] Intra-group feature interaction refers to feature interaction of anchor query units within the same query unit group, and inter-group feature interaction refers to feature interaction of anchor query units between different query unit groups. Through intra-group feature interaction and inter-group feature interaction, the features between query units can be fully interacted, which can help capture the relevance between different map elements, and also can strengthen the dependency relationship between different features in the same map element, which is helpful for understanding the information in the image.

[0130] Step 230, generating point column data based on the updated anchor query units.

[0131] In the technical solution, the anchor neighborhood is introduced to explicitly combine the physical position information, thereby helping to improve the accuracy of coordinate prediction, enhance the understanding of the network on the position prior information, and further help to improve the accuracy of the finally generated point column data.

[0132] Further, referring to Figure 4 , step 220, grouping the anchor query units based on the instances of the map elements in the historical high-precision map, and performing intra-group feature interaction and inter-group feature interaction on the anchor query units to update the anchor query units, including the following steps:

[0133] Step 221, for each query unit, fitting the query center anchor point to the target point, and fitting the non-center anchor point to a random point in the ground truth neighborhood.

[0134] Wherein, the ground truth neighborhood refers to a local spatial range centered on the labeled real object. In an example, the ground truth neighborhood is a circular region. In an example, the radius of the ground truth neighborhood is determined by the distance D between the vertices, specifically calculated by the following formula:

[0135]

[0136] Wherein, r is the radius of the ground truth neighborhood; ω is a scaling factor; D is the distance between the vertices.

[0137] It should be noted that in the high-precision map, each map element can be represented as a polyline composed of ordered vertices (geometric key points of the high-precision map element), i.e. the vertices determine its geometric shape. In this embodiment, the vertices on the polyline corresponding to the ground truth are uniformly sampled, i.e. the distance D between adjacent vertices is fixed. The neighborhood of each vertex is defined as a circular region with a radius of r, where the upper limit of r is half of the distance between the vertices.

[0138] In the fitting process, the offset range of the non-center anchor point is generated within the ground truth neighborhood by a random number, specifically represented by the following formula:

[0139]

[0140] Wherein, Δm i and Δn i are used to identify the offset range, β1 and β2 are random numbers in the interval (-1, 1); r is the radius of the ground truth neighborhood.

[0141] Further, the fitting process is optimized by a loss function to ensure that the center anchor point can accurately fit the target point and the non-center anchor point can accurately fit the random point in the ground truth neighborhood.

[0142] In particular, the loss function is represented by the following formula:

[0143] L = λ1L center + λ2L non-center + λ3L seg + λ4L seg

[0144] Wherein: L center is the loss of the supervised central anchor fitting the target point (the vertex of the ground truth neighborhood); L non-center is the loss of the supervised non-central anchor fitting the random point in the ground truth neighborhood (reducing fitting ambiguity); L seg is the segmentation loss, used to enhance the representation of the map element region in the BEV feature; L depth is the depth estimation loss, used to improve the accuracy of depth information. By introducing the ground truth neighborhood and the design of the loss function, the fitting accuracy can be effectively improved.

[0145] In step 222, for each query unit group, the local query of the query unit group is split to obtain a local feature.

[0146] Wherein, the local feature corresponds to the split local query.

[0147] In particular, in the above technical solution, the process of splitting the local query is the process of doubling the dimension of the local query, and the dimension doubling enhances the feature aggregation capability. Because the number of anchor queries is doubled (including non-central anchor points), and higher dimensions can accommodate more information.

[0148] In one example, a set of learnable parameters is defined, wherein L i is the local query of the i-th group. The local query is divided into two parts and In actual implementation, the local query can also be further doubled.

[0149] In one example, the determination manner of the local feature includes: through the local query (hereinafter referred to as in the previous example) through a linear layer (linear transformation matrix W a ) to generate a query vector The anchor query generates a key and a value Based on the query vector q L and the key generate a weight matrix The weight matrix and the value determine the local feature.

[0150] In particular, based on the query vector qL and key Generating weight matrix is expressed by the following formula:

[0151]

[0152] is multiplied by the value to obtain the local feature

[0153]

[0154] Step 223, inter-group feature interaction is performed on the local features of each query unit group to obtain the local features after inter-group interaction.

[0155] Specifically, for the position part P of each group of anchor queries Q i (including the central anchor point and the non-central anchor point) i , a sine position encoding (sinPE) is added, and is added to the content part C i , and the global feature Q m is obtained after averaging all groups.

[0156]

[0157] The local features extracted from each group and are averaged and added to the global feature Q m to enhance the context and obtain the preliminary fusion feature L 2′ .

[0158]

[0159] L 2′ is input into the standard self-attention mechanism, and the local queries of different groups exchange information with each other, and the local features after inter-group interaction L 2 are output.

[0160] Step 224, elements in the local features after inter-group interaction are assigned to the corresponding query unit groups to realize intra-group feature interaction.

[0161] Specifically, after the elements in the local features after inter-group interaction are assigned to the corresponding query unit groups, in each group, the anchor query Q i and the local query Q exchange information with each other, thereby realizing intra-group feature interaction.

[0162] Specifically, first, the intra-group anchor query Q i and the local query Q generate the key and the value through linear layers respectively.After merging, we get:

[0163]

[0164] Then, the anchor query Q i Generate query vector With Calculate attention weight matrix

[0165]

[0166] Finally, weighted aggregate value Output the updated anchor query That is, the anchor query after intra-group interaction In the above embodiment, the ground truth neighborhood is introduced in the process of feature interaction. Since the ground truth neighborhood is a neighborhood representation based on the real target position, it provides a clearer fitting target for the model by combining each real target with the surrounding information. This not only helps to alleviate the ambiguity problem in the fitting process, but also enhances the robustness of the model, so that the model can more accurately identify and locate the target when facing complex scenes.

[0167] In some embodiments, with reference to Figure 5 , step 300, the fused BEV feature and the point list data are fused to obtain the current map element, including the following steps:

[0168] Step 310, using cross-attention mechanism to interact the fused BEV feature and the point list data to obtain the interaction result.

[0169] Specifically, through the interaction of the point list data and the fused BEV feature, the local context information related to the point list can be extracted from the fused BEV feature, and finally an environment enhanced interaction result is obtained.

[0170] In one example, the cross-attention mechanism is used to interact the fused BEV feature and the point list data, including: generating a query vector Q from the point list data, generating a key vector K and a value vector from the fused BEV feature; based on the query vector Q and the key vector K, calculate the attention weight matrix A; using the attention weight matrix A to weighted sum the value vector V, to obtain the output of the cross-attention mechanism, that is, the interaction result.

[0171] Wherein, based on the query vector Q and the key vector K, the attention weight matrix A is calculated by the following formula:

[0172]

[0173] Wherein, d is the dimension of the vector; Softmax is used to normalize the attention weight.

[0174] The output of the cross-attention mechanism is obtained by using the attention weight matrix A to perform weighted summation on the value vector V, and is represented by the following formula:

[0175] Output=A·V

[0176] At step 320, the interaction result is fused with the point column data to obtain fused data.

[0177] Specifically, in the process of feature fusion, the interaction result is spliced with the point column to obtain the fused data. In this way, in the case that the interaction result is obtained by increasing the environment of the features through the cross-attention mechanism, the environment-enhanced features are further spliced with the original point column to integrate the environmental information and the geometric prior of the point column (such as the initial position, the order relationship), thereby improving the robustness of the vector representation.

[0178] At step 330, the fused data is input into the multi-layer perception to obtain the current map element.

[0179] In the above embodiment, the fused BEV features and the point column data can be interacted using the interaction attention mechanism first, and then the interaction data and the point column data are fused. In this way, the correlation between the data of different modalities can be captured before data fusion and taken into account in the process of data fusion, which can help to improve the accuracy of the current map element obtained finally.

[0180] The embodiments of the present application also provide an electronic device, such as Figure 6 As shown in the figure, Figure 6 The electronic device 500 shown in the figure includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, such as through a bus 502. Optionally, the electronic device 500 can also include a transceiver 504. It should be noted that in actual applications, the transceiver 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0181] The processor 501 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure content of the present application. The processor 501 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessor, etc.

[0182] Bus 502 can include a path for conveying information between above-mentioned components. Bus 502 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into an address bus, a data bus, etc. For convenience, only one bus is shown in the figure, but it does not mean that there is only one bus or only one type of bus. Figure 6

[0183] Memory 503 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this.

[0184] Memory 503 is used to store application program code for implementing the scheme of the present application, and is controlled by processor 501 to execute. Processor 501 is used to execute the application program code stored in memory 503 to realize the content shown in the foregoing method embodiments.

[0185] The electronic device includes but is not limited to: mobile terminals such as mobile phones, notebook computers, PDAs (Personal Digital Assistants), PADs (Tablet PCs), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 6 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0186] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and when the computer program is executed in a computer, the computer executes the online vector map construction method provided by the above-mentioned embodiments.

[0187] ​The embodiments of the present application also disclose a computer program product. Specifically, the computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions of the embodiments of the present application are implemented wholly or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, training device or data center to another website, computer, training device or data center through wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by the computer or data storage device such as training device, data center, etc. integrated with one or more available medium sets. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0188] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in the specification (including the abstract and drawings) can be replaced by other equivalent or similar features with similar purposes unless specifically stated. That is, each feature is only an example of a series of equivalent or similar features unless specifically stated. The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in the specification (including the abstract and drawings) can be replaced by other equivalent or similar features with similar purposes unless specifically stated. That is, each feature is only an example of a series of equivalent or similar features unless specifically stated.

Claims

1. A method for constructing an online vector map, characterized in that, The method includes: Reliability analysis is performed on the lidar point cloud collected by the vehicle-mounted lidar and the multi-view images collected by the vehicle-mounted camera, and the lidar point cloud and the multi-view images are fused into fused BEV features based on the reliability analysis results. Point column data for representing map elements is generated based on historical high-precision maps; The fused BEV features are then fused with the point data to obtain the current map element; The historical high-precision map is processed based on the current map elements to obtain a high-precision vector map; The process of performing reliability analysis on the lidar point cloud acquired by the vehicle-mounted lidar and the multi-view images acquired by the vehicle-mounted camera, and fusing the lidar point cloud and the multi-view images into fused BEV features based on the reliability analysis results, includes: projecting the point cloud features corresponding to the lidar point cloud and the image features corresponding to the multi-view images into a shared embedding space to obtain point cloud embedding features corresponding to the point cloud features and image embedding features corresponding to the image features; analyzing the quality of the point cloud embedding features and the image embedding features through a comparative loss function to obtain point cloud confidence scores corresponding to the point cloud embedding features and image confidence scores corresponding to the image embedding features; weighting the image embedding features based on the point cloud confidence scores to obtain weighted image embedding features; weighting the point cloud embedding features based on the image confidence scores to obtain weighted point cloud embedding features; and fusing the weighted image embedding features and the weighted point cloud embedding features to obtain the fused BEV features. The step of processing the historical high-precision map based on the current map elements to obtain a high-precision vector map includes: aligning the current map elements with the historical high-precision map; optimizing the current map elements by optimizing the loss function so that the optimized map elements are more consistent with the historical high-precision map; and correcting the historical high-precision map based on the optimized map elements to obtain the high-precision vector map.

2. The method according to claim 1, characterized in that, The point column data generated based on historical high-precision maps to represent map elements includes: Initialize the central anchor point and the anchor neighborhood corresponding to the central anchor point based on the historical high-precision map, generate the non-central anchor point corresponding to the central anchor point in the anchor neighborhood, and construct the anchor query unit based on the central anchor point and the non-central anchor point. The anchor query unit is grouped based on instances of map elements in the historical high-precision map, and intra-group and inter-group feature interactions are performed on the anchor query unit queries to update the anchor query unit. The point column data is generated based on the updated anchor query unit.

3. The method according to claim 2, characterized in that, The process of grouping the anchor query units and performing intra-group feature interaction and component feature interaction on the queries includes: For each query unit, the queried central anchor point is fitted to the target point, and the non-central anchor point is fitted to a random point in the ground reality neighborhood. For each query unit group, the local queries of the query unit group are split to obtain local features; Inter-group feature interaction is performed on the local features of each group of query units to obtain the local features after inter-group interaction; The elements in the local features after the inter-group interaction are assigned to the corresponding query unit groups to achieve intra-group feature interaction.

4. The method according to claim 1, characterized in that, The step of processing the fused BEV features with the point data using a cross-attention mechanism to obtain the current map element includes: The fused BEV features and the point data are interacted using a cross-attention mechanism to obtain the interaction result; The interaction results are fused with the point data to obtain fused data. The fused data is input into a multilayer perceptron to obtain the current map elements.

5. The method according to claim 1, characterized in that, The optimized loss function includes at least one of the following: center point loss term, non-center point loss term, segmentation loss term, and depth loss term; The center point loss term is used to monitor the alignment between the center point of the current map element and the center point of the corresponding map element in the historical high-precision map; The non-center point loss term is used to monitor the alignment between the non-center point of the current map element and the non-center point of the corresponding map element in the historical high-precision map; The segmentation loss is used to enhance the semantic segmentation accuracy of the region corresponding to the current map element; The depth estimation loss is used to enhance the depth information representation of the current map element.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, said at least one application being configured to: perform the online vector map construction method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed in the computer, the computer is instructed to perform the online vector map construction method according to any one of claims 1 to 5.

8. A computer program product comprising computer instructions, characterized in that, When the computer program product is run on a computer device, the computer device performs the online vector map construction method as described in any one of claims 1 to 5.

Images