Automatic driving local vector map construction method based on standard precision map and fusion perception

Abstract

The invention belongs to the technical field of automatic driving, and particularly relates to an automatic driving local vector map construction method based on a standard precise map and fusion perception. The method comprises the following steps: preprocessing standard precise map information, and constructing a fusion sensing network; in the standard and precise map information preprocessing, original standard and precise map data are processed into vectorization expression suitable for model input; the fusion sensing network comprises a feature extraction backbone network, a BEV (aerial view) view conversion network, a multi-mode BEV feature fusion network and a map element decoding module; in the map element decoding module, the processed standard precise map elements are interacted with BEV aerial view features of the sensor; in local map model training, query denoising is introduced as an additional training task, and query sorting selection is introduced to accelerate the model convergence speed. According to the method, higher accuracy, better robustness and higher operation efficiency can be achieved, and the influence of environmental factors is smaller.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a method for constructing a local vector map for autonomous driving. Background Art

[0002] High-precision maps (HD maps) are a crucial component of today's autonomous driving solutions. HD maps can be considered high-precision sensors that can handle obstacles like occlusion, limited detection range, and inclement weather. They provide autonomous vehicles with accurate and detailed road information, including lane markings, pedestrian crossings, intersections, traffic signs, and traffic signals. These maps effectively complement the vehicle's perception of static road elements, supporting downstream positioning, prediction, decision-making, and planning tasks, enabling more efficient and safe completion of autonomous driving tasks. Despite their high accuracy, HD maps have inherent drawbacks, such as high acquisition and production costs, untimely updates, and limited coverage, which have hindered the large-scale application of autonomous driving.

[0003] Human drivers, on the other hand, do not require HD maps. After acquiring navigation information, they use their environmental perception to determine lane position and topology. Some methods rely solely on onboard sensors to determine lane lines and their topology, which introduces perception uncertainties (e.g., due to environmental occlusion and limited field of view).

[0004] In the task of local map perception for autonomous driving, the introduction of lightweight Standard Definition Map (SD Map) information effectively compensates for the limitations of on-board sensors (such as distance limitations and occlusions). Compared to methods that rely solely on on-board sensors, this method can enhance the recognition accuracy and robustness of the local vector map construction model. Compared to traditional high-precision map methods, which are expensive to produce and slow to update, the accuracy of this method is slightly reduced. However, it can help reduce the cost of autonomous driving, cope with China's frequently updated road infrastructure, and expand the scope of autonomous driving functions.

[0005] MapTR and its follow-up, MapTRv2, proposed a perception-based online map construction method that identifies map elements such as lane markings, road boundaries, zebra crossings, and lane centerlines. This method models map elements as a set of permutation-invariant points and uses point-to-point loss and edge direction loss to supervise the geometry of points and edges, improving the performance and efficiency of end-to-end online map detection. The MapTR family of methods uses a fixed number of points to represent complex map elements, which can easily cause distortion for complex shapes and redundant information for simple shapes.

[0006] Bird's-eye-view (BEV)-based perception algorithms are commonly used in autonomous driving perception. Their goal is to convert input image sequences from different perspectives into bird's-eye-view (BEV) features. Visual perspective images inherently lack the depth and size information of 3D physical space, making accurate 3D spatial distance estimation a natural weakness of visual methods. Therefore, transforming from visual pixel space to BEV space is a long-standing challenge and the core of vision-based BEV perception.

[0007] Currently, transformation methods can be divided into two categories: explicit and implicit. The former is based on deterministic geometric relationships, while the latter uses deep neural networks to fit the mapping function of view transformation. Explicit methods require the prediction of depth values ​​or depth distribution, which can usually be directly supervised by depth annotation or indirectly supervised by the final 3D detection task. The predicted depth and camera internal and external parameters are combined to achieve view transformation. Implicit methods do not directly predict depth, and usually use multi-layer perceptrons (MLPs) or transformer networks (Transformers) to directly map pixel space to BEV space.

[0008] Compared with visual sensors, lidar sensors can provide accurate 3D point cloud information. Therefore, in autonomous driving perception tasks, fusion algorithms are often used to fuse the lidar point cloud and visual image features in the BEV space.

[0009] Most existing technologies use only sensor input to model local vector maps, such as camera and lidar sensor input. Since these methods rely solely on sensor input, they lack prior information about road structure and are easily limited by the sensors themselves. For example, the sensor's perception range is limited, resulting in insufficient perception accuracy at long distances. Road elements are also affected by factors such as occlusion by dynamic or static obstacles or weather and lighting, resulting in limited sensor perception accuracy and insufficient performance in the absence of prior information.

[0010] A small number of methods incorporate navigation or precision maps as prior information input. These methods typically represent the precision map as a rasterized bird's-eye view image and then combine or interact with the sensor-perceived BEV feature map. Although this representation is a dense image, the actual useful pixels in the image are sparse, and the road's point and line geometry is lost.

[0011] Some other methods vectorize map priors, representing them as a set of points and lines, and use them as sparse query input models. However, because they lack explicit encoding of the locations and categories of map elements in the refined map, they weaken the use of prior information and result in low accuracy. Alternatively, these modeling methods are more complex than the present invention, resulting in increased time-consuming deployment on real vehicles.

[0012] CN118115963A (A method for constructing an online semantic vector map based on navigation maps and vision) uses only visual images as perceptual input. When constructing visual bird's-eye view features, the image features are projected into the bird's-eye view feature space by predicting the depth distribution. In the use of navigation maps / precision maps, the map is encoded in the form of a raster map and spliced ​​with the image bird's-eye view. Since the road data in the map has a strong structured characteristic, expressing the map a priori as a picture under the bird's-eye view will result in sparse useful pixels and the loss of the point-line geometric structure of the road. When decoding the map elements from the bird's-eye view feature space, a hierarchical query method is adopted to express each map element instance query as a combination of instance and point query, which has the problems of modeling redundancy and high computational complexity. Summary of the Invention

[0013] The purpose of the present invention is to provide a method for constructing a local vector map for autonomous driving based on a standard map and fusion perception, which has fast perception speed, high discrimination accuracy and good safety performance, so as to overcome the many shortcomings of the existing technology and meet the urgent needs of the rapid development of autonomous driving technology.

[0014] The method for constructing a local vector map for autonomous driving provided by the present invention is based on standard and precise maps and fusion perception technology, including standard and precise map information preprocessing, and constructing and training a fusion perception network. The specific contents are as follows:

[0015] (1) Preprocessing of accurate map information;

[0016] The standard map contains the basic topological structure of the road, such as Figure 2 (a) shows the key road-level centerline position, traffic flow direction, and number of lanes N. For non-intersection areas, multiple parallel lines are symmetrically extended from the road centerline to both sides as lane centerlines, with the distance between adjacent parallel lines being the preset fixed lane width. Specifically:

[0017] When the traffic direction corresponding to the center line of the road is one-way, if the number of lanes N is an odd number, the center line of the road is retained as the central axis and the lanes are expanded to the left and right sides. parallel lines; if the number of lanes N is an even number, the center line of the road is used as the central axis and extended to the left and right sides. The original road centerline is not retained. The direction of the expanded lane centerline is consistent with the traffic flow direction of the road centerline.

[0018] When the traffic direction of the road centerline is bidirectional, N is extended to both sides of the centerline. l and N r parallel lines, where N l and N rare the number of lanes on both sides. For each lane centerline, 20 points are uniformly sampled as a priori queries for non-intersection lane centerlines and provided as model input.

[0019] See specifically Figure 2 The blue line in (c) is the expanded lane centerline vector, and the arrow indicates the direction of traffic flow. When the road centerline corresponds to a one-way traffic flow and there are three lanes, one parallel line is extended to the left and right of the road centerline, forming a total of three parallel lines representing the three lane centerlines. When there are two lanes, one parallel line is extended to the left and right of the road centerline, and the original road centerline is not retained, forming a total of two parallel lines representing the two lane centerlines. When the road centerline corresponds to a two-way traffic flow and there are one and two lanes on both sides, one and two parallel lines are extended to the left and right of the road centerline, respectively, and the original road centerline is not retained, forming one and two lane centerlines for different traffic flow directions.

[0020] Then, based on lane-level traffic information, the road topology within the intersection area is constructed, and the centerline of the lane before the intersection is connected to the centerline of the target lane via the input and output vectors. Specifically, for the centerline of the lane entering the intersection (the lane centerline is pointing towards the intersection), the input vector is taken from the end of the lane centerline and extended 5 meters into the intersection as the input vector end point, forming a directional line segment. For the centerline of the lane exiting the intersection (the lane centerline is pointing away from the intersection), the outgoing vector is extended 5 meters into the intersection from the starting point of the lane centerline and extended 5 meters into the intersection as the starting point of the outgoing vector, forming a directional line segment with the starting point of the lane centerline as the output vector end point. 10 points are uniformly sampled for each of the input and output vector segments, forming a vector polyline with a total of 20 points, which serves as a prior query for the intersection centerline and is provided as model input. Figure 2 The red line in (c) indicates the road connection before and after the intersection.

[0021] This preprocessing is important and is used in conjunction with the map element decoding module described later.

[0022] (2) Build and train a fusion perception network model;

[0023] The fusion perception network model includes a feature extraction backbone network, a BEV (bird's eye view) perspective transformation network, a multimodal BEV feature fusion network, and a map element decoding module; among them:

[0024] Feature extraction backbone network: For the image input of 7 cameras covering 360 degrees on the vehicle (including forward close-range, forward long-range, left front, right front, left rear, right rear, and rear, the configuration of different vehicles will be different), a convolutional neural network (CNN) is used to extract features from each image, and multi-scale features after 4, 8, 16, and 32 times downsampling are obtained respectively. Then, the multi-scale features are input into a network with a bidirectional feature pyramid (Bi-FPN) structure to obtain the image features after aggregation of each perspective. For a 120-degree forward laser radar on the vehicle, the point cloud data is converted into columnar features in the BEV space as a 2D pseudo image. Subsequently, a convolutional neural network is used as the backbone network for point cloud feature extraction, and SECONDFPN is used as the neck network to fuse multi-scale features to obtain the BEV features of the point cloud.

[0025] BEV perspective transformation network: predefines a set of grid-like learnable BEV queries, interacts the BEV queries with the aforementioned image features through an attention mechanism, thereby generating multi-scale features under BEV, and then performs multi-scale fusion of image BEV features through a bidirectional feature pyramid.

[0026] Multimodal BEV feature fusion network: In the fusion module, the aforementioned image BEV features and point cloud BEV features are concatenated after convolution operation.

[0027] Map Element Decoding Module: The map decoding module performs multiple attention interactions between the map query from the standard map information preprocessing phase and the fused BEV features. The results obtained through the classification and regression heads are used as the final prediction. Specifically, it includes two steps: query construction and attention interaction.

[0028] (a) Constructing a query

[0029] The map element decoding module performs multiple attention interactions between map queries and fused BEV features. Unlike some methods that use point queries, which lead to redundant modeling and high computational complexity of the self-attention module, this method uses instance-level queries. Instance-level queries include initialization queries, precision map prior queries, and noisy queries.

[0030] Suppose the initialization query: Prior query of precise map: Noisy query: Every element Indicated as N v The set of ordered points on the BEV plane, N init ,N sd ,g·N noised They represent the number of initialization queries, prior queries and noisy queries respectively, g is the number of groups of noisy queries, N noisedis the number of queries in each group.

[0031] For the initial query Q init , each query i corresponds to a set of reference points in the two-dimensional normalized BEV coordinate system The reference points are obtained by a set of learnable parameters through a multi-layer perceptron (MLP) operation, where the j-point coordinates of the i-th group of reference points are defined as rf ij =(x ij ,y ij ).

[0032] For the prior query Q sd After preprocessing the map data, the map provides additional instance query input. Among them, 1 is the dimension corresponding to the category, here we use the intersection centerline or road centerline attribute of the precision map, 2 represents the number of coordinates of each point (x and y in BEV space), and the category dimension is encoded as the query through the multi-layer perceptron N v After the BEV range is normalized, the element points of the standard map are used as reference points corresponding to the query. Here N v is 20, which means 20 points uniformly sampled on the lane centerline-level intersection centerline after preprocessing.

[0033] For the noisy query Q noised The goal is to recover the correct labels from noisy queries without using Hungarian bipartite matching to address the instability of Hungarian matching. To make the noisy queries as diverse as possible, we introduce g groups of noisy queries, each containing the true labels of all samples, and add random noise to the true labels:

[0034] (i) For each point of the map element label, add random position offset noise (Δx, Δy) and ensure where λ∈(0,1), d min The minimum distance between two adjacent points on a map element to ensure that the map element after adding noise has no unreasonable shape.

[0035] (ii) For the categories of map element labels, a certain proportion (specifically, the proportion range can be 30%-70%, preferably 50%) of the labels are reversed to other categories. The denoising task is only used during training. During inference, the denoising part will be removed, so it does not incur additional computational overhead on the actual vehicle.

[0036] After the above three types of queries and reference points are spliced ​​together, the total instance query is obtained and BEV upper reference point

[0037] (b) Attention mechanism operation

[0038] The query Q at level l l :Exchange information between instance queries through a multi-head self-attention mechanism (Self-Attention, SA), and interact with the multi-scale BEV feature F through cross-attention (Cross-Attention, CA), and then update the query Q through a fully connected layer (FFN) l+1 For a map element instance, there are usually multiple queries competing for a match with the ground-truth label. The decoding process at each layer can be viewed as a process where the weight of important queries gradually increases. Therefore, during the decoding process, a ranking between queries is introduced, rewarding high-ranking queries in self-attention and penalizing low-ranking queries in cross-attention.

[0039] The ranking score R of the lth layer (l>0) decoding layer l (i,j) is defined as:

[0040]

[0041] in, and is the classification score corresponding to the i-th and j-th query in the l-1 layer. Using this order, in the normal self-attention operation process, only the original attention weight Updated to A l :

[0042] D l =log(sigmoid(Conv 1×1 (R l ))), (2)

[0043]

[0044] Among them, Conv 1×1 is a convolution operation with a 1x1 convolution kernel. Similarly, the suppression amount D is calculated for queries with low ranking scores in the cross attention. l , and is used to update the attention weight W of the cross attention l ;

[0045]

[0046] D l =log(sigmoid(Conv 1×1 (L l))), (5)

[0047] In the cross-attention interaction, for each reference point, N is calculated using the query and position encoding. np Sampling points S l and the corresponding attention weight W l , use these sample points to aggregate features on BEV through deformable attention (Deformable Attention, Deform_Attn, DADA) to enhance the features of the reference points:

[0048] ΔS l =Conv 1×1 (Q l′ +PE l ), (6)

[0049] S l =(RF l +ΔS l ), (7)

[0050] W l =D l *Softmax(Conv 1×1 (Q l′ +PE l )), (8)

[0051] Q l″ =Deform_Attn(W l ,S l ,F), (9)

[0052] Among them, l represents the number of decoding layers, Q l′ is the query after self-attention update, PE l is the position embedding of the lth layer, ΔS l The sampling point offset calculated for the lth layer is extended to the same dimension as the offset and added to it to obtain the sampling point S l F is the fused BEV feature, which is used to sample key features in variable attention interaction. Q l″ The query is updated after the cross attention operation. Finally, the query input Q of the next layer is obtained through the fully connected network. l+1 .

[0053] In each decoding layer, the updated query Q l+1 By classifying the header CLS l and the return header REG l Predict the category scores and point set coordinates of lane lines, lane center lines, and other road elements. Sum the element point coordinates obtained by regression with the reference point to obtain the updated reference point:

[0054] RF l+1 =sigmoid(sigmoid -1 (RF l )+REG l (Q l+1 )), (10)

[0055] The results obtained by the last layer of decoding, classification head, and regression head are used as the final prediction.

[0056] The constructed network is trained and optimized on the training data to obtain a trained model. The optimization objectives include minimizing the category loss L cls and point-to-point position loss L p2p :

[0057]

[0058]

[0059] Among them, N pos and N are the number of positive samples and the total number of samples respectively. Focal represents the Focal loss function, D Manhattan represents the Manhattan distance.

[0060] This method uses sampling point features with prior information to update instance-level queries to better capture the categories and shapes of road elements. By introducing noise reduction queries and a query selection mechanism based on classification score ranking, it accelerates model convergence and improves model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Network structure diagram.

[0062] Figure 2 Schematic diagram of the expansion of the high-precision map. (a) is a schematic diagram of the high-precision map, which contains the basic topological structure of the road (road connection coordinates) and key metadata (number of lanes, traffic flow direction, road type). The high-precision map uses nodes to represent intersections, and the edges between nodes represent the road centerlines, with attributes including road name, number of lanes, road level, speed limit, etc. (b) The high-precision map can provide lane-level information and a rich set of map elements including lane markings, centerlines, traffic signs, and traffic lights. Lane markings and centerlines are all directed line segments. (c) By expanding the high-precision map, a lane centerline prior can be constructed as a model input. The blue line is the expanded lane centerline vector, and the direction of the arrow indicates the direction of traffic flow. The red line indicates the road connection before and after the intersection. DETAILED DESCRIPTION

[0063] The specific implementation method includes the following steps: sensor input, standard map data preprocessing module, feature extraction, BEV perspective transformation, multimodal BEV feature fusion, map element decoding, and model training.

[0064] Sensor input: Obtain time-matched camera images and lidar point cloud data from an autonomous vehicle. For example, an autonomous vehicle equipped with seven cameras and one lidar generates seven images and one point cloud.

[0065] Pre-processing of standard map data: Extract the key road-level centerline position, traffic flow direction and number of lanes N in the standard map data, and process the standard map data into a form that can be used as model input. For non-intersection areas, take the road centerline as the central axis and expand multiple parallel lines symmetrically to both sides as lane centerlines. The distance between adjacent parallel lines is the preset fixed lane width. Specifically, when the road centerline corresponds to a one-way traffic flow direction, if the number of lanes N is an odd number, keep the road centerline as the central axis and expand to the left and right sides. parallel lines; if the number of lanes N is an even number, the center line of the road is used as the central axis and extended to the left and right sides. The original road centerline is not retained. The direction of the expanded lane centerline is consistent with the traffic direction of the road centerline. When the traffic direction of the road centerline is bidirectional, N lanes are expanded on both sides of the centerline. l and N r parallel lines, where N l and N r are the number of lanes on both sides. For each lane centerline, 20 points are uniformly sampled as the prior query of the non-intersection lane centerline and provided as the model input. Figure 2 The blue line in (c) is the expanded lane centerline vector, and the arrow indicates the direction of traffic flow. When traffic flow is unidirectional and there are three lanes, one parallel line is extended to the left and right of the road centerline, forming three parallel lines to represent the three lane centerlines. When there are two lanes, one parallel line is extended to the left and right of the road centerline, leaving the original road centerline unchanged, forming two parallel lines to represent the two lane centerlines. When traffic flow is bidirectional and there are one and two lanes on either side, one and two parallel lines are extended to the left and right of the road centerline, respectively, leaving the original road centerline unchanged, forming one and two lane centerlines for different traffic flow directions.

[0066] Then, based on lane-level traffic information, the road topology within the intersection area is constructed, and the centerline of the lane before the intersection is connected to the centerline of the target lane via the input and output vectors. Specifically, for the centerline of the lane entering the intersection (the lane centerline is pointing towards the intersection), the input vector is taken from the end of the lane centerline and extended 5 meters into the intersection as the input vector end point, forming a directional line segment. For the centerline of the lane exiting the intersection (the lane centerline is pointing away from the intersection), the outgoing vector is extended 5 meters into the intersection from the starting point of the lane centerline and extended 5 meters into the intersection as the starting point of the outgoing vector, forming a directional line segment with the starting point of the lane centerline as the output vector end point. 10 points are uniformly sampled for each of the input and output vector segments, forming a vector polyline with a total of 20 points, which serves as a prior query for the intersection centerline and is provided as model input. Figure 2 The red line in (c) indicates the road connection before and after the intersection.

[0067] Feature extraction: From the 7 images and 1 point cloud input obtained from the sensor, the two-dimensional features of the multi-view images and the BEV features of the point cloud are extracted respectively. For image input, a convolutional neural network (CNN) is used to extract features from each image, and multi-scale features after 4, 8, 16, and 32 times downsampling are obtained respectively. Then, the multi-scale features are input into a network with a bidirectional feature pyramid (Bi-FPN) structure to obtain the image features after aggregation of each perspective. For a 120-degree forward laser radar on the vehicle, the point cloud data is converted into columnar features in the BEV space as a 2D pseudo image. Subsequently, a convolutional neural network is used as the backbone network for point cloud feature extraction, and SECONDFPN is used as the neck network to fuse multi-scale features to obtain the BEV features of the point cloud.

[0068] BEV perspective transformation: Convert the two-dimensional features of the multi-view image into image features in the BEV space. A set of learnable BEV queries are predefined in a grid format. The BEV queries are interacted with the aforementioned image features through an attention mechanism to generate multi-scale features under BEV. The resulting BEV features are then fused at multiple scales through a bidirectional feature pyramid.

[0069] Multimodal BEV feature fusion: Fusion of the image and point cloud features in the BEV space. Convolution operations are performed on the image BEV features and point cloud BEV features before concatenation.

[0070] Map element decoding: Inputs are fused features in the BEV space, randomly initialized initial queries, queries processed from standard map data, and noisy queries. After model calculations, the predicted map element category and location information are output. Specifically, this process involves two steps: query construction and attention interaction.

[0071] (a) Constructing a query

[0072] The map element decoding module performs multiple attention interactions between map queries and fused BEV features. Unlike some methods that use point queries, which lead to redundant modeling and high computational complexity of the self-attention module, this method uses instance-level queries. Instance-level queries include initialization queries, precision map prior queries, and noisy queries.

[0073] Suppose the initialization query: Prior query of precise map: Noisy query: Every element Indicated as N v The set of ordered points on the BEV plane, N init ,N sd ,g·N noised They represent the number of initialization queries, prior queries and noisy queries respectively, g is the number of groups of noisy queries, N noised is the number of queries in each group.

[0074] For the initial query Q init , each query i corresponds to a set of reference points in the two-dimensional normalized BEV coordinate system The reference points are obtained by a set of learnable parameters through a multi-layer perceptron (MLP) operation, where the j-point coordinates of the i-th group of reference points are defined as rf ij =(x ij ,y ij ).

[0075] For the prior query Q sd After preprocessing the map data, the map provides additional instance query input. Among them, 1 is the dimension corresponding to the category, here we use the intersection centerline or road centerline attribute of the precision map, 2 represents the number of coordinates of each point (x and y in BEV space), and the category dimension is encoded as the query through the multi-layer perceptron N v After the BEV range is normalized, the element points of the standard map are used as reference points corresponding to the query. Here N v is 20, which means 20 points uniformly sampled on the lane centerline-level intersection centerline after preprocessing.

[0076] For the noisy query Q noised The goal is to recover the correct labels from noisy queries without using Hungarian bipartite matching to address the instability of Hungarian matching. To make the noisy queries as diverse as possible, we introduce g groups of noisy queries, each containing the true labels of all samples, and add random noise to the true labels:

[0077] (i) For each point of the map element label, add random position offset noise (Δx, Δy) and ensure where λ∈(0,1), d min The minimum distance between two adjacent points on a map element to ensure that the map element does not have an unreasonable shape after adding noise.

[0078] (ii) For the categories of map element labels, a certain proportion of the labels are reversed to other categories. After being encoded by a multi-layer perceptron, it is converted into a noisy query. The denoising task is only used during training. During inference, the denoising part will be removed, so it does not incur additional computational overhead on the actual vehicle.

[0079] After the above three types of queries and reference points are spliced ​​together, the total instance query is obtained and BEV upper reference point

[0080] (b) Attention mechanism operation

[0081] The query Q at level l l :Exchange information between instance queries through a multi-head self-attention mechanism (Self-Attention, SA), and interact with the multi-scale BEV feature F through cross-attention (Cross-Attention, CA), and then update the query Q through a fully connected layer (FFN) l+1 For a map element instance, there are usually multiple queries competing for a match with the ground-truth label. The decoding process at each layer can be viewed as a process where the weight of important queries gradually increases. Therefore, during the decoding process, a ranking between queries is introduced, rewarding high-ranking queries in self-attention and penalizing low-ranking queries in cross-attention.

[0082] The ranking score R of the lth layer (l>0) decoding layer l (i,j) is defined as:

[0083]

[0084] in, and is the classification score corresponding to the i-th and j-th query in the l-1 layer. Using this order, in the normal self-attention operation process, only the original attention weight Updated to A l :

[0085] D l =log(sigmoid(Conv1×1 (R l )))

[0086]

[0087] Among them, Conv 1×1 is a convolution operation with a 1x1 convolution kernel. Similarly, the suppression amount D is calculated for queries with low ranking scores in the cross attention. l , and is used to update the attention weight W of the cross attention l .

[0088]

[0089] D l =log(sigmoid(Conv 1×1 (L l )))

[0090] In the cross-attention interaction, for each reference point, N is calculated using the query and position encoding. np Sampling points S l and the corresponding attention weight W l , use these sample points to aggregate features on BEV through deformable attention (Deformable Attention, Deform_Attn) to enhance the features of the reference points:

[0091] ΔS l =Conv 1×1 (Q l′ +PE l )

[0092] S l =(RF l +ΔS l )

[0093] W l =D l *Softmax(Conv 1×1 (Q l′ +PE l ))

[0094] Q l″ =Deform_Attn(W l ,S l ,F)

[0095] Among them, l represents the number of decoding layers, Q l′ is the query after self-attention update, PE l is the position embedding of the lth layer, ΔS lThe sampling point offset calculated for the lth layer is extended to the same dimension as the offset and added to it to obtain the sampling point S l F is the fused BEV feature, which is used to sample key features in variable attention interaction. Q l″ The query is updated after the cross attention operation. Finally, the query input Q of the next layer is obtained through the fully connected network. l+1 .

[0096] In each decoding layer, the updated query Q l+1 By classifying the header CLS l and the return header REG l Predict the category scores and point set coordinates of lane lines, lane center lines, and other road elements. Sum the element point coordinates obtained by regression with the reference point to obtain the updated reference point:

[0097] RF l+1 =sigmoid(sigmoid -1 (RF l )+REG l (Q l+1 ))

[0098] The results obtained by the last layer of decoding, classification head, and regression head are used as the final prediction.

[0099] Model training: The network constructed above needs to be trained and optimized on the training data to obtain a trained model. The optimization objectives include minimizing the category loss L cls and point-to-point position loss L p2p :

[0100]

[0101] Among them, N pos and N are the number of positive samples and the total number of samples respectively. Focal represents the Focal loss function, D Manhattan represents the Manhattan distance.

[0102] In this example, the experimental dataset spanned November 2023 to May 2024, collected in Jiading District, Shanghai. It contained over 5,000 30-second data segments. Keyframes were extracted at 5 Hz, and data not covered by the map or with map errors was removed, resulting in 843,593 frames of training data and 1,874 frames of test data. Each frame consisted of seven RGB images covering 360° around the vehicle and one LiDAR point cloud covering 120° in front of the vehicle.

[0103] The model and optimization target were constructed according to the above facts and trained on 8 NVIDIA A100 graphics cards. The optimizer used AdamW and cosine annealing learning rate scheduling with a learning rate of 4e -4 , the batch size is 16, and the training round is 24. The perception range of training and testing is the horizontal x-axis [-40m, 40m] and the vertical y-axis [-40m, 80m].

[0104] As evaluation indicators, mean Average Precision (mAP, the higher the better) is used to measure the overall quality of map reconstruction, and Chamfer Distance (CD, the lower the better) is used to measure the degree of positional match between predicted map elements and true values.

[0105] The comparison in the table below shows that compared with the traditional baseline model (a), the introduction of the standard map prior query of the present invention in (b) is more simple and direct, as the map prior provides beyond-line-of-sight perception and preliminary road topology information, and the present invention models the map as a set of vectorized points and lines identical to the initial query. This significantly improves both detection quality and position accuracy (mAP increased by 12.2% and CD error decreased by 0.003 meters), demonstrating its effectiveness. In (c), after replacing the traditional map element decoding module with the decoding module of the present invention, the mAP increased by 10.3% and CD decreased by 0.066 meters due to the noisy training and improved attention mechanism introduced by the present invention, demonstrating the effectiveness of the training method of the present invention. Finally, after adding the forward lidar in (d), the mAP increased by 2% and CD decreased by 0.048 meters due to the higher spatial position accuracy of the point cloud, demonstrating the superiority of the fusion module of the present invention. In combination with the aforementioned improvement method, the results are far superior to those of the baseline model.

[0106] mAP CD (a) Traditional baseline model 53.3 0.659 (b) Added prior query for standard and precise maps 65.5 0.656 (c) Modified to the decoding module of the present invention 75.8 0.590 (d) Add forward-facing lidar 77.8 0.552 .

Claims

1. A method for constructing a local vector map for autonomous driving based on a standard precision map and fusion perception, characterized in that: This includes preprocessing of accurate map information, building and training a fusion perception network. The specific steps are as follows: (1) Preprocessing of accurate map information; The basic topological structure of the road is contained in the map. First, the key road-level centerline position, traffic flow direction, and the number of lanes N in the road are extracted. For non-intersection areas, multiple parallel lines are symmetrically extended to both sides of the road centerline with the road centerline as the central axis. The distance between adjacent parallel lines is the preset fixed lane width. Specifically, When the traffic direction corresponding to the center line of the road is one-way, if the number of lanes N is an odd number, the center line of the road is retained as the central axis and the lanes are expanded to the left and right sides. parallel lines; if the number of lanes N is an even number, the center line of the road is used as the central axis and extended to the left and right sides. The original road centerline is not retained; the direction of the expanded lane centerline is consistent with the traffic flow direction of the road centerline; When the traffic direction of the road centerline is bidirectional, N is extended to both sides of the centerline. l and N r parallel lines, where N l and N r are the number of lanes on both sides respectively; for each lane centerline, 20 points are uniformly sampled as the prior query of the non-intersection lane centerline and provided as the model input; Then, based on lane-level traffic information, the road topology within the intersection area is constructed, and the centerline of the lane before the intersection is connected to the centerline of the target lane through the input and output vectors. Specifically, for the centerline of the lane entering the intersection, the input vector starts from the end of the lane centerline and extends 5 meters into the intersection as the input vector end, forming a directional line segment. For the centerline of the lane exiting the intersection, the outgoing vector starts from the starting point of the lane centerline and extends 5 meters into the intersection as the outgoing vector end, forming a directional line segment. Ten points are uniformly sampled from each of the input and output vector segments to form a vector polyline with a total of 20 points. This is used as a priori query for the intersection centerline and provided as model input. (2) Build and train a fusion perception network model; The fusion perception network model includes a feature extraction backbone network, a BEV perspective transformation network, a multimodal BEV feature fusion network, and a map element decoding module; among them: The feature extraction backbone network uses a convolutional neural network (CNN) to extract features from each image of the vehicle's seven cameras covering 360 degrees, namely, the front close-range, front long-range, left front, right front, left rear, right rear, and rear cameras, and obtains multi-scale features after 4, 8, 16, and 32 times downsampling, respectively. The multi-scale features are then input into a network with a bidirectional feature pyramid (Bi-FPN) structure to obtain image features after aggregation from each perspective. For a 120-degree forward laser radar on the vehicle, the point cloud data is converted into columnar features in the BEV space as a 2D pseudo image. The convolutional neural network is then used as the backbone network for point cloud feature extraction, and a SECONDFPN is used as the neck network to fuse the multi-scale features to obtain the BEV features of the point cloud. The BEV perspective transformation network predefines a set of grid-like learnable BEV queries, interacts the BEV queries with the aforementioned image features through an attention mechanism, thereby generating multi-scale features under BEV, and then performs multi-scale fusion of image BEV features through a bidirectional feature pyramid; The multimodal BEV feature fusion network performs convolution operation on the aforementioned image BEV features and point cloud BEV features and then splices them; The map element decoding module performs multiple attention mechanism interactions on the map query in the standard map information preprocessing stage and the fused BEV features; and the results obtained by the classification head and regression head are used as the final prediction.

2. The method for constructing a local vector map for autonomous driving according to claim 1, wherein: The map element decoding module includes two steps: query construction and attention interaction: (a) Constructing a query Specifically, instance-level queries are used, including initialization queries, prior queries on the labeled map, and noisy queries. Suppose the initialization query: Prior query of precise map: Noisy query: Every element Indicated as N v The set of ordered points on the BEV plane, N init ,N sd ,g·N noised They represent the number of initialization queries, prior queries and noisy queries respectively, g is the number of groups of noisy queries, N noised is the number of queries in each group; For the initial query Q init , each query i corresponds to a set of reference points in the two-dimensional normalized BEV coordinate system The reference point RF is obtained by a set of learnable parameters through a multi-layer perceptron (MLP) operation, where the j-point coordinates of the i-th group of reference points are defined as rf ij =(x ij ,y ij ); For the prior query Q sd After preprocessing the map data, the map provides additional instance query input. Among them, 1 is the dimension corresponding to the category, here we use the intersection centerline or road centerline attribute of the standard map, 2 represents the number of coordinates of each point, and the category dimension is encoded into the query through the multi-layer perceptron N v After the BEV range is normalized, the element points of the standard map are used as reference points corresponding to the query. Here N v is 20, which means 20 points uniformly sampled on the lane centerline-level intersection centerline after preprocessing; For the noisy query Q noised The goal is to recover the correct labels from noisy queries. To make the noisy queries as diverse as possible, we introduce g groups of noisy queries, each containing the true value labels of all samples, and add random noise to the true value labels: (i) For each point of the map element label, add random position offset noise (Δx, Δy) and ensure where λ∈(0,1), d min The minimum distance between two adjacent points on a map element to ensure that the map element after adding noise has no unreasonable shape; thus, the reference point of the noisy query is obtained. (ii) For the categories of map element labels, 30%-70% of the labels are reversed to other categories; the labels are encoded into noisy queries through a multi-layer perceptron. The denoising task is only used in training. During inference, the denoising part will be removed. After the above three types of queries and reference points are spliced ​​together, the total instance query is obtained and BEV upper reference point (b) Attention Mechanism Interaction The query Q at level l l :Exchange information between instance queries through a multi-head self-attention mechanism (SA), and perform cross-attention (CA) interaction with the multi-scale BEV feature F, and then update the query Q through a fully connected layer (FFN) l+1 For a map element instance, multiple queries compete for a match with the true value label. Each layer of decoding is regarded as a process in which the weight of important queries gradually increases. Therefore, in the decoding process, the ranking between queries is introduced, and high-ranking queries are rewarded in self-attention and low-ranking queries are penalized in cross-attention. The ranking score R of the lth layer (l>0) decoding layer l (i,j) is defined as: in, and is the classification score corresponding to the i-th and j-th query in the l-1 layer; using this order, in the normal self-attention operation process, only the original attention weight Updated to A l : Among them, Conv 1×1 is a convolution operation with a 1x1 convolution kernel; similarly, the suppression amount D is calculated for queries with low ranking scores in the cross attention. l , and is used to update the attention weight W of the cross attention l ; In the cross-attention interaction, for each reference point, N is calculated using the query and position encoding. np Sampling points S l and the corresponding attention weight W l , use these sample points to aggregate features on BEV through deformable attention (DADA) to enhance the features of the reference points: ΔS l =Conv 1×1 (Q l′ +PE l ),(6) S l =(RF l +ΔS l ),(7) W l =D l *Softmax(Conv 1×1 (Q l′ +PE l )),(8) Q l″ =Deform_Attn(W l ,S l ,F),(9) Among them, l represents the number of decoding layers, Q l′ is the query after self-attention update, PE l is the position embedding of the lth layer, ΔS l The sampling point offset calculated for the lth layer is extended to the same dimension as the offset and added to it to obtain the sampling point S l ; F is the fused BEV feature, which is used to sample key features in variable attention interaction; Q l″ The query is updated after the cross attention operation; finally, the query input Q of the next layer is obtained through the fully connected network l+1 ; In each decoding layer, the updated query Q l+1 The classification head CLS and the regression head REG predict the category scores and point set coordinates of lane lines, lane center lines, and other road elements. The coordinates of the element points obtained by regression are summed with the reference point to obtain the updated reference point: RF l+1 =sigmoid(sigmoid -1 (RF l )+REG l (Q l+1 )),(10) The results obtained by the last layer of decoding, classification head, and regression head are used as the final prediction; The constructed network is trained and optimized on the training data to obtain a trained model; its optimization objectives include minimizing the category loss L cls and point-to-point position loss L p2p : Among them, N pos , N are the number of positive samples and the total number of samples respectively; L Focal represents the Focal loss function, D Manhattan represents the Manhattan distance.

Images