Automatic driving map uncertainty quantification method and device, equipment and medium

Abstract

The invention discloses an automatic driving map uncertainty quantification method and device, equipment and a medium, and relates to the field of cognition of autonomous unmanned systems, and the method comprises the steps: obtaining a target multi-view image of a target scene where an automatic driving vehicle runs; preprocessing the target multi-view image to obtain target data to be predicted; inputting target to-be-predicted data into the trained online estimation map network, and obtaining fused map uncertainty vehicle trajectory data based on the gamma distribution uncertainty probability model; and inputting the vehicle trajectory data fused with the map uncertainty into the trained trajectory prediction model, and predicting to obtain the future driving trajectory of the autonomous vehicle. The method employs an online map generation method based on Gamma distribution, achieves the quantification of the map uncertainty, integrates the quantized uncertainty into trajectory prediction, and improves the prediction precision of the autonomous vehicle. And the track prediction precision of the automatic driving vehicle is improved.

Description

Technical Field

[0001] The present application relates to the cognitive field of autonomous unmanned systems, and in particular to a method, device, equipment and medium for quantifying uncertainty in autonomous driving maps. Background Art

[0002] As a complex system that integrates artificial intelligence, robotics, computer vision and sensor technology, autonomous driving technology aims to achieve a safe, reliable and efficient unmanned driving experience through autonomous perception, decision-making and control. In this process, environmental perception and map construction are the core prerequisites for achieving vehicle autonomous navigation.

[0003] Environmental perception is the foundation of autonomous driving systems, which rely on a variety of sensors, such as lidar, cameras, etc. These sensors work together to provide detailed information about the vehicle's surroundings, including road structures, traffic signs, the location of pedestrians and other vehicles, etc. However, sensor data is susceptible to noise, occlusion, and environmental changes, resulting in high uncertainty in perception results. Therefore, how to effectively deal with sensor data noise to improve perception accuracy is an important area in autonomous driving research.

[0004] Map building is another key step for autonomous vehicles to achieve autonomous navigation. High-precision maps (HD Maps) contain not only geometric information of the road, but also lane lines, traffic signs, signal lights and other static environmental features. This information is crucial for vehicle path planning and decision-making. However, HD maps are expensive to annotate and maintain, and can only be used in pre-mapped areas, limiting the scalability of autonomous vehicles. To address these issues, many studies in recent years have been devoted to online estimation of HD maps from sensor data, enabling autonomous vehicles to operate in unmapped areas.

[0005] Current online map estimation methods mainly focus on predicting the position and category of map mapping elements based on camera images and lidar scan data, usually represented as polygons or polylines. However, these methods are usually developed independently of downstream autonomous driving tasks and lack estimates of uncertainty or credibility. Sensor noise, data loss, and sudden changes in the environment will lead to increased uncertainty in map estimation. If the above uncertainties are not effectively quantified and handled, the downstream trajectory prediction module may mistakenly assume that these components are accurate when processing the inferred HD map components, which may lead to potential erroneous behavior and affect the accuracy of trajectory prediction. Summary of the invention

[0006] The purpose of this application is to provide a method, device, equipment and medium for quantifying uncertainty in autonomous driving maps, which realizes map uncertainty quantification and improves the trajectory prediction accuracy of autonomous driving vehicles.

[0007] To achieve the above objectives, this application provides the following solutions.

[0008] In a first aspect, the present application provides a method for quantifying uncertainty in an autonomous driving map, comprising the following steps.

[0009] A target multi-perspective image of a target scene in which an autonomous driving vehicle is traveling is acquired; the target multi-perspective image includes surround view images of different perspectives; the surround view image is captured by a camera on the autonomous driving vehicle.

[0010] The target multi-view images are preprocessed to obtain target data to be predicted.

[0011] The target data to be predicted is input into a trained online estimated map network to obtain vehicle trajectory data with fused map uncertainty; the online estimated map network includes a feature extractor, a detection head, a map construction module and a post-processing module.

[0012] The feature extractor is used to extract features from the target multi-view images to obtain bird's-eye view features.

[0013] The detection head includes a regression head and a classification head; the regression head is used to: generate the position of each map mapping element vertex in each surround image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view features based on a gamma distribution uncertainty probability model; the classification head is used to: generate a confidence score for the category of each map mapping element vertex according to the bird's-eye view features; each map mapping element corresponds to a number of vertices.

[0014] The map construction module is used to generate a predicted estimated map of the target scene based on the positions of all map mapping element vertices in all surround images, the confidence scores of the categories, the regression uncertainty parameters corresponding to all map mapping elements, and the target multi-view images.

[0015] The post-processing module is used to fuse the predicted estimation map with the historical trajectory data of the autonomous driving vehicle to obtain fused map uncertainty vehicle trajectory data.

[0016] The fused map uncertainty vehicle trajectory data is input into a trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

[0017] In a second aspect, the present application provides an autonomous driving map uncertainty quantification device, comprising the following modules.

[0018] The target multi-view image acquisition module is used to: acquire a target multi-view image of a target scene in which an autonomous driving vehicle is traveling; the target multi-view image includes surround view images of different view angles; and the surround view image is captured by a camera on the autonomous driving vehicle.

[0019] The preprocessing module is used to preprocess the target multi-view images to obtain target data to be predicted.

[0020] The map online estimation module is used to input the target data to be predicted into the trained online estimation map network to obtain the fused map uncertainty vehicle trajectory data; the online estimation map network includes a feature extractor, a detection head, a map construction module and a post-processing module; wherein the feature extractor is used to extract features from the target multi-view image to obtain the bird's-eye view features; the detection head includes a regression head and a classification head; the regression head is used to generate the position of each map mapping element vertex in each surround view image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view features based on the gamma distribution uncertainty probability model; the classification head is used to generate the confidence score of the category of each map mapping element vertex according to the bird's-eye view features; each map mapping element corresponds to a number of vertices; the map construction module is used to generate the predicted estimation map of the target scene according to the positions of all map mapping element vertices in all surround view images, the confidence scores of the categories, the regression uncertainty parameters corresponding to all map mapping elements and the target multi-view images; the post-processing module is used to fuse the predicted estimation map with the historical trajectory data of the autonomous driving vehicle to obtain the fused map uncertainty vehicle trajectory data.

[0021] The future trajectory prediction module is used to: input the fused map uncertainty vehicle trajectory data into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

[0022] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned autonomous driving map uncertainty quantification method.

[0023] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned autonomous driving map uncertainty quantification method.

[0024] According to the specific embodiments provided in the present application, the present application discloses the following technical effects: The present application provides a method, device, equipment and medium for quantifying uncertainty in an autonomous driving map, utilizing multi-perspective image information of a target scene captured by a camera, and adopting an online map generation method based on a gamma distribution uncertainty probability model to generate the position of each map mapping element vertex and the regression uncertainty parameter corresponding to each map mapping element, aiming to capture the uncertainty when generating a high-definition map, and integrate the quantified uncertainty into the downstream trajectory prediction task, thereby improving the trajectory prediction accuracy of the autonomous driving vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 This is an application environment diagram of an autonomous driving map uncertainty quantification method in one embodiment of the present application.

[0027] Figure 2 A flowchart of a method for quantifying uncertainty in an autonomous driving map provided in one embodiment of the present application.

[0028] Figure 3 This is a schematic diagram of the specific process of the autonomous driving map uncertainty quantification method provided in one embodiment of the present application.

[0029] Figure 4 This is a schematic diagram of the online map estimation process provided in one embodiment of the present application.

[0030] Figure 5 This is a schematic diagram of uncertainty transfer provided in an embodiment of the present application.

[0031] Figure 6 This is a schematic diagram of the fusion of the predicted estimation map and historical trajectory data provided in one embodiment of the present application.

[0032] Figure 7 A schematic diagram of the functional modules of an autonomous driving map uncertainty quantification device provided in one embodiment of the present application.

[0033] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] The autonomous driving map uncertainty quantification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the target multi-view image to the server 104. After the server 104 receives the target multi-view image, the server 104 pre-processes the target multi-view image to obtain the target data to be predicted, and inputs the target data to be predicted into the trained online estimation map network to obtain the fused map uncertainty vehicle trajectory data, and inputs the fused map uncertainty vehicle trajectory data into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle. The server 104 can feed back the obtained future driving trajectory of the autonomous driving vehicle to the terminal 102. In addition, in some embodiments, the autonomous driving map uncertainty quantification method can also be implemented separately by the server 104 or the terminal 102, such as the terminal 102 can directly perform trajectory prediction processing on the target multi-view image, or the server 104 can obtain the target multi-view image from the data storage system and perform trajectory prediction processing on the target multi-view image.

[0037] The terminal 102 may be, but is not limited to, various desktop computers and laptop computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0038] In an exemplary embodiment, Figure 2 and Figure 3 As shown, a method for quantifying uncertainty of an autonomous driving map is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1The server 104 in the example is used for explanation, and the steps include the following steps 201 to 204.

[0039] Step 201: Acquire a target multi-view image of a target scene in which an autonomous driving vehicle is traveling; the target multi-view image includes surround view images of different view angles; the surround view image is captured by a camera on the autonomous driving vehicle. The camera is installed on the top of the autonomous driving vehicle.

[0040] Step 202: preprocess the target multi-view image to obtain target data to be predicted.

[0041] Step 203: input the target data to be predicted into the trained online estimation map network to obtain fused map uncertainty vehicle trajectory data.

[0042] The online estimated map network includes a feature extractor, a detection head, a map construction module and a post-processing module.

[0043] The feature extractor is used to extract features from the target multi-view images to obtain bird's-eye view features.

[0044] The detection head includes a regression head and a classification head; the regression head is used to: generate the position of each map mapping element vertex in each surround image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view features based on a gamma distribution uncertainty probability model; the classification head is used to: generate a confidence score for the category of each map mapping element vertex according to the bird's-eye view features; each map mapping element corresponds to a number of vertices.

[0045] The map construction module is used to generate a predicted estimated map of the target scene based on the positions of all map mapping element vertices in all surround images, the confidence scores of the categories, the regression uncertainty parameters corresponding to all map mapping elements, and the target multi-view images.

[0046] The post-processing module is used to fuse the predicted estimation map with the historical trajectory data of the autonomous driving vehicle to obtain fused map uncertainty vehicle trajectory data.

[0047] Step 204: Input the fused map uncertainty vehicle trajectory data into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

[0048] Implement the above-mentioned steps 201 to 204, use the target multi-view image information of the target scene captured by the camera, and adopt the online map generation method of the gamma distribution uncertainty probability model to generate the position of each map mapping element vertex and the regression uncertainty parameter corresponding to each map mapping element, in order to capture the uncertainty when generating a high-definition map, integrate the quantified uncertainty into the downstream trajectory prediction task, and improve the trajectory prediction accuracy of the autonomous driving vehicle.

[0049] Before the step of inputting the target data to be predicted into the trained online estimated map network, the autonomous driving map uncertainty quantification method also includes a training process of the online estimated map network, including steps 301 to 302.

[0050] Step 301: Obtain a sample data set; the sample data set includes a number of sample multi-view images, positions and categories of points in the sample multi-view images, and true trajectories of the samples.

[0051] Step 302: Using the sample data set, train the online estimated map network to obtain a trained online estimated map network.

[0052] In a specific embodiment, a sample dataset is obtained using the nuScenes dataset, which contains image data captured by a camera. The sample dataset is formatted and preprocessed by code to generate a .pkl file containing basic information, export 2D annotation information (the 2D annotation information is the location and category of the points in the sample multi-view image), and create a ground-truth database, converting the original data into a format suitable for deep learning tasks, so as to facilitate the training and evaluation of the online estimation map network.

[0053] In step 202, preprocessing includes noise reduction preprocessing.

[0054] The nuScenes dataset provides ground truth high-definition maps, sensor data, and agent trajectories. Sensor data refers to the multi-view image data captured by the on-board camera, and agent trajectories refer to the real historical trajectory data of the autonomous vehicle. The nuScenes dataset consists of 1,000 driving scenes. Each scene is sampled at a frequency of 2 Hz to obtain a sample dataset, and is divided into training, validation, and test sets, containing 500, 200, and 150 scenes, respectively. trajdata is used to provide a unified interface between the vectorized online estimation map network and the downstream trajectory prediction model, and the time interpolation tool is used to increase the data frequency of nuScenes from the original 2 Hz to 10 Hz.

[0055] Build an online map estimation network that can quantify uncertainty. The online map estimation network is based on the MapTR network, which is an encoder-decoder Transformer architecture. Using the principles and training methods of the MapTR network, an online map estimation network that can quantify the uncertainty in the map estimation stage is designed, such as Figure 4 As shown in Figure 1, the online map estimation network includes a feature extractor, a detection head, a map construction module, and a post-processing module. The network can quantify map uncertainty for classification and regression tasks during map estimation, and pass the uncertainty to the downstream trajectory prediction task.

[0056] Feature extractor (i.e. Figure 4 The map encoder in the figure uses the surround view image captured by the camera on the top of the autonomous vehicle as input. The feature extractor includes an initial feature extraction module and a bird's eye view (BEV) conversion module (PV-to-BEV conversion module). The initial feature extraction module uses a 2D backbone network to extract features of the input data and extract 2D image features from multi-view image data. Preferably, the 2D backbone network can use a ResNet50 model.

[0057] Bird’s-eye view conversion module: A geometry-guided Kernel Transformer (GKT) is used to convert 2D image features into bird’s-eye view features. The goal of the feature extractor is to convert the raw data (sample multi-view images) into a unified high-dimensional feature representation for subsequent processing.

[0058] The detection head consists of two submodules: regression head and classification head. The regression head is used to predict the position of the polyline vertices, and the classification head is used to predict the category to which the polyline vertices belong. Polylines refer to map mapping elements, and the classification head mainly recognizes three map mapping elements: road boundaries, crosswalks, and lane dividers (also called "lane dividers").

[0059] The regression head adopts the Gamma distribution modeling method and includes the following key components: feature extraction submodule and uncertainty calculation submodule.

[0060] The first is the feature extraction submodule, which consists of two linear layers and a ReLU activation function. It further transforms and refines the input BEV features to make them more suitable for subsequent uncertainty prediction tasks. The ReLU activation function is used to introduce nonlinear transformations to enhance the expressive power of the online estimation map network.

[0061] Then, based on the properties of the Gamma distribution, the mean and variance are calculated through the parameters of the network output, and they are spliced ​​together as the final output to characterize the uncertainty of the prediction. The softplus activation function and clamp operation are used to ensure that the parameters are within a reasonable range.

[0062] For the regression task, the regression head outputs a 2D vector representing the position coordinates of the normalized BEV. The Gamma distribution is introduced to convert it into a probability model that can output the uncertainty associated with the prediction point, namely the Gamma distribution uncertainty probability model. The regression uncertainty is quantified by the mean and standard deviation of the Gamma distribution output, and the position of each map mapping element vertex in each surround image and the regression uncertainty parameter corresponding to each map mapping element are obtained. The Gamma distribution is introduced to quantify the uncertainty in the online estimated map. The vertex of each map mapping element is modeled as , are the horizontal and vertical coordinates of the vertices respectively. So we have Map elements of vertices The joint probability density of , that is, the gamma distribution uncertainty probability model is expressed in the form shown in the following formula.

[0063] (1).

[0064] in, Mapping elements for the map The joint probability density of , Is the map element The vertex The shape parameter of the dimensional gamma distribution, Is the mapping element The vertex The scale parameter of the dimensional gamma distribution, is the gamma function; =1,2,3... ; Mapping elements for the map The total number of corresponding vertices.

[0065] Shape parameters Determines the shape of the gamma distribution. The larger the shape parameter, the higher the kurtosis of the distribution, which means that the data is more concentrated around the mean. Controls the scalability of the distribution. A larger scale parameter makes the distribution wider. Compared to the Gaussian distribution, the gamma distribution can present a variety of shapes by adjusting the shape parameter and scale parameter, and can effectively handle long-tailed, thick-tailed data and abnormal distribution situations. In addition, the gamma distribution is a right-skewed estimator, which is suitable for modeling regression uncertainty using variance and standard deviation.

[0066] In addition, in order to train a map estimation model that generates uncertainty, the negative log-likelihood (NLL) loss is introduced as the regression loss, and its formula is as follows.

[0067] (2).

[0068] in, represents the regression loss function, Represents the vertex The vertex The position coordinates of the dimensional gamma distribution. The regression loss function is used to calculate the negative log-likelihood loss of the 2D point position prediction and quantify the uncertainty of the estimated position prediction. Then the above step 302 specifically includes: using the negative log-likelihood loss function and the sample data set to train the online estimated map network to obtain a trained online estimated map network.

[0069] For classification tasks, the classification head predicts the category confidence score of each map element vertex, including road boundary vertices, pedestrian crossing vertices, and lane dividing line vertices. Since the classification head is a probabilistic classification distribution, the confidence score of each vertex category is directly output downstream to quantify the classification uncertainty.

[0070] The structure of the classification head is a multilayer perceptron (MLP). The classification head includes several classification submodules connected in sequence, each of which contains three key components: an input layer, a layer normalization layer, and a ReLU activation function connected in sequence. Its detailed loop structure includes the following components: an input layer, a layer normalization layer, a ReLU activation function, and an output layer; the input layer and the output layer are linear layers. The input layer is a linear layer that maps the input feature dimensions to the same dimension size. The layer normalization layer is used to normalize the output of the linear layer. The ReLU activation function is used to introduce nonlinearity. After passing through multiple classification submodules, the feature dimensions are finally mapped to the number of classification output channels through the output linear layer for the final classification output. These layers are encapsulated in a Sequential container and executed sequentially.

[0071] The map construction module integrates the predictions of the detection head (the detection results include the positions of all map element vertices in all surround images, the confidence scores of the categories, and the regression uncertainty parameters corresponding to all map elements) and the input data (i.e., multi-view images) into a standardized map representation by the prediction result converter, and outputs a structured dictionary containing all map element information. The map construction module introduces a hierarchical query embedding scheme to explicitly encode each map element and generate a predicted map representation of the scene.

[0072] Train the network model and input the processed data set into the online estimated map network for training. The model sets the total training cycle to 24 epochs. The training uses the AdamW optimizer and the learning rate is set to 1.5×10 -4 , and the maximum norm of gradient clipping is set to 3 to prevent gradient explosion. After this process, a trained map network is obtained.

[0073] The trained map network is evaluated in the validation set and the test set. The evaluation uses the Chamfer distance. to measure the distance between two maps. The average value of the closest distance from one point set to another point set is calculated, as shown in the following formula.

[0074] (3).

[0075] In the formula, To predict the set of vertices in the map, for A point in the point set, is the vertex set in the real map, for A point in a set of points.

[0076] The specific calculation process of the chamfer distance between the predicted map and the real map includes the following steps 401 to 404.

[0077] Step 401: Get A first point set point in , calculate the first point set point and The distance between all the second point set points in is taken as the minimum distance value between the first point set and The distance between them is recorded as the initial distance.

[0078] Step 402: Get The other first point set points in the , calculate the first point set points and The distance between each second point set in the All the first point set points in the , get the initial distances corresponding to all the first point set points; divide the initial distances corresponding to all the first point set points by The total number of points in the first point set is The first point in the set is The first average distance of the points in the second point set.

[0079] Step 403: The second point set point replaces the first point set point in step 401 and step 402, and steps 401 and 402 are repeated to obtain The second point set in The second average distance of the points in the first point set.

[0080] Step 404: Add the first average distance and the second average distance to obtain the chamfer distance between the predicted map and the real map .

[0081] To evaluate the accuracy of probabilistic map construction, the average precision (AP) is used as the evaluation metric, covering three map elements: road boundaries, crosswalks, and lane dividers, and the average AP (mAP) is calculated under three different chamfer distance thresholds (0.5m, 1.0m, and 1.5m).

[0082] The trained model is tested to generate a prediction map of the scene (i.e., a high-definition map). The post-processing module fuses the generated prediction map with the real historical trajectory data of the autonomous driving vehicle to obtain the fused map uncertainty vehicle trajectory data, such as Figure 6 shown.

[0083] The post-processing module adopts a multi-stage data fusion architecture: first, the historical trajectory data of the autonomous driving vehicle is loaded through UnifiedDataset, including vehicle location, orientation and other information. Then, the MapAPI is used to extract road element information from the original map, including lane dividing lines, road boundaries, etc. The predicted map mapping elements (including lane dividing lines, crosswalks, road boundaries, etc.) are aligned with the vehicle coordinate system through the coordinate transformation function, and the map mapping elements within the valid area are screened out. Finally, the processed predicted map and historical trajectory data are integrated into a unified data structure, and the fused data containing vehicle status, trajectory and corresponding map information is output, that is, the fused map uncertainty vehicle trajectory data.

[0084] The predicted map obtained by network testing is fused with the real historical trajectory data of the autonomous vehicle as the input of the subsequent trajectory prediction network. The fused map uncertainty vehicle trajectory data is input into the trajectory prediction model to predict the future driving trajectory of the autonomous vehicle.

[0085] The trajectory prediction model uses an encoder-decoder architecture. The encoder is responsible for processing the input historical trajectory and encoding it according to the scene context. The decoder uses the feature information extracted by the encoder to generate predictions for future trajectories.

[0086] In another exemplary embodiment of the present application, the trajectory prediction model is a model such as a HiVT model or a DenseTNT model. The HiVT model and the DenseTNT model are used to predict the future trajectory of the vehicle. The HiVT model uses a hierarchical Transformer architecture to capture the interaction between agents through local context extraction and global interaction modeling. It uses rotation and translation invariant scene representation to improve the accuracy and efficiency of prediction. The DneseTNT model uses an anchor-free end-to-end Transformer architecture to generate target probability distributions through sparse context encoding and dense target encoding, and directly outputs a set of trajectories through a target set predictor. For the HiVT trajectory prediction model, the map mapping element vertex is usually encoded as a token and vectorized (encoded) by a multi-layer perceptron (MLP) before entering the cross-attention layer of the Transformer to provide appropriate features for calculation. For the DenseTNT trajectory prediction model, the map mapping element vertices are usually encoded as nodes in the GNN to help the model understand the configuration of the physical space, and then take into account the physical constraints and road specifications of the real world when predicting the trajectory. In the trajectory prediction model, the vertex position coordinates are preliminarily encoded by the MLP, as shown in the following formula.

[0087] (4).

[0088] in, Indicates the encoded The position coordinates of the vertices, represents the MLP used to encode features, For the The original features of the vertices.

[0089] The encoded node features or token features are then incorporated into the message passing layer or attention layer. The trajectory prediction model is able to use these features for higher-level information processing, which enables the trajectory prediction model to integrate information from different input sources and form a global understanding of the scene.

[0090] In order to integrate the uncertainty information quantified in the estimated map stage into the trajectory prediction model, the uncertainty parameters of the vertices obtained based on the gamma distribution are encoded. The uncertainty parameters of the vertices include the shape parameters and scale parameter , the uncertainty scale parameter for each vertex It is directly connected to the mean of the point set and then encoded by the local neighborhood encoder together with the trajectory information of the autonomous driving vehicle, as shown in the following equation.

[0091] (5).

[0092] in, Represents the fusion information, including the positions of all map element vertices in all surround images, the confidence scores of the categories, and the regression uncertainty parameters corresponding to all map elements; Indicates that the information is connected in series. Indicates This encoding method allows the model to consider not only the location but also the uncertainty and other related information when processing map elements. The vertices of all map elements are encoded by the local neighborhood encoder to obtain the fused information corresponding to each vertex.

[0093] like Figure 5 As shown, this integration helps the trajectory prediction model better understand the uncertainty of map element estimates when making predictions and adjust its output accordingly, thereby improving the accuracy and confidence of trajectory predictions.

[0094] The following uses both the HiVT trajectory prediction model and the DenseTNT trajectory prediction model to predict the future trajectory of the vehicle. In order to introduce map uncertainty, the HiVT model takes the predicted map as a point set input, combines the vertex uncertainty directly in the encoder, and encodes it together with the proxy trajectory information. The DenseTNT model combines the uncertainty of the map mapping element vertices with the vertex coordinates into the lane feature encoding, and encodes it with VectorNet, aggregating local and global information through a layered graph neural network. Each trajectory prediction model in this application predicts the trajectory movement for the next 3 seconds from 2 seconds of historical moments. For trajectory prediction, standard evaluation indicators are used, including minimum average displacement error (minADE), minimum final displacement error (minFDE), and miss rate (MR). minADE calculates the average Euclidean distance between the most accurate predicted trajectory and the true trajectory in all future time steps. minFDE only calculates the error of the final prediction time step. MR measures the proportion of scenes where the end point of the best predicted trajectory deviates from the end point of the true trajectory by more than 2.0 meters.

[0095] The fused map uncertainty vehicle trajectory data is input into the trajectory prediction network: HiVT model and DenseTNT model for training. In the HiVT model, the batch size is set to 32 and the learning rate is set to 5×10 -4, the dropout rate is set to 0.2, and a total of 24 epochs of training are performed. In the DenseTNT model, the batch size is set to 16 and the learning rate is set to 5×10 -4 , the dropout rate is set to 0.1, and a total of 32 epochs of training are performed. Finally, the parameters are saved to obtain the final model. The relevant experiments quantitatively evaluate the trajectory prediction accuracy through evaluation indicators such as minimum average displacement error (minADE), minimum final displacement error (minFDE) and miss rate (MR). The results are shown in Table 1.

[0096] Table 1 Trajectory prediction accuracy performance comparison

[0097] MapTR-trajectory prediction refers to the trajectory prediction model that removes the uncertainty transfer process of the autonomous driving map uncertainty quantification method proposed in this application, that is, does not introduce uncertainty in the map estimation stage.

[0098] According to the data in Table 1, among all the compared methods, the autonomous driving map uncertainty quantification method proposed in this application presents the best performance. In addition, this application also explores the impact of uncertainty quantification on trajectory prediction accuracy, and evaluates the trajectory prediction model that does not introduce uncertainty in the map estimation stage. As shown in Table 1. When predicting the trajectory, not using the uncertainty passed down from the map estimation has a greater impact on the trajectory prediction accuracy. For both trajectory prediction models, the prediction errors have increased significantly. By passing the uncertainty quantified in the map estimation stage to the trajectory prediction model, the trajectory prediction accuracy is significantly improved.

[0099] The present application also provides an application scenario, which applies the above-mentioned autonomous driving map uncertainty quantification method. Specifically: The autonomous driving map uncertainty quantification method provided in this embodiment can be applied in autonomous driving control. The autonomous driving control scenario includes an information collection link, a trajectory prediction link and an autonomous driving link; the multi-view image enters the trajectory prediction link from the information collection link, obtains the corresponding future predicted trajectory through human-machine collaboration, and enters the downstream autonomous driving link. The autonomous driving map uncertainty quantification method provided in this embodiment belongs to the trajectory prediction link. Specifically, in the trajectory prediction link process for multi-view images, the target multi-view image can be preprocessed to obtain the target data to be predicted, and the target data to be predicted can be input into the trained online estimation map network to obtain the fused map uncertainty vehicle trajectory data, and the fused map uncertainty vehicle trajectory data can be input into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

[0100] The uncertainty quantification method of autonomous driving map proposed in this application uses a method based on Gamma distribution to quantify the uncertainty of online map estimation in autonomous driving. This method mainly processes the image data captured by the on-board camera to construct a deep learning model that can simultaneously output the map estimation and its uncertainty. Based on the MapTR network architecture, the Gamma distribution is introduced to model the position uncertainty of each map mapping element vertex, and the training is performed using negative log-likelihood loss. The model first uses a convolutional neural network to extract features from multi-view images and converts them into bird's-eye view features. Then the position and uncertainty of the vertex of the map mapping element, as well as the category information of the vertex, are predicted by the regression head and the classification head respectively. Among them, the distribution parameters output by the regression head can effectively quantify the uncertainty of the position prediction, while the classification head directly outputs the confidence score of the category to represent the classification uncertainty. This map information with uncertainty estimation can be integrated into the downstream trajectory prediction task. By encoding the uncertainty information of the map mapping element into trajectory prediction models such as HiVT and DenseTNT, the accuracy of the prediction can be improved. Experimental results show that this method can not only effectively quantify the uncertainty of map estimation, but also significantly improve the performance of trajectory prediction, providing an important guarantee for the safety and credibility of the autonomous driving system.

[0101] Based on the same inventive concept, the embodiment of the present application also provides an autonomous driving map uncertainty quantification device for implementing the autonomous driving map uncertainty quantification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more autonomous driving map uncertainty quantification device embodiments provided below can refer to the limitations of the autonomous driving map uncertainty quantification method above, and will not be repeated here.

[0102] In an exemplary embodiment, Figure 7 As shown, an autonomous driving map uncertainty quantification device is provided, which includes the following modules.

[0103] The target multi-view image acquisition module T1 is used to: acquire a target multi-view image of a target scene where the autonomous driving vehicle is traveling; the target multi-view image includes surround view images of different view angles; the surround view image is captured by a camera on the autonomous driving vehicle; The preprocessing module T2 is used to preprocess the target multi-view image to obtain target data to be predicted; The map online estimation module T3 is used to: input the target data to be predicted into the trained online estimation map network to obtain the fused map uncertainty vehicle trajectory data; the online estimation map network includes a feature extractor, a detection head, a map construction module and a post-processing module; wherein the feature extractor is used to extract features from the target multi-view image to obtain the bird's-eye view features; the detection head includes a regression head and a classification head; the regression head is used to: generate the position of each map mapping element vertex in each surround view image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view features based on the gamma distribution uncertainty probability model; the classification head is used to: generate the confidence score of the category of each map mapping element vertex according to the bird's-eye view features; each map mapping element corresponds to a number of vertices; the map construction module is used to: generate the predicted estimation map of the target scene according to the position of all map mapping element vertices in all surround view images, the confidence score of the category, the regression uncertainty parameters corresponding to all map mapping elements and the target multi-view image; the post-processing module is used to fuse the predicted estimation map with the historical trajectory data of the autonomous driving vehicle to obtain the fused map uncertainty vehicle trajectory data; The future trajectory prediction module T4 is used to: input the fused map uncertainty vehicle trajectory data into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

[0104] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store trajectory prediction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for quantifying uncertainty in an autonomous driving map is implemented.

[0105] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0106] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0107] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0110] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0111] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for quantifying uncertainty in an autonomous driving map, characterized in that: The autonomous driving map uncertainty quantification method comprises: Acquire a target multi-perspective image of a target scene where the autonomous driving vehicle is traveling; the target multi-perspective image includes surround view images at different perspectives; the surround view image is captured by a camera on the autonomous driving vehicle; Preprocessing the target multi-view images to obtain target data to be predicted; Inputting the target data to be predicted into a trained online estimated map network to obtain vehicle trajectory data fused with map uncertainty; the online estimated map network includes a feature extractor, a detection head, a map construction module and a post-processing module; The feature extractor is used to extract features from the target multi-view image to obtain bird's-eye view features; The detection head includes a regression head and a classification head; the regression head is used to: generate the position of each map mapping element vertex in each surround image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view feature based on the gamma distribution uncertainty probability model; the classification head is used to: generate the confidence score of the category of each map mapping element vertex according to the bird's-eye view feature; each map mapping element corresponds to a number of vertices; The map construction module is used to generate a predicted estimated map of the target scene according to the positions of all map mapping element vertices in all surround images, the confidence scores of the categories, the regression uncertainty parameters corresponding to all map mapping elements, and the target multi-view images; The post-processing module is used to fuse the predicted estimated map with the historical trajectory data of the autonomous driving vehicle to obtain fused map uncertainty vehicle trajectory data; The fused map uncertainty vehicle trajectory data is input into a trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

2. The method for quantifying uncertainty of an autonomous driving map according to claim 1, characterized in that: The gamma distribution uncertainty probability model is expressed as follows: ； in, Mapping elements for the map The joint probability density of Is the map element The vertex The shape parameter of the dimensional gamma distribution, Is the mapping element The vertex The scale parameter of the dimensional gamma distribution, is the gamma function; =1,2,3... ; Mapping elements for the map The total number of corresponding vertices.

3. The method for quantifying uncertainty of an autonomous driving map according to claim 1, characterized in that: The categories of the vertices of the map mapping elements include road boundary vertices, pedestrian crossing vertices and lane dividing line vertices.

4. The method for quantifying uncertainty of an autonomous driving map according to claim 1, characterized in that: The classification head includes a plurality of classification submodules connected in sequence and an output linear layer, each of the classification submodules includes an input layer, a layer normalization layer and a ReLU activation function connected in sequence; the input layer is a linear layer.

5. The method for quantifying uncertainty of an autonomous driving map according to claim 1, characterized in that: Before the step of inputting the target data to be predicted into the trained online estimation map network, the autonomous driving map uncertainty quantification method further includes: Acquire a sample data set; the sample data set includes a number of sample multi-view images, positions and categories of points in the sample multi-view images, and true trajectories of the samples; The sample data set is used to train the online estimated map network to obtain a trained online estimated map network.

6. The method for quantifying uncertainty of an autonomous driving map according to claim 5, characterized in that: The sample data set is used to train the online estimated map network to obtain a trained online estimated map network, which specifically includes: The negative log-likelihood loss function is used to train the online estimated map network using the sample data set to obtain a trained online estimated map network.

7. The method for quantifying uncertainty of an autonomous driving map according to claim 1, characterized in that: The trajectory prediction model is a HiVT model or a DenseTNT model.

8. An autonomous driving map uncertainty quantification device, characterized in that: The autonomous driving map uncertainty quantification device comprises: A target multi-view image acquisition module is used to: acquire a target multi-view image of a target scene where the autonomous driving vehicle is traveling; the target multi-view image includes surround view images of different view angles; the surround view image is captured by a camera on the autonomous driving vehicle; A preprocessing module is used to: preprocess the target multi-view image to obtain target data to be predicted; The map online estimation module is used to input the target data to be predicted into the trained online estimation map network to obtain the fused map uncertainty vehicle trajectory data; the online estimation map network includes a feature extractor, a detection head, a map construction module and a post-processing module; wherein the feature extractor is used to extract features from the target multi-view image to obtain the bird's-eye view features; the detection head includes a regression head and a classification head; the regression head is used to generate the position of each map mapping element vertex in each surround view image and the regression uncertainty parameter corresponding to each map mapping element according to the bird's-eye view features based on the gamma distribution uncertainty probability model; the classification head is used to generate the confidence score of the category of each map mapping element vertex according to the bird's-eye view features; each map mapping element corresponds to a number of vertices; the map construction module is used to generate the predicted estimation map of the target scene according to the position of all map mapping element vertices in all surround view images, the confidence score of the category, the regression uncertainty parameters corresponding to all map mapping elements and the target multi-view image; the post-processing module is used to fuse the predicted estimation map with the historical trajectory data of the autonomous driving vehicle to obtain the fused map uncertainty vehicle trajectory data; The future trajectory prediction module is used to: input the fused map uncertainty vehicle trajectory data into the trained trajectory prediction model to predict the future driving trajectory of the autonomous driving vehicle.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the autonomous driving map uncertainty quantification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for quantifying uncertainty of an autonomous driving map described in any one of claims 1 to 7 is implemented.

Images