An automatic driving car perception uncertainty quantification method and device

Abstract

The application relates to an automatic driving car perception uncertainty quantification method and device; the method comprises the following steps: collecting environment information in the vehicle driving process; labeling the collected data to form a data set with labels; the data set is divided into a training set, a test set and a verification set; a variance inference neural network is constructed; each variance inference neural network is pre-trained by using unordered data sets; each variance inference neural network is trained; all the variance inference neural networks are integrated to construct an integrated network; the uncertainty index of the perception result of each variance inference neural network is calculated; an uncertainty-based result fusion strategy is constructed; the uncertainty index of the perception result of the integrated network is calculated; and the final perception result of the model is output; the uncertainty quantification index and the uncertainty index based on the Gaussian error and the model voting mechanism provided by the application provide a unified numerical quantification method for the uncertainty of the perception model output result.

Description

Technical Field

[0001] This invention relates to the field of intelligent connected vehicle technology, and specifically to a method and apparatus for quantifying the perception uncertainty of autonomous vehicles. Background Technology

[0002] In recent years, autonomous driving technology has made significant progress, transforming traditional cars into intelligent vehicles that integrate perception, decision-making, and control. The environmental perception module of an intelligent vehicle is the channel through which the vehicle senses and understands its external environment, enabling it to respond appropriately to complex and dynamic surroundings. Deep learning-based perception algorithms have become the mainstream, continuously extracting knowledge and optimizing parameters from massive amounts of data, achieving excellent performance in many tasks such as object detection, semantic segmentation, and depth detection. However, facing the complex and highly dynamic real world, datasets cannot represent all real-world scenarios, meaning models cannot learn how to handle all scenarios from the dataset. Due to the "black box" nature of deep learning algorithms, their interpretability is poor, often producing unpredictable outputs in certain extreme scenarios, posing a significant challenge to the robustness of vehicle behavior decisions and trajectory planning, and seriously affecting the safety of autonomous vehicles.

[0003] Therefore, while outputting perception results, the environmental perception model should quantify the uncertainty of its output results. For obstacles whose existence is uncertain or whose detection accuracy is uncertain, a higher level of uncertainty should be assigned so that downstream modules can react in a timely manner and even remind the driver to take over the vehicle.

[0004] The quantification of uncertainty in output results is of great significance to the driving capabilities of autonomous vehicles. However, most existing target detection methods cannot guarantee accuracy while effectively quantifying the uncertainty of the output results. Furthermore, there is a lack of a unified metric for measuring the uncertainty of obstacle perception. To address these issues, we propose a method and metric for quantifying the uncertainty of perception in autonomous vehicles, which has strong theoretical and technical feasibility. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the above-mentioned problems existing in the prior art and to provide a method and device for quantifying the perception uncertainty of autonomous vehicles.

[0006] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0007] This invention proposes a method for quantifying the perception uncertainty of autonomous vehicles. Using point cloud data as input, it performs environmental perception and uncertainty quantification. This invention utilizes a direct modeling approach to construct multiple variance inference neural networks and an uncertainty-based fusion strategy, enabling effective feature extraction, environmental perception, and variance estimation from point cloud information. Furthermore, this invention proposes a unified uncertainty quantification index—the Uncertainty Index (UI)—to effectively quantify the uncertainty of the output results, providing comprehensive perception information for the decision-making and planning modules of intelligent vehicles and improving the driving capabilities of autonomous vehicles.

[0008] A method for quantifying perception uncertainty in autonomous vehicles includes the following steps:

[0009] Step 1: The vehicle uses lidar sensors to collect environmental information during its driving process;

[0010] Step 2: Label the data collected in Step 1 to form a labeled dataset;

[0011] Step 3: Divide the dataset from Step 2 into a training set, a test set, and a validation set;

[0012] Step 4: Based on the deep integration paradigm, build Variance inference neural networks with the same structure but different parameters;

[0013] Step 5: Pre-train each variance inference neural network using an unordered dataset;

[0014] Step 6: Train each variance inference neural network using an unordered dataset;

[0015] Step 7: Integrate all variance inference neural networks to construct an ensemble network;

[0016] Step 8: Calculate the uncertainty index of the neural network perception result for each variance inference;

[0017] Step 9: Construct an outcome fusion strategy based on uncertainty;

[0018] Step 10: Calculate the uncertainty index of the integrated network sensing results;

[0019] Step 11: Output the final perception result of the model.

[0020] Furthermore, in steps one, two, and three, the vehicle uses lidar sensors to collect and label environmental information during its driving process, as detailed below:

[0021] Intelligent vehicles are equipped with LiDAR sensors, which enable them to perceive their surroundings and obtain point cloud data information, denoted as... Each point cloud All contain There are three obstacles, denoted as:

[0022]

[0023] The dataset includes information on the type of each obstacle. For example, pedestrians, bicycles, and cars, 3D envelope information ;in, It is an obstacle Location information, It is an obstacle The properties and orientation of the 3-D envelope.

[0024] Defined in each set of point cloud data In Chinese, the tag is defined as:

[0025]

[0026] The above dataset The dataset is divided into training, validation, and test sets according to a certain ratio (e.g., 8:1:1).

[0027] Furthermore, in step four, based on the deep ensemble paradigm, multiple variance inference neural networks with identical structures but different parameters are constructed. The specific details are as follows:

[0028] Build A variance inference neural network model is used, with each sub-network containing a point cloud encoder, a class decoder, an envelope decoder, and a variance decoder. Each sub-network has the same structure but different initialization parameters. Point cloud data is used. After inputting each neural network model, the data flow is represented as follows:

[0029]

[0030] in, In the parameters The following encoding process, It is high-level semantic information about the environment. It is the obstacle category information detected by the model. These are the obstacle envelope attributes predicted by the model. It is the covariance matrix of the obstacles predicted by the model. In the parameters The following category decoding process, In the parameters The following is the process of decoding the envelope frame. In the parameters The variance decoding process and variance inference process output include: Layers are used to ensure that the output result is positive;

[0031] Because existing datasets lack label information for uncertainty, traditional optimization algorithms cannot be directly used to train the variance decoder. Therefore, we directly model the obstacle information in the environment, assuming that the bounding box attributes of obstacles follow a multivariate Gaussian distribution. Each sub-network outputs the distribution parameters of obstacle information, using the mean of the multivariate Gaussian distribution as the attribute information of the bounding box, and the variance information as the covariance matrix of the distribution, which can be expressed as:

[0032]

[0033] in, This represents the probability distribution of detected obstacle attributes. and These represent the ground truth value of the obstacle envelope and the attributes of the model-detected envelope, respectively. Indicates the first The network parameters of each subnetwork are a set of learnable variables. This indicates taking the determinant of a matrix. This represents the transpose operation. Assuming each attribute of the envelope is independent, the covariance matrix... It is a diagonal matrix.

[0034] Clearly, the label information of the dataset can be viewed as a multivariate Gaussian distribution with a covariance matrix of all zeros. Treating this as a Dirac function, it can be expressed as:

[0035]

[0036] in, Represents the distribution of the true values. This represents the Dirac function.

[0037] During the training phase, network parameters are optimized. This minimizes the difference between the model's predicted distribution and the true distribution, enabling each subnetwork to have environmental awareness and variance inference capabilities, and can be expressed as:

[0038]

[0039] in, This represents the Kullback-Leibler (KL) divergence between two distributions.

[0040] Using KL divergence as a metric to measure the degree of difference between two distributions, and taking this degree of difference as the learning optimization objective of the neural network, a loss function is constructed. Optimize the regression and variance inference tasks for each variance inference neural network. The loss function can be expressed as:

[0041]

[0042] in, Represents the KL loss function. This indicates the number of obstacles detected by the model.

[0043] Furthermore, in step five, each variance inference neural network is pre-trained using an unordered dataset, as detailed below:

[0044] For each subnetwork, a two-stage training strategy is designed.

[0045] First, in the pre-training phase, the network's encoder, class decoder, and envelope decoder are trained using the entire dataset, with the class loss and regression loss as follows:

[0046]

[0047] in, It is the class loss function of the model. It is the regression loss function of the model. It is the output of the category decoder. These are empirical parameters in the loss function. This indicates the degree of difference between the predicted and ground truth bounding boxes. Indicates the confidence level of the output anchor box.

[0048] In summary, the loss function during the pre-training phase can be expressed as:

[0049]

[0050] in, It is the total loss of the model. , and It is the balance coefficient for the three loss categories. It is Classification loss.

[0051] After training each neural network for multiple rounds, the network's encoder and decoder gradually acquire the ability to extract and predict point cloud information features, achieving good perception results.

[0052] Furthermore, in step six, the unordered dataset is used to train each variance inference neural network, as detailed below:

[0053] During the network training phase, for the two regression tasks of the envelope decoder and variance inference decoder, the following methods were used: Retraining is then performed. In this stage, the loss function can be expressed as:

[0054]

[0055] in, , and It is the balance parameter for the three loss categories during the training phase.

[0056] Furthermore, in step seven, all variance inference neural networks are integrated to construct an ensemble network, the details of which are as follows:

[0057] In steps four through six, the present invention has constructed a total of Variance inference neural networks with different parameters but the same structure, when dealing with point cloud data... The above is input into the ensemble model. A variance inference neural network performs parallel inference on the input data. Each model outputs its perception results. Each of them Includes obstacle category information Envelope attribute information Covariance Matrix , It is a sub-network Number of obstacles detected.

[0058] Furthermore, in step eight, the uncertainty index of each variance inference neural network output is calculated, as detailed below:

[0059] Clearly, different types of obstacles exhibit varying sensitivities to different variances. Relying solely on the variance of obstacle attributes estimated by the model cannot accurately and reasonably reflect the uncertainty of the detection results. Furthermore, variance cannot model false positives. Therefore, based on the Gaussian error function and model voting mechanism, a novel uncertainty quantification index—the Uncertainty Index (UI)—is proposed. Specifically, the Gaussian error function is used to capture the dispersion of different obstacle distributions. To calculate the uncertainty of obstacles, the Gaussian error values ​​of different attributes are accumulated and a scaling factor is applied to group the indices of different types into the same range for comparison and adjustment. It can be expressed as:

[0060]

[0061] in, The first element of the covariance matrix represents the first element of the covariance matrix. OK List the elements, Indicates the number of properties of the envelope. This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher its confidence level. It is a proportionality coefficient, a variable related to the type of obstacle.

[0062] Step eight calculates the detection results of all models. .

[0063] Furthermore, step nine involves constructing an outcome fusion strategy based on uncertainty, the details of which are as follows:

[0064] Object detection tasks include classification and regression tasks. The fusion method for classification tasks is relatively simple, and there is already much research on this. However, regression tasks have higher complexity due to the introduction of multi-dimensional attributes and the disordered nature of neural network model outputs. In step seven, different sub-networks, while performing environmental perception, can also quantify the uncertainty of their outputs, providing a good basis for result fusion. Therefore, an uncertainty-based fusion strategy is proposed to fuse and filter multiple results, outputting a complete and accurate result.

[0065] First, iterate through the reasoning results input in step seven. For each category, it is divided into overlapping and non-overlapping results based on its intersection-overall ratio (IoU). When the IoU of two perceived results is greater than a threshold, they are considered overlapping detection results; otherwise, they are considered non-overlapping results.

[0066] For overlapping results, following the idea of ​​deep integration, the attributes of their envelopes are averaged and weighted to reduce the impact of random uncertainty, which can be expressed as:

[0067]

[0068] The variance of overlapping obstacles can be expressed as:

[0069]

[0070] in, The number of results indicating overlapping regions. and This represents the predicted envelope attributes and covariance matrix of different networks for the same obstacle. It is the final envelope perception result of the ensemble model. This represents the covariance matrix of the model's final predictions.

[0071] For non-overlapping results, an uncertainty index threshold is set for filtering. When the perceived result... If the value is less than the threshold, the perception result is output as the final result; if the perception result is less than the threshold, the perception result is output as the final result. If the value exceeds this threshold, it is considered a false detection target.

[0072] Finally, the detection results are subjected to non-maximum suppression, and the perception results are further extracted as the final output of the perception model.

[0073] Furthermore, step ten calculates the uncertainty index of the integrated network sensing results, as detailed below:

[0074] Step eight introduced a method for calculating the uncertainty index of a single neural network output. Further, for an ensemble network, if an object in the environment is detected by more networks, it indicates a higher probability of the object's existence, and the object should have a lower uncertainty index. Conversely, if an object is detected by only a single network and not perceived by other networks, it suggests that the result is likely a false detection by the model, and the object should have a higher uncertainty index. Based on the above analysis, considering the number of detected models as a coefficient and adding it to the uncertainty calculation process to improve the judgment criteria in the uncertainty index calculation process, it can be expressed as:

[0075]

[0076] in, This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher its confidence level. It is a vector that reflects the gain of the number of detected models on the uncertainty index.

[0077] Based on this, the uncertainty index of the final perceived result in step nine is calculated. ,Will Together with other basic attributes of the perception results, they are input into downstream modules of the intelligent vehicle to provide more comprehensive information guidance for the vehicle's decision-making and planning.

[0078] An apparatus comprising one or more processors;

[0079] Memory, used to store one or more programs;

[0080] When the one or more programs are executed by the one or more processors, the one or more processors implement the autonomous vehicle perception uncertainty quantification method as described above.

[0081] Compared with the prior art, the beneficial effects of the present invention are:

[0082] This invention proposes a method for quantifying the perception uncertainty of autonomous vehicles. The method first uses a multivariate Gaussian distribution to abstractly model obstacles in the driving environment, derives the neural network training loss function, and uses a two-stage training strategy to construct multiple variance inference networks and integrate them using a deep ensemble paradigm.

[0083] This invention proposes an uncertainty index, based on Gaussian error and model voting mechanism, to quantify uncertainty. It provides a unified numerical quantification method for perceiving the uncertainty of model output results.

[0084] This invention designs an uncertainty-based fusion strategy that, targeting the characteristics of 3-D target detection tasks, fuses and filters the results of multivariate inference neural networks, reduces the number of false targets detected by the model, and provides more complete perception information for downstream decision planning modules. Attached Figure Description

[0085] The invention will now be further described with reference to the accompanying drawings:

[0086] Figure 1 This is a flowchart of a method for quantifying the perception uncertainty of an autonomous vehicle according to the present invention. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this invention. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The embodiments of this invention will be described in detail below with reference to the accompanying drawings.

[0088] In the description of this invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the scope of protection of this invention.

[0089] The present invention will now be described in detail with reference to the accompanying drawings:

[0090] like Figure 1 As shown, this invention proposes a method for quantifying the perception uncertainty of autonomous vehicles. First, we construct a dataset using LiDAR sensors to collect and label environmental information. Second, we design a two-stage training strategy to build multiple variance inference neural networks with identical structures but different parameters, enabling effective feature extraction, environmental perception, and variance estimation of point cloud information. In the inference stage, we use the above networks for parallel inference and construct an uncertainty-based fusion strategy to filter and fuse the results. Furthermore, this invention proposes a unified uncertainty quantification index—the Uncertainty Index (UI)—to effectively quantify the uncertainty of the output results. In summary, this invention can quantify the perception uncertainty of autonomous driving, providing complete perception information for the decision-making and planning modules of intelligent vehicles and improving the driving capabilities of autonomous vehicles.

[0091] Specifically, it includes:

[0092] Step 1: The vehicle uses lidar sensors to collect environmental information during its driving process;

[0093] Step 2: Label the data collected in Step 1 to form a labeled dataset;

[0094] Step 3: Divide the dataset from Step 2 into training, testing, and validation sets:

[0095] In steps one, two, and three, the intelligent vehicle is equipped with a LiDAR sensor, which can perceive the surrounding environment and obtain point cloud data information, denoted as... Each point cloud All contain There are three obstacles, denoted as:

[0096]

[0097] The dataset includes information on the type of each obstacle. For example, pedestrians, bicycles, and cars, 3D envelope information ;in, It is an obstacle Location information, It is an obstacle The properties and orientation of the 3-D envelope.

[0098] Defined in each set of point cloud data In Chinese, the tag is defined as:

[0099]

[0100] The above dataset The dataset is divided into training, validation, and test sets according to a certain ratio (e.g., 8:1:1).

[0101] Step 4: Based on the deep ensemble paradigm, construct multiple variance inference neural networks with identical structures but different parameters:

[0102] Build A variance inference neural network model is used, with each sub-network containing a point cloud encoder, a class decoder, an envelope decoder, and a variance decoder. Each sub-network has the same structure but different initialization parameters. Point cloud data is used. After inputting each neural network model, the data flow is represented as follows:

[0103]

[0104] in, In the parameters The following encoding process, It is high-level semantic information about the environment. It is the obstacle category information detected by the model. These are the obstacle envelope attributes predicted by the model. It is the covariance matrix of the obstacles predicted by the model. In the parameters The following category decoding process, In the parameters The following is the process of decoding the envelope frame. In the parameters The variance decoding process and variance inference process output include: Layers are used to ensure that the output result is positive;

[0105] Because existing datasets lack label information for uncertainty, traditional optimization algorithms cannot be directly used to train the variance decoder. Therefore, we directly model the obstacle information in the environment, assuming that the bounding box attributes of obstacles follow a multivariate Gaussian distribution. Each sub-network outputs the distribution parameters of obstacle information, using the mean of the multivariate Gaussian distribution as the attribute information of the bounding box, and the variance information as the covariance matrix of the distribution, which can be expressed as:

[0106]

[0107] in, This represents the probability distribution of detected obstacle attributes. and These represent the ground truth value of the obstacle envelope and the attributes of the model-detected envelope, respectively. Indicates the first The network parameters of each subnetwork are a set of learnable variables. This indicates taking the determinant of a matrix. This represents the transpose operation. Assuming each attribute of the envelope is independent, the covariance matrix... It is a diagonal matrix.

[0108] Clearly, the label information of the dataset can be viewed as a multivariate Gaussian distribution with a covariance matrix of all zeros. Treating this as a Dirac function, it can be expressed as:

[0109]

[0110] in, Represents the distribution of the true values. This represents the Dirac function.

[0111] During the training phase, network parameters are optimized. This minimizes the difference between the model's predicted distribution and the true distribution, enabling each subnetwork to have environmental awareness and variance inference capabilities, and can be expressed as:

[0112]

[0113] in, This represents the Kullback-Leibler (KL) divergence between two distributions.

[0114] Using KL divergence as a metric to measure the degree of difference between two distributions, and taking this degree of difference as the learning optimization objective of the neural network, a loss function is constructed. Optimize the regression and variance inference tasks for each variance inference neural network. The loss function can be expressed as:

[0115]

[0116] in, Represents the KL loss function. This indicates the number of obstacles detected by the model.

[0117] Step 5: Pre-train each variance inference neural network using an unordered dataset:

[0118] For each subnetwork, a two-stage training strategy is designed.

[0119] First, in the pre-training phase, the network's encoder, class decoder, and envelope decoder are trained using the entire dataset, with the class loss and regression loss as follows:

[0120]

[0121] in, It is the class loss function of the model. It is the regression loss function of the model. It is the output of the category decoder. These are empirical parameters in the loss function. This indicates the degree of difference between the predicted and ground truth bounding boxes. Indicates the confidence level of the output anchor box.

[0122] In summary, the loss function during the pre-training phase can be expressed as:

[0123]

[0124] in, It is the total loss of the model. , and It is the balance coefficient for the three loss categories. It is Classification loss.

[0125] After training each neural network for multiple rounds, the network's encoder and decoder gradually acquire the ability to extract and predict point cloud information features, achieving good perception results.

[0126] Step Six: Train each variance inference neural network using an unordered dataset:

[0127] During the network training phase, for the two regression tasks of the envelope decoder and variance inference decoder, the following methods were used: Retraining is then performed. In this stage, the loss function can be expressed as:

[0128]

[0129] in, , and It is the balance parameter for the three loss categories during the training phase.

[0130] Step 7: Integrate all variance inference neural networks to construct an ensemble network:

[0131] In steps four through six, the present invention has constructed a total of Variance inference neural networks with different parameters but the same structure, when dealing with point cloud data... The above is input into the ensemble model. A variance inference neural network performs parallel inference on the input data. Each model outputs its perception results. Each of them Includes obstacle category information Envelope attribute information Covariance Matrix , It is a sub-network Number of obstacles detected.

[0132] Step 8: Calculate the uncertainty index of each variance inference neural network output:

[0133] Clearly, different types of obstacles exhibit varying sensitivities to different variances. Relying solely on the variance of obstacle attributes estimated by the model cannot accurately and reasonably reflect the uncertainty of the detection results. Furthermore, variance cannot model false positives. Therefore, based on the Gaussian error function and model voting mechanism, a novel uncertainty quantification index—the Uncertainty Index (UI)—is proposed. Specifically, the Gaussian error function is used to capture the dispersion of different obstacle distributions. To calculate the uncertainty of obstacles, the Gaussian error values ​​of different attributes are accumulated and a scaling factor is applied to group the indices of different types into the same range for comparison and adjustment. It can be expressed as:

[0134]

[0135] in, The first element of the covariance matrix represents the first element of the covariance matrix. OK List the elements, Indicates the number of properties of the envelope. This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher its confidence level. It is a proportionality coefficient, a variable related to the type of obstacle.

[0136] Step eight calculates the detection results of all models. .

[0137] Step Nine: Construct an outcome fusion strategy based on uncertainty:

[0138] Object detection tasks include classification and regression tasks. The fusion method for classification tasks is relatively simple, and there is already much research on this. However, regression tasks have higher complexity due to the introduction of multi-dimensional attributes and the disordered nature of neural network model outputs. In step seven, different sub-networks, while performing environmental perception, can also quantify the uncertainty of their outputs, providing a good basis for result fusion. Therefore, an uncertainty-based fusion strategy is proposed to fuse and filter multiple results, outputting a complete and accurate result.

[0139] First, iterate through the reasoning results input in step seven. For each category, it is divided into overlapping and non-overlapping results based on its intersection-overall ratio (IoU). When the IoU of two perceived results is greater than a threshold, they are considered overlapping detection results; otherwise, they are considered non-overlapping results.

[0140] For overlapping results, following the idea of ​​deep integration, the attributes of their envelopes are averaged and weighted to reduce the impact of random uncertainty, which can be expressed as:

[0141]

[0142] The variance of overlapping obstacles can be expressed as:

[0143]

[0144] in, The number of results indicating overlapping regions. and This represents the predicted envelope attributes and covariance matrix of different networks for the same obstacle. It is the final envelope perception result of the ensemble model. This represents the covariance matrix of the model's final predictions.

[0145] For non-overlapping results, an uncertainty index threshold is set for filtering; when the result... If the value is less than the threshold, the perception result is output as the final result; if the perception result is less than the threshold, the perception result is output as the final result. If the value exceeds this threshold, it is considered a false detection target.

[0146] Finally, the detection results are subjected to non-maximum suppression, and the perception results are further extracted as the final output of the perception model.

[0147] Step 10: Calculate the uncertainty index of the ensemble network sensing results:

[0148] Step eight introduced a method for calculating the uncertainty index of a single neural network output. Further, for an ensemble network, if an object in the environment is detected by more networks, it indicates a higher probability of the object's existence, and the object should have a lower uncertainty index. Conversely, if an object is detected by only a single network and not perceived by other networks, it suggests that the result is likely a false detection by the model, and the object should have a higher uncertainty index. Based on the above analysis, considering the number of detected models as a coefficient and adding it to the uncertainty calculation process to improve the judgment criteria in the uncertainty index calculation process, it can be expressed as:

[0149]

[0150] in, This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher its confidence level. It is a vector that reflects the gain of the number of detected models on the uncertainty index.

[0151] Based on this, the uncertainty index of the final perceived result in step nine is calculated. ,

[0152] Step 11: Output the final perception result of the model:

[0153] The calculation obtained in step ten Together with other basic attributes of the perception results, they are input into downstream modules of the intelligent vehicle to provide more comprehensive information guidance for the vehicle's decision-making and planning.

[0154] The present invention provides another embodiment of an apparatus comprising one or more processors;

[0155] Memory, used to store one or more programs;

[0156] When the one or more programs are executed by the one or more processors, the one or more processors implement the autonomous vehicle perception uncertainty quantification method as described above.

[0157] Memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions corresponding to the autonomous vehicle perception uncertainty quantification method in this embodiment of the invention. The processor executes the software programs, instructions, and modules stored in the memory to perform various vehicle functions and data processing, thereby implementing the aforementioned autonomous vehicle perception uncertainty quantification method.

[0158] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the terminal. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0159] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be included within the scope of protection of the present invention. Furthermore, all content not described in detail in this specification is prior art known to those skilled in the art.

Claims

1. A method for quantifying perception uncertainty in autonomous vehicles, characterized in that, include: Based on the deep integration paradigm, build Variance inference neural networks with the same structure but different parameters; The variance inference neural network for each variance is pre-trained using an unordered dataset; The variance inference neural network is trained using an unordered dataset; Integrate all variance inference neural networks to construct an ensemble network; Calculate the uncertainty index of the neural network perception result for each variance; Constructing an outcome fusion strategy based on uncertainty; Calculate the uncertainty index of the integrated network sensing results; The final perception result of the output model; In construction Before creating variance inference neural networks with identical structures but different parameters, the following steps are required: Step 1: The vehicle uses lidar sensors to collect environmental information during its driving process; Step 2: Label the data collected in Step 1 to form a labeled dataset; Step 3: Divide the dataset from Step 2 into a training set, a validation set, and a test set; In steps one, two, and three, the vehicle uses lidar sensors to collect and label environmental information during its driving process. The specific details are as follows: Intelligent vehicles are equipped with LiDAR sensors to perceive their surroundings and obtain point cloud data, denoted as... Each point cloud All contain There are three obstacles, denoted as: Annotate the ground truth information of obstacles in the point cloud data, including each obstacle. Types of information and 3-D envelope information ;in, It is an obstacle Location information, It is an obstacle The properties and orientation of the 3D envelope; Defined in each set of point cloud data In Chinese, the tag is defined as: The above dataset It is divided into training set, validation set and test set. It is the number of samples in the dataset; The calculation of the uncertainty index of each variance inference neural network perception result is as follows: The Gaussian error function is used to capture the dispersion of different obstacle distributions. To calculate the uncertainty of the obstacles, the Gaussian error values ​​of different attributes are accumulated and a scaling factor is applied to bring different types of indices into the same range for comparison and adjustment. This is expressed as: in, The first element of the covariance matrix represents the first element of the covariance matrix. OK List the elements, Indicates the number of properties of the envelope. This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher the confidence level. It is a proportionality coefficient, a variable related to the type of obstacle.

2. The method for quantifying perception uncertainty in autonomous vehicles according to claim 1, characterized in that, The construction based on the deep integration paradigm Several variance inference neural networks with identical structures but different parameters; details are as follows: A neural network model is constructed, with each sub-network containing a point cloud encoder, a category decoder, an envelope decoder, and a variance decoder. Each sub-network has the same structure but different initialization parameters. Point cloud data is also included. After inputting each neural network model, the data flow is represented as follows: in, In the parameters The following encoding process, It is high-level semantic information about the environment. It is the obstacle category information detected by the model. These are the obstacle envelope attributes predicted by the model. It is the covariance matrix of the obstacles predicted by the model. In the parameters The following category decoding process, In the parameters The following is the process of decoding the envelope. In the parameters The variance decoding process and variance inference process output include: Layers are used to ensure that the output result is positive; We directly model obstacle information in the environment, assuming that the bounding box attributes of obstacles follow a multivariate Gaussian distribution. Each sub-network outputs the distribution parameters of obstacle information. The mean of the multivariate Gaussian distribution is used as the attribute information of the bounding box, and the variance information constitutes the covariance matrix of the distribution, expressed as: in, This represents the probability distribution of detected obstacle attributes. and These represent the ground truth value of the obstacle envelope and the attributes of the model-detected envelope, respectively. Indicates the first The network parameters of each subnetwork are a set of learnable variables; This indicates taking the determinant of a matrix; This represents the transpose operation; it assumes that each attribute of the envelope is independent, so the covariance matrix... It is a diagonal matrix; The label information of the dataset is considered as a multivariate Gaussian distribution with a covariance matrix of all zeros. This distribution is treated as a Dirac function and expressed as: in, Represents the distribution of the true values. Represents the Dirac function; During the training phase, network parameters are optimized. This minimizes the difference between the model's predicted distribution and the true distribution, enabling each subnetwork to have environmental awareness and variance inference capabilities, and is expressed as: in, This represents the KL divergence between two distributions; Using KL divergence as a metric to measure the degree of difference between two distributions, and employing KL divergence as the learning optimization objective of the neural network, a loss function is constructed. Optimize the regression and variance inference tasks for each variance inference neural network; the loss function is expressed as: in, Represents the KL loss function. This indicates the number of obstacles detected by the model.

3. A method for quantifying perception uncertainty in autonomous vehicles according to claim 1 or 2, characterized in that, The pre-training of each variance inference neural network using an unordered dataset is detailed below: During the pre-training phase, the network's encoder, class decoder, and envelope decoder are trained using the entire dataset. The class loss and regression loss are as follows: in, It is the class loss function of the model. It is the regression loss function of the model. It is the output of the category decoder. These are empirical parameters in the loss function. This indicates the degree of difference between the predicted and ground truth bounding boxes. Indicates the confidence level of the output anchor box; The loss function during the pre-training phase is expressed as: in, It is the total loss of the model. , and It is the balance coefficient for the three loss categories. It is Classification loss.

4. The method for quantifying perception uncertainty in autonomous vehicles according to claim 3, characterized in that, The specific details of training each variance inference neural network using an unordered dataset are as follows: During the network training phase, for the two regression tasks of the envelope decoder and variance inference decoder, the following methods were used: Retraining is performed; the loss function is expressed as: in, , and It is the balance parameter for the three loss categories during the training phase.

5. The method for quantifying perception uncertainty in autonomous vehicles according to claim 4, characterized in that, The ensemble network, which integrates all variance inference neural networks, is constructed as follows: Build Variance inference neural networks with different parameters but the same structure, when point cloud data... The above is input into the ensemble model. Each variance inference neural network performs parallel inference on the input data and outputs its perception results. Each of them Includes obstacle category information Envelope attribute information Covariance Matrix , It is a sub-network Number of obstacles detected.

6. The method for quantifying perception uncertainty in autonomous vehicles according to claim 5, characterized in that, The specific details of the uncertainty-based result fusion strategy are as follows: First, iterate through the inference results of the input. For each category, all target detection results are divided into overlapping results and non-overlapping results based on the intersection-union ratio (IoU) between obstacles. When the IoU of two perception results is greater than a threshold, they are considered to be overlapping detection results; otherwise, they are considered to be non-overlapping results. Secondly, for the overlapping results, the envelope attributes are averaged and weighted to reduce the impact of random uncertainty, as shown below: The variance of overlapping obstacles is expressed as: in, The number of results indicating overlapping regions. and This represents the predicted envelope attributes and covariance matrix of different networks for the same obstacle. It is the final envelope perception result of the ensemble model. This represents the covariance matrix of the model's final predictions; For non-overlapping results, an uncertainty index threshold is set for filtering; when the perceived result... If the value is less than the threshold, the perception result is output as the final result; if the perception result is less than the threshold, the perception result is output as the final result. If the value exceeds this threshold, the perceived result is considered a false detection. Finally, the detection results are subjected to non-maximum suppression, and the perception results are further extracted as the final output of the perception model. The uncertainty index of the calculation integration network perception results is detailed below: Consider incorporating the number of models to be tested as coefficients into the uncertainty calculation process to improve the judgment criteria in the uncertainty index calculation process, expressed as: in, This represents the uncertainty index of a variance inference neural network, ranging from... between, A larger value indicates greater uncertainty and lower confidence in the model's perception results; conversely, a smaller value indicates greater uncertainty and lower confidence in the model's perception results. The smaller the value, the less uncertainty the model has about the perceived results, and the higher the confidence level. It is a vector that reflects the gain of the number of detected models on the uncertainty index.

7. An apparatus, characterized in that: Includes one or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the autonomous vehicle perception uncertainty quantification method as described in claim 6.

Images