A bounding box prediction uncertainty modeling method based on residual learning

Abstract

The application discloses a bounding box prediction uncertainty modeling method based on residual learning, and the method is as follows: 1, constructing a bounding box prediction uncertainty network based on a full connection layer and a residual module; 2, preparing trajectory prediction data; 3, calculating a prediction confidence label; 4, training a prediction uncertainty network; and 5, performing prediction network reasoning. Through the network structure of the fusion of the full connection block and the residual block and the design of the uncertainty label, the application realizes accurate quantification, efficient calculation and stable evaluation of the bounding box prediction uncertainty, can significantly improve the modeling precision and reduce the calculation cost, and can provide reliable prediction credibility evaluation for high safety demand applications in the automatic driving, robot navigation and other scenes, and avoid the decision risk caused by the prediction uncertainty. The network has good generalization adaptation capability, the modular design of the network enables the network to be seamlessly embedded into various existing trajectory prediction networks, and the network provides a flexible and efficient solution for algorithm upgrading.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to a method for modeling uncertainty in bounding box prediction, specifically a method for modeling uncertainty in bounding box prediction based on residual learning. Background Technology

[0002] In the field of computer vision, trajectory prediction models are widely used in scenarios such as autonomous driving and robot navigation. Currently, most trajectory prediction models only output deterministic results, such as directly outputting the bounding box coordinates of the target location. However, such models lack a quantitative evaluation mechanism for prediction reliability and cannot indicate the degree of reliability of the prediction results. In practical applications, if the model's output prediction results have high uncertainty but go undetected, it may lead to serious consequences such as autonomous driving systems misjudging road conditions and missing important targets. Currently, although some research attempts to evaluate the uncertainty of prediction results, existing methods have shortcomings in network structure design and training strategies, making it difficult to quickly and accurately model and quantify the uncertainty of bounding box predictions. For example, Monte Carlo dropout-based methods simulate model uncertainty by randomly dropping neurons in the network multiple times, but they rely on a large amount of forward propagation computation, resulting in low efficiency. Bayesian neural network-based methods introduce prior distributions to model uncertainty, but precise Bayesian inference has extremely high computational complexity, and the use of approximate inference can lead to large deviations in uncertainty estimation. Ensemble learning-based methods acquire uncertainty by training multiple independent models, but they require a large amount of computational resources and storage space, and the differences between models are difficult to control, leading to unstable evaluation results. Summary of the Invention

[0003] To address the shortcomings of existing trajectory prediction models, which only output deterministic results and lack a mechanism for quantifying prediction reliability, as well as the deficiencies in existing uncertainty assessment methods such as low computational efficiency, large inference bias, high resource consumption, and unstable evaluation results in network structure design and training strategies, thus failing to accurately model and quantify bounding box prediction uncertainty, this invention provides a bounding box prediction uncertainty modeling method based on residual learning. This method proposes a bounding box prediction uncertainty network that fuses fully connected blocks and residual blocks, and designs an uncertainty labeling method. It exhibits significant advantages in improving computational efficiency, accurately capturing uncertain features, and enhancing evaluation accuracy. This method can efficiently and accurately model and quantify bounding box prediction uncertainty, helping to understand the reliability of predictions and avoiding serious consequences caused by undetected prediction uncertainty.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A method for modeling uncertainty in bounding box prediction based on residual learning includes the following steps:

[0006] Step 1: Construct a bounding box prediction uncertainty network based on fully connected layers and residual modules:

[0007] The bounding box prediction uncertainty network consists of fully connected layers and residual blocks. The predicted bounding box results are first mapped to a feature space of a specific dimension through the fully connected layers:

[0008]

[0009] in, The feature vector after the bounding box is mapped. It is the weight matrix of the fully connected layer. It is a bias weight. It is the predicted bounding box result. It is the trajectory of history;

[0010] The output of the fully connected layer is used as input and passes through two residual modules for feature extraction and nonlinear transformation. Then, it is connected to another fully connected layer to integrate the features extracted by the residual modules, map them to the output space, and directly output the predicted confidence value. ;

[0011] In the residual module, the input data first passes through a fully connected layer to obtain the output. Next, the ReLU activation function is used to process it, followed by a fully connected layer. Then, a residual connection mechanism is used to add the original input to the result after two fully connected layers. After the residual connection is completed, a ReLU activation function is used to further introduce nonlinear factors to enhance the network's expressive power.

[0012] Step 2: Trajectory Prediction Data Preparation

[0013] Step 2: 1. Obtain the historical trajectory sequence data of the target in consecutive frames, and record in detail the positional change information of each target in the time series to provide the network with a reliable historical motion feature reference;

[0014] Step 22: Based on the existing trajectory prediction network, generate predicted bounding boxes for subsequent frames. These predicted bounding boxes contain the estimated position information of the target in the next frame and serve as another important input to the network. Together with the historical trajectory sequence data, they form a complete input data pair.

[0015] Step 3: Calculate the prediction confidence label:

[0016] Step 31: Calculate the predicted bounding box results. Transform into ,in The coordinates of the left and top boundaries are... Let the coordinates of the right and bottom boundaries be , and the coordinates of the vertices of the actual labeled bounding box be . Calculate the root mean square error between the predicted bounding box vertex coordinates and the actual labeled coordinates. :

[0017]

[0018] Step 3.2: Normalize the root mean square error to the [0,1] interval:

[0019]

[0020] The minimum value of the error is The maximum value is ;

[0021] Step 3: Subtract the error value from 1 to convert it into a prediction confidence index:

[0022]

[0023] Step 4: Predicting Uncertainty - Network Training

[0024] By minimizing the confidence level of prediction uncertainty Confidence label of true uncertainty The differences between them enable the training of the network for bounding box prediction uncertainty:

[0025]

[0026] in: It is a loss function. It is the number of samples in the batch; It is the network to the first Uncertainty prediction for a single sample; It is the first True uncertainty confidence label for each sample;

[0027] Step 5: Predictive Network Inference:

[0028] Predicted bounding box of input target and historical trajectory The trained network outputs confidence scores. .

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] This invention achieves accurate quantification, efficient computation, and stable evaluation of bounding box prediction uncertainty through a network structure that fuses fully connected blocks and residual blocks, along with the design of uncertainty labels. This significantly improves modeling accuracy and reduces computational costs. In scenarios such as autonomous driving and robot navigation, it provides reliable prediction credibility assessments for applications with high safety requirements, effectively avoiding decision-making risks caused by prediction uncertainty. Furthermore, this network possesses excellent generalization and adaptability capabilities; its modular design allows for seamless embedding into various existing trajectory prediction networks, providing a flexible and efficient solution for algorithm upgrades. Attached Figure Description

[0031] Figure 1 A schematic diagram illustrating the basic principle of a bounding box prediction uncertainty modeling method based on residual learning;

[0032] Figure 2 The structure diagram of the network for predicting uncertainty in bounding boxes. Detailed Implementation

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0034] This invention provides a method for modeling uncertainty in bounding box prediction based on residual learning. It learns the mapping relationship between trajectory features and prediction output through a lightweight network architecture composed of fully connected layers and residual blocks. Simultaneously, it designs a label calculation method for the confidence level of predicted bounding box uncertainty to optimize the network training process, achieving efficient and accurate modeling of prediction confidence, thereby significantly improving the reliability of decision-making in scenarios such as autonomous driving. Figure 1 As shown, the specific steps include the following:

[0035] Step 1: Construct a bounding box prediction uncertainty network based on fully connected layers and residual modules:

[0036] like Figure 2 As shown, the bounding box prediction uncertainty network combines fully connected layers with residual blocks. The network takes the generated predicted bounding box results as input. and historical trajectory The predicted bounding box result is first mapped to a feature space of a specific dimension through a fully connected layer:

[0037]

[0038] in, The feature vector after the bounding box is mapped. It is the weight matrix of the fully connected layer. These are bias weights. The output of the fully connected layer is passed as input to the residual module to achieve feature extraction and nonlinear transformation. In the residual module, the input data first passes through a fully connected layer to obtain the output. Next, the input is processed using the ReLU activation function and then connected to another fully connected layer. Then, a residual connection mechanism is used to add the original input to the result processed by the two fully connected layers, thus preserving input information and preventing gradient vanishing in deep networks. After the residual connection is completed, another ReLU activation function is used to further introduce nonlinearity, enhancing the network's expressive power. This completes the construction of a standard residual module.

[0039] After the input data has been processed by two such residual modules, the features are fully extracted and enhanced. Then, a fully connected layer is added. This fully connected layer integrates the previously extracted features and maps them to the output space. Finally, the predicted confidence value is directly output. This value can be used to represent the probability of the model's prediction accuracy for each predicted bounding box result.

[0040] Step 2: Trajectory Prediction Data Preparation

[0041] During the data preparation phase, two types of key data need to be acquired to meet the network input requirements. Firstly, historical trajectory sequence data of the target in consecutive frames is obtained, recording detailed positional changes of each target over time, providing the network with reliable historical motion feature references. Secondly, based on the existing trajectory prediction network, predicted bounding boxes are generated for subsequent frames. These predicted bounding boxes encompass the target's estimated position in the next frame, serving as another important input to the network and together with the historical trajectory sequence data, forming a complete input data pair.

[0042] To ensure the richness and diversity of data and enhance the adaptability and robustness of the uncertainty prediction model in complex scenarios, this invention uses the outputs of bounding box prediction uncertainty networks at different training iteration stages as training data. By collecting prediction results from different training stages, the output differences of the prediction model during parameter updates and learning process evolution can be effectively captured. This allows for the simulation of the model's response to various data distributions in real-world applications, providing a more representative and challenging dataset for model training and helping the model accurately learn the uncertainty characteristics of prediction results.

[0043] Step 3: Calculate the prediction confidence label:

[0044] The output of this bounding box prediction uncertainty network is the uncertainty confidence level; therefore, the definition of confidence level in the training labels is another key point. For the uncertainty confidence level, this invention uses the predicted target location result... Transform into ,in The coordinates of the left and top boundaries are... Given the right and bottom boundary coordinates, calculate the root mean square error between the predicted bounding box vertex coordinates and the actual labeled coordinates. Let the vertex coordinates of the actual labeled bounding box be... The root mean square error between the predicted bounding box vertex coordinates and the actual labeled coordinates is... The calculation formula is as follows:

[0045]

[0046] The root mean square error is normalized to the [0,1] interval, and the calculation method is as follows:

[0047]

[0048] The minimum value of the error is The maximum value is .

[0049] Convert the prediction confidence index by subtracting the error value from 1:

[0050]

[0051] The closer the confidence level is to 1, the lower the prediction uncertainty; the closer it is to 0, the higher the uncertainty and the lower the prediction reliability.

[0052] Step 4: Predicting Uncertainty - Network Training

[0053] The training of the bounding box prediction uncertainty network is mainly achieved by minimizing the root mean square error between the prediction uncertainty and the true label. The network's objective is to minimize the confidence level of the prediction uncertainty. Confidence label of true uncertainty Differences between them:

[0054]

[0055] in: It is a loss function. It is the number of samples in the batch; It is the network to the first Uncertainty prediction for a single sample; It is the first The confidence label for the true uncertainty of each sample.

[0056] Step 5: Predictive Network Inference:

[0057] Predicted bounding box of input target and historical trajectory The trained network outputs confidence scores. Quantitative prediction uncertainty confidence level The closer to 1, the lower the uncertainty and the higher the reliability of the prediction.

[0058] Example:

[0059] (1) Network parameter specification

[0060] The network input consists of two parts: one is a 4D predicted bounding box. Its form is Second, the 40-dimensional historical trajectory. The trajectory is composed of the bounding box coordinates of the target over the past 10 frames. When the trajectory length is less than 10, the missing parts are filled with [0,0,0,0]. The total dimension of the network input is 44.

[0061] (2) Data preparation and preprocessing

[0062] The training and validation of the network for predicting uncertainty are based on a self-made campus dataset with an image resolution of 640×360, covering three types of target objects: vehicles, pedestrians, and roadblocks.

[0063] Based on the above dataset, extract the ground truth bounding boxes of 10 consecutive frames for each target as the historical trajectory. , , This represents the true prediction value corresponding to the trajectory. During the experiment, an existing spatiotemporal prediction network was used, with historical trajectories as input, to obtain the predicted bounding box. .

[0064] (3) Calculation of prediction confidence labels

[0065] Taking one of the trajectories as an example, where the bounding box is predicted... True bounding box Calculated according to the formula Then it is normalized, and the dataset is centralized. , Then normalized Confidence level labels: Here, the closer the confidence level is to 1, the more reliable the prediction result.

[0066] (4) Network training

[0067] Training parameter settings: Batch size (N): 16; Number of iterations: 100 epochs; Optimizer: Adam.

[0068] (5) Reasoning effect

[0069] The predicted confidence and calculated confidence were quantitatively evaluated using the root mean square error (RMSE). The results showed that the RMSE value was 0.0684. This low error value indicates that the predicted confidence and the true confidence labels on the validation set have high consistency, fully demonstrating that the constructed network model has excellent training performance and can accurately fit the data samples.

[0070] Meanwhile, a statistical analysis was conducted on the intersection-union ratio (IUU) between the prediction confidence and the predicted bounding box and the true value. The results showed that the two exhibited a significant positive correlation, which is completely consistent with the inherent laws in real-world scenarios, further confirming the reliability and rationality of the model's output.

[0071]

Claims

1. A method for modeling uncertainty in bounding box prediction based on residual learning, characterized in that... The method includes the following steps: Step 1: Construct a bounding box prediction uncertainty network based on fully connected layers and residual modules; Step 2: Trajectory Prediction Data Preparation Step 2:

1. Obtain the historical trajectory sequence data of the target in consecutive frames, and record in detail the positional change information of each target in the time series; Step 22: Based on the existing trajectory prediction network, generate predicted bounding boxes for subsequent frames. These predicted bounding boxes contain the estimated position information of the target in the next frame and serve as another important input to the network. Together with the historical trajectory sequence data, they form a complete input data pair. Step 3: Calculate the prediction confidence label: Step 31: Calculate the predicted bounding box results. Transform into ,in The coordinates of the top left boundary are... Given the coordinates of the bottom right boundary, calculate the root mean square error between the predicted bounding box vertex coordinates and the actual labeled coordinates. ; Step 32: Normalize the root mean square error to the [0,1] interval; Step 3: Subtract the error value from 1 to convert it into a prediction confidence index: Step 4: Predicting Uncertainty - Network Training By minimizing the confidence level of prediction uncertainty Confidence label of true uncertainty The differences between them enable the training of the network for predicting uncertainty in bounding boxes; Step 5: Predictive Network Inference: Predicted bounding box of input target and historical trajectory The trained network outputs confidence scores. .

2. The method for modeling uncertainty in bounding box prediction based on residual learning according to claim 1, characterized in that... The bounding box prediction uncertainty network includes fully connected layers and residual blocks. The predicted bounding box results are first mapped to a feature space of a specific dimension through the fully connected layers. in, The feature vector after the bounding box is mapped. It is the weight matrix of the fully connected layer. It is a bias weight. It is the predicted bounding box result. It is the trajectory of history; The output of the fully connected layer is used as input and passes through two residual modules for feature extraction and nonlinear transformation. Then, it is connected to another fully connected layer to integrate the features extracted by the residual modules, map them to the output space, and directly output the predicted confidence value. .

3. The method for modeling uncertainty in bounding box prediction based on residual learning according to claim 2, characterized in that... In the residual module, the input data first passes through a fully connected layer to obtain the output. Next, the ReLU activation function is used to process it, followed by a fully connected layer. Then, a residual connection mechanism is used to add the original input to the result after processing by two fully connected layers. After the residual connection is completed, a ReLU activation function is used to further introduce nonlinear factors to enhance the network's expressive power.

4. The method for modeling uncertainty in bounding box prediction based on residual learning according to claim 1, characterized in that... The root mean square error between the predicted bounding box vertex coordinates and the actual labeled coordinates The calculation formula is: In the formula, Provide the vertex coordinates of the actual bounding box.

5. The method for modeling uncertainty in bounding box prediction based on residual learning according to claim 1, characterized in that... The formula for normalizing the root mean square error is: The minimum value of the error is The maximum value is .

6. The method for modeling uncertainty in bounding box prediction based on residual learning according to claim 1, characterized in that... The confidence level of the prediction uncertainty Confidence label of true uncertainty The formula for the difference between them is as follows: in: It is a loss function. It is the number of samples in the batch; It is the network to the first Uncertainty prediction for a single sample; It is the first The confidence label for the true uncertainty of each sample.

Images