Mixed-Precision DNN Ensemble for Uncertainty Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classification networks, such as deep neural networks (DNNs), lack uncertainty estimates, leading to overfitting and poor performance with out-of-distribution data, which can result in fatal failures in safety-critical systems due to the absence of precision in prediction accuracy.
Innovation Solution
A mixed-precision DNN ensemble is introduced, comprising a primary model and auxiliary models with different precision levels, where predictions from both are combined to generate a prediction with an uncertainty estimate, using a fusion module to calculate an average mean predicted category and variance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single high-precision DNN model is used, then prediction accuracy is improved, but the system cannot provide uncertainty estimates and is vulnerable to overfitting and out-of-distribution data
Solution Approach 1:
The system segments the prediction task into multiple parallel DNN models with different precision levels (e.g., 8-bit, 16-bit, 32-bit). Each model processes the same input independently, and their predictions are aggregated to produce both a final prediction and an uncertainty estimate. This segmentation allows the system to maintain high accuracy while providing reliability information through the variance of predictions across models.
Solution Approach 2:
The system creates a composite prediction structure by combining outputs from multiple DNN models with heterogeneous precision characteristics. Rather than relying on a single model, the ensemble combines predictions from models of varying precision levels, creating a composite output that includes both the predicted class and an uncertainty measure derived from the distribution of predictions across the ensemble.
2Reliability
If a Bayesian network is used to provide probabilistic predictions, then uncertainty estimates are improved, but the network is hard to train and does not scale for large-scale problems
Solution Approach 1:
Instead of using a single complex Bayesian network, the system creates multiple copies of simpler DNN models with different precision configurations. These copied models are trained using standard techniques, and their collective behavior provides Bayesian-like uncertainty estimates without requiring the complex training procedures associated with traditional Bayesian networks. This approach maintains scalability while achieving probabilistic prediction capabilities.
Solution Approach 2:
The system varies the precision parameters (bit-width) across multiple DNN model copies to create an ensemble with diverse computational characteristics. By changing the precision parameter across models rather than changing the fundamental network architecture or training methodology, the system achieves scalability and ease of training while still providing uncertainty estimates through the variation in predictions across different precision levels.
3Reliability
If multiple DNN models with different precision levels are used, then uncertainty estimates are provided, but computational complexity increases
Solution Approach 1:
The system applies local quality by assigning different precision levels to different models in the ensemble based on their specific roles or input characteristics. Rather than uniformly increasing precision across all computations, the system strategically distributes computational resources, using lower-precision models for tasks where high accuracy is less critical and higher-precision models where uncertainty estimation is most valuable, thereby optimizing the overall computational complexity-reliability trade-off.
Data Source
AI summary
A hardware accelerator and method for a mixed-precision deep neural network (DNN) ensemble are provided. The hardware accelerator includes a DNN primary module, a number of DNN auxiliary modules and a fusion module. The DNN primary module processes a DNN primary model having a primary precision level, and each DNN auxiliary module processes a DNN auxiliary model having an auxiliary precision level less than the primary precision level. The DNN primary model and each DNN auxiliary model are configured to determine a mean predicted category and a variance based on input data. The fusion module is configured to receive the mean predicted categories and variances from the DNN primary model and each DNN auxiliary model, determine an average mean predicted category and an average variance based on the mean predicted categories and variances, and output the average mean predicted category and the average variance.