Estimating Output Uncertainty in Deterministic Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deterministic artificial neural networks (ANNs) face challenges in estimating output uncertainty, which is crucial for improving reliability and establishing efficient learning strategies, as they do not inherently consider Bayesian prediction distributions, making it difficult to quantify the range of possible outputs for the same input.
Innovation Solution
The method involves generating a proxy Gaussian process model using a dataset combined with the output of a trained deterministic ANN, allowing for indirect estimation of output uncertainty through predictive variance, leveraging the equivalence between Gaussian process models and probabilistic neural networks, and Bayesian interpretations of kernel ridge regression algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a probabilistic Bayesian neural network is used to directly measure uncertainty, then uncertainty measurement capability is improved, but computational complexity and data application issues worsen
Solution Approach 1:
The patent introduces a Gaussian process model as an intermediary to estimate the output uncertainty of a deterministic neural network. Instead of modifying the deterministic network to add uncertainty measurement capability directly, the Gaussian process acts as a separate model that learns from the deterministic network's outputs and provides uncertainty estimates, thus avoiding the computational complexity of Bayesian neural networks while achieving uncertainty measurement.
2Productivity
If a deterministic neural network is used for bigdata learning, then computational efficiency is improved, but uncertainty measurement capability worsens
Solution Approach 1:
The patent segments the uncertainty measurement function from the deterministic neural network's primary prediction function. The deterministic network maintains its computational efficiency for bigdata learning, while a separate Gaussian process model handles the uncertainty measurement task by learning from the deterministic network's outputs, allowing both functions to operate independently with their respective optimizations.
3Measurement precision
If the structure of a deterministic neural network is modified to enable uncertainty estimation, then uncertainty measurement capability is improved, but model complexity and training difficulty worsen
Solution Approach 1:
The patent uses a Gaussian process model as an intermediary that takes the deterministic neural network's outputs as input and generates uncertainty estimates. This approach avoids modifying the deterministic network's structure, maintaining its simplicity and ease of training, while the Gaussian process handles the uncertainty estimation without requiring changes to the original model architecture.
Data Source
AI summary
Disclosed is a method and system for estimating output uncertainty of a deterministic artificial neural network (ANN). An output uncertainty estimation method of a deterministic ANN may include generating a dataset by combining training data used for training of a deterministic ANN model and output of the deterministic ANN model trained with the training data; and estimating output uncertainty of the deterministic ANN model based on output for test data of a proxy Gaussian process model trained through the generated dataset.