Output Signal Aggregation Using Synthetic Reference Instances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems, particularly Bayesian neural networks, struggle with accurately modeling uncertainty and choosing appropriate priors, leading to unreliable predictions in safety-critical applications like self-driving cars, especially when encountering data outside the training distribution.
Innovation Solution
The use of Functional Neural Processes (FNPs) to model distributions over functions by learning a graph of dependencies on latent representations, allowing for Bayesian modeling without explicit priors, and determining synthetic reference instances through continuous optimization to enhance prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Bayesian neural networks are used to model uncertainty, then uncertainty estimation capability is improved, but device complexity and difficulty of implementation increase due to the need to choose appropriate priors and perform complex inference
Solution Approach 1:
The patent extracts and removes the complex prior distribution selection and weight-space inference from the Bayesian neural network framework. Instead of working with full Bayesian networks requiring prior specifications, the invention extracts only the essential uncertainty modeling capability and implements it through functional processes that operate directly on input data without requiring complex prior definitions or weight-space inference.
Solution Approach 2:
The patent substitutes the mechanical Bayesian inference process (which requires specifying priors and performing complex posterior inference over weight spaces) with a functional process approach. The new system uses functional processes that directly map inputs to outputs with inherent uncertainty estimation, replacing the complex mechanical Bayesian framework with a more tractable functional alternative that achieves similar uncertainty modeling goals.
2Speed
If conventional neural networks are used for classification, then prediction speed is improved, but reliability deteriorates when encountering data outside the training distribution due to overconfident incorrect predictions
Solution Approach 1:
The patent introduces dynamics by making the prediction process adaptive based on input characteristics. The functional process dynamically adjusts its behavior based on whether the input resembles training data or represents out-of-distribution examples. This dynamic adaptation allows the system to maintain high confidence for reliable predictions while appropriately expressing uncertainty for unreliable predictions, all without sacrificing prediction speed.
3Ease of operation
If priors are employed in Bayesian neural networks for computational convenience, then ease of operation is improved, but manufacturing precision deteriorates due to difficulty in understanding parameter interactions with data
Solution Approach 1:
The patent creates a functional copy of the Bayesian inference capability that operates in function space rather than weight space. Instead of copying the complex Bayesian neural network framework with all its prior specification requirements, the invention creates a simplified functional process copy that captures the essential uncertainty modeling capability while being computationally tractable and easier to operate without sacrificing accuracy.
Data Source
AI summary
A computer-implemented method of training a function for use in controlling or monitoring a physical system operating in an environment. The function maps an input instance comprising sensor measurements to an output signal. The function is parameterized by a set of parameters including representations of multiple reference instances. Given a training input instance, a number of reference instances are identified as being similar to the training input instance, and their representations and/or output signals are aggregated into an aggregate latent representation for the training input instance. Based on this aggregate latent representation, an output signal for the training input instance is determined, which is compared to a training output signal to derive a training signal. At least a representation of a reference instance is adjusted according to the training signal, obtaining a reference instance not comprised in the training dataset.