Integrated Dropout FC Layer for Analytical Variance Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks lack the ability to compute variance representing a confidence interval for estimation results, making it difficult to determine reliable outputs, especially in safety-critical applications, as they typically require numerous computations to achieve accurate variance estimation, leading to a heavy computational burden.
Innovation Solution
An information estimation apparatus and method that incorporates an integrated layer combining a dropout layer and a fully connected layer, allowing for analytical variance computation using a data analysis unit to determine the data type of output elements and apply appropriate approximate computation methods, thereby reducing the need for extensive computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional neural networks perform numerous computations to estimate variance, then measurement precision of confidence interval is improved, but productivity deteriorates due to heavy computational burden
Solution Approach 1:
The patent replaces the mechanical computation system (repeated neural network forward propagation) with a mathematical analysis system (analytical variance computation). By deriving and applying variance computation formulas that directly calculate uncertainty without requiring multiple inference passes, the system achieves accurate variance estimation while dramatically reducing computational burden.
Solution Approach 2:
The patent performs preliminary computation during the learning phase by calculating and storing statistical properties (means and variances) of neuron activations. These pre-computed statistics are then reused during the estimation phase to quickly determine confidence intervals without requiring additional forward propagation computations.
2Reliability
If conventional neural networks perform numerous computations to obtain reliable variance, then reliability is improved, but loss of time increases due to extensive computation required
Solution Approach 1:
The patent substitutes time-consuming repeated mechanical computations with efficient mathematical formulas. The analytical variance computation method directly yields reliable confidence intervals using closed-form expressions that leverage pre-computed statistical properties, eliminating the need for multiple inference passes while maintaining reliability.
Solution Approach 2:
The system performs preliminary statistical analysis during training to compute and store the means and variances of neuron outputs. This preliminary action enables rapid, reliable variance estimation during deployment without requiring additional computation time, as all necessary statistical properties are already available from the learning phase.
Data Source
AI summary
A technique for stable and fast computation of a variance representing a confidence interval for an estimation result in an estimation apparatus using a neural network including an integrated layer that combines a dropout layer for dropping out part of input data and an FC layer for computing a weight is provided. When input data having a multivariate distribution is supplied to the integrated layer, a data analysis unit 30 determines, based on a numerical distribution of terms formed by respective products of each vector element of the input data and the weight, a data type of each vector element of output data from the integrated layer. An estimated confidence interval computation unit 20 applies an approximate computation method associated with the data type, to analytically compute a variance of each vector element of the output data from the integrated layer based on the input data to the integrated layer.