Neural Network Sensitivity Analysis Complexity Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sensitivity analysis methods for neural networks are computationally intensive and resource-consuming, especially when dealing with a large number of input variables, as they require multiple model score calculations for each record and input perturbation, leading to high complexity and processing time.
Innovation Solution
The method involves caching base outputs for neural network hidden layer nodes and efficiently calculating perturbed outputs using these cached values, reducing the complexity of score calculation from O(N*N*J) to O(J*N) by reusing base outputs and performing only one additional multiplication for each perturbation, thereby speeding up sensitivity analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional sensitivity analysis methods are used to analyze each input variable, then sensitivity information can be obtained, but the computation complexity increases to O(N*N*J) and processing time becomes very slow
Solution Approach 1:
The patent pre-calculates and caches the base outputs for all hidden layer nodes before performing sensitivity analysis. This preliminary computation stores intermediate results that can be reused for all subsequent perturbation calculations, eliminating the need to recompute these values for each input variable perturbation and thereby reducing the overall computational complexity from O(N*N*J) to O(J*N).
Solution Approach 2:
The patent combines multiple perturbation calculations by reusing the same cached base outputs across all input variable perturbations. Instead of performing independent full model scoring for each perturbation, the method merges the computational effort by systematically reusing intermediate results, reducing redundant calculations and improving efficiency.
2Measurement precision
If conventional sensitivity analysis methods are used, then sensitivity information can be obtained, but significant processing resources are consumed
Solution Approach 1:
The patent pre-calculates and caches the base outputs for all hidden layer nodes before performing sensitivity analysis. This preliminary computation stores intermediate results that can be reused for all subsequent perturbation calculations, eliminating the need to recompute these values for each input variable perturbation and thereby reducing the overall computational complexity from O(N*N*J) to O(J*N).
Solution Approach 2:
The patent identifies and discards redundant calculations in the conventional sensitivity analysis process. By caching base outputs and reusing them across multiple perturbations, the method recovers computational effort by avoiding repeated calculations of the same intermediate values, thereby reducing overall processing resource consumption.
3Measurement precision
If thorough sensitivity analysis is performed with many input variables, then complete sensitivity information is obtained, but the complexity of model scoring increases to O(N) and total complexity becomes O(N*N*J)
Solution Approach 1:
The patent pre-calculates and caches the base outputs for all hidden layer nodes before performing sensitivity analysis. This preliminary computation stores intermediate results that can be reused for all subsequent perturbation calculations, eliminating the need to recompute these values for each input variable perturbation and thereby reducing the overall computational complexity from O(N*N*J) to O(J*N).
Solution Approach 2:
The cached base outputs serve multiple purposes: they are used as the foundation for calculating sensitivity to each individual input variable perturbation. This universal intermediate result enables a single set of base outputs to support all N perturbation calculations, making the computation multi-functional and reducing overall complexity.
Data Source
AI summary
As part of neural network sensitivity analyses, base outputs of hidden layer nodes of a neural network model for non-perturbed variables can be reused when perturbing the variables. Such an arrangement greatly reduces complexity of the calculations required to generate outputs of the model. Related apparatus, systems, techniques and articles are also described.