Probabilistic Neural Network for Shapley Value Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for understanding how trained neural networks operate are inefficient and unreliable, particularly due to the computational intensity of exact Shapley value calculations, which makes it impractical for complex networks, and the inaccuracies of heuristic-based approaches.
Innovation Solution
A computer-implemented method that generates input distributions based on input points associated with a trained neural network, propagates these distributions through a probabilistic neural network to compute estimated Shapley values, reducing computational resources and improving accuracy by linearizing the number of network evaluations required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exact Shapley value calculations are used to understand neural network operations, then measurement precision is improved, but productivity deteriorates due to exponential computational requirements
Solution Approach 1:
The patent changes the parameter representation from exact discrete Shapley value calculations to continuous probabilistic distributions. By modeling input variations as probability distributions and propagating them through the network, the method transforms the computational problem from exponential complexity to polynomial complexity, achieving both accuracy and efficiency
Solution Approach 2:
The patent replaces the mechanical enumeration process of exact Shapley value calculation with a probabilistic modeling approach. Instead of systematically evaluating all possible input combinations, the method uses probability distribution propagation through the neural network, substituting computational enumeration with statistical inference
2Productivity
If heuristic-based attribution methods are used to understand neural network operations, then productivity is improved by reducing computational requirements, but measurement precision deteriorates due to inaccurate and misleading results
Solution Approach 1:
The patent introduces probability distributions as an intermediary between the input data and the contribution analysis. Rather than directly computing attributions from raw inputs, the method uses probabilistic representations that capture input uncertainty and variability, enabling more accurate and reliable contribution measurements while maintaining computational efficiency
Data Source
AI summary
In various embodiments, a relevance application quantifies how a trained neural network operates. In operation, the relevance application generates a set of input distributions based on a set of input points associated with the trained neural network. Each input distribution is characterized by a mean and a variance associated with a different neuron included in the trained neural network. The relevance application propagates the set of input distributions through a probabilistic neural network to generate at least a first output distribution. The probabilistic neural network is derived from at least a portion of the trained neural network. Based on the first output distribution, the relevance application computes a contribution of a first input point included in the set of input points to a difference between a first output point associated with a first output of the trained neural network and an estimated mean prediction associated with the first output.