Neural Network Input Error Detection via Activation Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks lack transparency and predictability due to their complex structures, leading to user distrust and reduced adoption in healthcare and other applications, as users cannot easily verify the validity of outputs or understand the decision-making processes.
Innovation Solution
A method that uses a neural network to process multiple inputs, determining similarity indicators between activation values of selected neurons across layers to predict whether one input has been inaccurately processed, enabling identification of inconsistencies and prompting retraining or user attention to potential inaccuracies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep neural networks are used to improve accuracy and performance, then the technology becomes more powerful, but the system complexity increases making it hard for users to understand the decision-making processes
Solution Approach 1:
The patent introduces an intermediary system that compares activation values of selected neurons across multiple inputs to assess processing consistency. This intermediary layer provides transparency without modifying the complex neural network structure itself, allowing users to verify decision-making processes while maintaining high accuracy performance.
Solution Approach 2:
The patent segments the neural network analysis by selecting specific neurons from specific layers for comparison, rather than analyzing the entire network. This segmentation makes the complex system more manageable and understandable by focusing on key components that contribute to decision-making.
2Reliability
If deep neural networks are used to improve accuracy, then better performance is achieved, but transparency is reduced making it hard for users to verify output validity
Solution Approach 1:
The patent implements a feedback mechanism where activation values from multiple inputs are compared to assess whether the neural network is processing inputs consistently. This feedback loop provides users with information about processing reliability, enabling them to verify output validity while maintaining the high accuracy of the deep neural network.
3Reliability
If deep neural networks are used to improve accuracy, then better performance is achieved, but predictability is reduced making user behavior less predictable
Solution Approach 1:
The patent introduces an intermediary assessment system that monitors neural network processing consistency by comparing activation values. This intermediary provides predictable transparency about decision-making processes without altering the neural network's accurate but complex internal operations, making system behavior more predictable to users.
4Loss of information
If the entire neural network structure is analyzed, then complete understanding is achieved, but the computational burden and complexity increase significantly
Solution Approach 1:
The patent extracts only the necessary information for transparency by selecting specific neurons from specific layers for comparison, rather than analyzing the entire neural network structure. This extraction provides sufficient understanding of decision-making processes while keeping the analysis computationally efficient and manageable.
Data Source
AI summary
A method, system and computer-program product for identifying neural network inputs for a neural network that may have been incorrectly processed by the neural network. A set of activation values (of a subset of neurons of a single layer) associated with a neural network input is obtained. A neural network output associated with the neural network input is also obtained. A determination is made as to whether a first and second neural network input share similar sets of activation values, but dissimilar neural network outputs or vice versa. In this way a prediction can be made as to whether one of the first and second neural network inputs has been incorrectly processed by the neural network.