Latent-Representation Explainability Masks for Neural Image Classifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current state-of-the-art techniques in machine learning lack quantitative metrics for explainability of neural network models, particularly in real-world deployment scenarios, leading to potential biases and inaccuracies in model predictions.
Innovation Solution
A method for determining an explainability mask by optimizing an objective function that adjusts a latent representation of an input image to identify the most critical regions for a neural network's classification, using a modification component and classification score component to ensure minimal perturbation while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks are trained to provide high confidence predictions, then prediction accuracy is improved, but explainability and reliability deteriorate as models can be easily tricked into providing false predictions
Solution Approach 1:
The patent implements a feedback mechanism where the neural network's own predictions are continuously evaluated and used to adjust the network's behavior. The system feeds back the difference between predicted and actual outcomes to improve future predictions, creating a self-correcting system that enhances both accuracy and reliability through iterative learning
Solution Approach 2:
The patent introduces an intermediary evaluation layer that acts as a mediator between the neural network's predictions and the final output. This intermediary component verifies and validates predictions before they are finalized, providing an additional check that prevents false high-confidence predictions while maintaining accurate forecasting
2Measurement precision
If more data is collected to improve model training, then model performance is improved, but loss of time and computational resources worsens
Solution Approach 1:
The patent applies preliminary action by pre-processing and pre-filtering data before it enters the main training pipeline. Data is prepared, cleaned, and organized in advance, reducing the computational burden during actual training. This preliminary preparation allows the model to train faster while maintaining high performance by working with pre-optimized data
Solution Approach 2:
The patent implements partial action by using a two-stage training approach where the model is first trained on a subset of representative data to achieve baseline performance, then fine-tuned on additional data. This selective application of training data reduces overall training time while still achieving high model performance through focused learning on critical datasets
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
The invention relates to a computer-implemented method of determining an explainability mask for classification of an input image by a trained neural network. The trained neural network is configured to determine the classification and classification score of the input image by determining a latent representation of the input image at an internal layer of the trained neural network. The method comprises accessing the trained neural network, obtaining the input image and the latent representation thereof and initialising a mask for indicating modifications to the latent representation. The mask is updated by iteratively adjusting values of the mask to optimise an objective function, comprising i) a modification component indicating a degree of modifications indicated by the mask, and ii) a classification score component, determined by applying the indicated modifications to the latent representation and determining the classification score thereof. The mask is scaled to a spatial resolution of the input image and output.