Neural Network Training Using Interpretability Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks often require large amounts of diverse training data, which is resource-intensive and inefficient, as it's unclear which data is valuable for training, leading to stagnation in generating optimized networks, especially in the era of BigData.
Innovation Solution
A method that involves training a neural network using a subset of data fields selected based on interpretability scores, iteratively refining the network by combining relevant data fields to improve prediction accuracy, and utilizing these insights to train a second neural network for specific predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large amounts of diverse training data are used to train artificial neural networks, then prediction accuracy is improved, but resource consumption (time, money, storage/memory) increases significantly
Solution Approach 1:
The patent extracts and identifies the most valuable subset of training data features using interpretability scores (e.g., SHAP values) that measure the importance of each feature. By taking out only the essential features rather than using all available data, the system achieves accurate predictions while significantly reducing the quantity of training data required, thus resolving the contradiction between prediction accuracy and resource consumption.
Solution Approach 2:
The patent changes the parameter of data selection from using all available features to using a filtered subset based on interpretability thresholds. By adjusting the threshold parameter, the system can optimize the balance between prediction accuracy and training efficiency, selecting only those features that meet the minimum relevance criteria, thereby reducing resource consumption while maintaining accuracy.
2Adaptability or versatility
If all available data fields are used for training, then comprehensive coverage is achieved, but training efficiency decreases due to inability to identify valuable data
Solution Approach 1:
The patent performs preliminary action by calculating interpretability scores for all data fields before the actual training process. This pre-analysis step identifies which features are most valuable, allowing the training process to focus only on relevant data. This preliminary filtering action maintains comprehensive coverage of important features while dramatically improving training efficiency by eliminating irrelevant data from the training pipeline.
Solution Approach 2:
The patent applies partial action by using only the necessary subset of features identified through interpretability analysis rather than all available features. The system determines the minimum sufficient set of features needed for accurate predictions, applying exactly the right amount of data processing effort without excess, thus optimizing training efficiency while maintaining adequate coverage of important data aspects.
3Measurement precision
If interpretability analysis is performed on all data fields, then feature relevance is accurately determined, but computational overhead increases
Solution Approach 1:
The patent segments the data analysis process into distinct phases: first, a preliminary interpretability analysis to identify potentially relevant features, and second, a focused training phase using only those identified features. This segmentation allows accurate feature relevance determination to be performed efficiently on a reduced subset rather than all data fields, reducing computational overhead while maintaining precision in identifying important features.
Data Source
AI summary
An example method disclosed herein includes receiving data from a database, where the data is indicative of a first instance of the database, and where the data comprises a plurality of fields. The method further includes training a first neural network for a first data field, where the training is based at least in part on utilizing the plurality of data fields and determining a first interpretability score for each data field of the plurality of data fields used to train the first neural network, where each first interpretability score is indicative of a relevance for each data field of the plurality of data fields used to train the first neural network. The method further includes selecting a subset of data fields from the plurality of data fields used to train the first neural network, where the selection is based at least in part on the first interpretability score for each data field exceeding a first relevance threshold. The method further includes training a second neural network using the selected subset of data fields, where the second neural network is trained to provide predictions related to the first data field.