Teacher-Student Model Architecture for Interpretable Decision Recommendations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online machine learning models lack global and local interpretability, and require extensive training data to achieve accuracy, limiting their ability to provide transparent and reliable decision recommendations in processes like travel reimbursement.
Innovation Solution
An online learning-based model architecture that uses a teacher model to incrementally update a student model, allowing for global and local interpretability while reducing the need for extensive training data, by extracting and updating weights from the teacher model to refine the student model's predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing online machine learning models are used to provide decision recommendations, then prediction capability is achieved, but global and local interpretability is lost and extensive training data is required
Solution Approach 1:
The patent segments the machine learning model into two distinct components: a teacher model that provides interpretability and a student model that delivers prediction accuracy. This segmentation allows each model to specialize - the teacher model focuses on maintaining interpretability through its structure, while the student model focuses on accuracy through learning from the teacher, thereby resolving the contradiction between interpretability and data efficiency
Solution Approach 2:
The teacher model acts as an intermediary between the available training data and the student model. Instead of the student model directly consuming extensive training data to achieve both accuracy and interpretability, the teacher model processes the data and transfers distilled knowledge to the student model, reducing the training data burden while maintaining interpretability through the teacher-student knowledge transfer mechanism
2Measurement precision
If extensive training data is collected to improve prediction accuracy, then model performance increases, but the complexity and resource requirements increase
Solution Approach 1:
The student model creates a simplified copy or representation of the teacher model's knowledge rather than directly replicating the teacher model's complex training process. The student model copies the essential predictive patterns from the teacher model through knowledge distillation, achieving similar prediction accuracy with reduced training complexity and fewer resources
Solution Approach 2:
The teacher model performs preliminary processing and knowledge extraction from the training data before the student model begins its training. By pre-processing the data and extracting key patterns in advance through the teacher model, the student model inherits this preliminary work, reducing its own training complexity and resource requirements while maintaining prediction accuracy
Data Source
AI summary
Aspects of the invention include selecting an activity as a selected activity. A method includes designating a subset of the set of activities as classes, collecting a log of inputs and outputs of each encountered activity as a data point each time the process is implemented, and extracting features from each data point that is collected to generate a feature vector from each data point. A teacher model is initialized with a first data point and updated with each data point subsequent to the first data point. A student model is initialized with a set of data points including the first data point such that every one of the classes is encountered at least once. The student model is updated with the teacher model. A set of features is input to the student model to obtain a prediction of the outcome of the selected activity.