Surrogate Model Active Learning for Virtual Assistant Data Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models for virtual assistants is challenging due to the lack of training data and the resource-intensive nature of creating accurate and reliable data, leading to poor predictions and erroneous functions.
Innovation Solution
The use of a surrogate machine learning model via knowledge distillation to improve active learning by efficiently selecting data samples from an unlabeled pool, generating a proxy model that outputs additional information like confidence scores to enhance the training dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a third-party machine learning model is used to process unlabeled data, then prediction accuracy is improved, but resource consumption and cost increase
Solution Approach 1:
The patent creates a surrogate model that copies the essential predictive capabilities of the third-party model. This local copy enables the system to make predictions using the surrogate model instead of repeatedly querying the resource-intensive third-party model, thereby reducing resource consumption while maintaining prediction accuracy
Solution Approach 2:
The surrogate model is trained in advance on labeled data before being deployed for predictions. This preliminary training action prepares the surrogate model to handle prediction tasks independently, eliminating the need for continuous resource-intensive queries to the third-party model during operation
2Measurement precision
If more training data is collected to improve model performance, then prediction accuracy is improved, but data annotation time and cost increase
Solution Approach 1:
The system uses the trained surrogate model to automatically predict labels for unlabeled data samples. This self-service capability allows the system to generate its own training data with minimal human intervention, significantly reducing data annotation time and cost while improving model performance through increased training data volume
3Use of energy by moving object
If a surrogate model is trained locally, then resource consumption during prediction is reduced, but initial training time and computational cost increase
Solution Approach 1:
The surrogate model undergoes preliminary training during an offline phase using available labeled data. This upfront investment in training time creates a ready-to-use model that can then serve predictions efficiently without requiring continuous resource-intensive operations, balancing initial training cost against ongoing prediction efficiency
Data Source
AI summary
Systems and methods of training a model is provided. The system can identify an unlabeled data set with phrases received by a virtual assistant that interfaces with one or more virtual applications to execute one or more functions. The system can query the unlabeled data set to select a first set of phrases based at least on one or more confidence scores output by a surrogate model that corresponds to a third-party model maintained by a third-party system. The system can receive, via a user interface, indications of functions to be executed by the one or more virtual applications responsive to the selected first set of phrases. The system can provide, to the third-party system, the indications of functions for the selected first set of phrases to train the third-party model and configure the virtual assistant to execute a function responsive to a phrase in the first set of phrases.