Digital Assistant Extension Ranking via Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital assistants face challenges in managing third-party extensions, including identifying when to invoke bots, preventing over or under triggering, and ranking multiple bots that can provide similar functionalities, leading to a suboptimal user experience.
Innovation Solution
The implementation of a machine learning-based ranking and selection model that evaluates bots using curated queries to identify features and train a model for automatic ranking and selection, ensuring seamless integration of new extensions with built-in and existing functionalities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple third-party bots are added to extend digital assistant functionality, then the versatility and capability of the digital assistant improve, but the complexity of managing and ranking these bots increases
Solution Approach 1:
The system automatically ranks and selects bots using machine learning models without requiring manual configuration or intervention. The bots are evaluated based on their performance metrics and features, and the system self-manages the ranking process, reducing the complexity of bot management while maintaining versatility.
Solution Approach 2:
The system uses multiple parameters including precision, recall, F1 scores, and other performance metrics to evaluate and rank bots. By changing and optimizing these parameters automatically through machine learning, the system manages the complexity of multiple bots while preserving their functional diversity.
2Measurement precision
If automated machine learning ranking is implemented, then the precision of bot selection improves, but the computational resources and time required increase
Solution Approach 1:
The machine learning models are trained in advance using historical bot performance data and evaluation metrics. This preliminary training allows the system to quickly rank new bots without requiring extensive real-time computation, thus improving selection accuracy while minimizing time loss during actual bot ranking operations.
3Reliability
If strict bot ranking criteria are applied, then the reliability of bot performance improves, but the quantity of available bots decreases
Solution Approach 1:
The system uses multiple performance parameters including precision, recall, and F1 scores to evaluate bots. By adjusting and optimizing these parameters dynamically, the system can maintain high reliability standards while still accommodating a larger number of bots that meet the criteria, thus balancing reliability with quantity.
Data Source
AI summary
Representative embodiments disclose mechanisms to automatically rank and select extensions triggered in a digital assistant. A sample set of extensions are executed against a set of curated queries in order to extract a set of features and/or statistics. The system trains a machine learning model based on the features and/or statistics to rank and select extensions based on their response to a query. New extension incorporated into the system are executed against a second set of curated queries to obtain a set of extracted features and/or statistics which are saved for use at runtime. At runtime, a query phrase received by the system triggers one or more tasks from extensions. Extracted features for the triggered extensions are combined with stored features/statistics and at least a subset of the results presented to the trained ranking and selection model. The model ranks and selects appropriate tasks which are presented to the user.