Digital Assistant Command Mapping for New App Actions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital assistants face challenges in efficiently interacting with new applications and understanding new commands, requiring cumbersome and time-intensive training processes that hinder developer integration and user access.
Innovation Solution
The method involves receiving spoken inputs, determining command associations with application metadata, and executing actions without lengthy registration, using lightweight natural language models to reduce processing power, and integrating applications over time to enhance user intent determination and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If digital assistant requires training to interact with new applications and process new commands, then the assistant can accurately execute complex application-specific tasks, but the training process becomes cumbersome and time-intensive
Solution Approach 1:
The application provides metadata about its actions and associated commands in advance, before the digital assistant needs to execute them. This preliminary provision of action descriptions and command mappings allows the assistant to understand and execute commands without time-consuming training or registration processes.
Solution Approach 2:
Metadata acts as an intermediary between the application and the digital assistant. It bridges the gap by providing a standardized interface that describes application actions and their corresponding commands, enabling the assistant to interact with applications without direct training on each specific application.
2Measurement precision
If digital assistant uses comprehensive training methods to learn new commands, then command recognition accuracy improves, but the complexity of system integration increases
Solution Approach 1:
The patent extracts the essential information needed for command recognition into a simplified metadata structure. By taking out only the necessary command-action mappings and action descriptions from the complex application code, the system achieves accurate command recognition without requiring complex integration processes.
Solution Approach 2:
Instead of training the digital assistant to understand each application's unique command structure, the system creates a standardized copy of command mappings through metadata. This allows the assistant to recognize commands across different applications using a unified approach rather than learning each application individually.
3Measurement precision
If digital assistant processes every spoken command through full analysis, then command accuracy is maintained, but processing power consumption increases
Solution Approach 1:
The application itself provides the command-action mappings through its metadata, eliminating the need for the digital assistant to perform extensive analysis to understand application-specific commands. The assistant simply queries the pre-provided metadata to determine the appropriate action.
Solution Approach 2:
Instead of performing complete analysis of every spoken command against all possible application behaviors, the system uses partial action by querying only the relevant metadata entries for the current application context. This reduces processing requirements while maintaining accuracy for recognized commands.
Data Source
AI summary
Systems and processes for operating a digital assistant are provided. An example method includes, at an electronic device with one or more processors and memory, while an application is open on the electronic device: receiving a spoken input including a command, determining whether the command matches at least a portion of a metadata associated with an action of the application, and in accordance with a determination that the command matches at least the portion of the metadata associated with the action of the application, associating the command with the action, storing the association of the command with the action for subsequent use with the application by the digital assistant, and executing the action with the application.