Automated Assistant Speech Biasing for Non-Configured Apps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated assistants face limitations when interacting with application interfaces that are not pre-configured or are in a different language, leading to inefficient user interactions and increased resource usage due to frequent manual intervention and misrecognition errors.
Innovation Solution
An automated assistant that processes application interface content to identify controllable features and synonymous terms, allowing it to provide voice-based control even when the application is not pre-configured, by using historical data and language translation to improve speech recognition accuracy across languages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automated assistant processes application interface content to identify synonymous terms and bias speech processing, then speech recognition accuracy is improved, but computational resources and processing time are increased
Solution Approach 1:
The system performs preliminary processing by identifying synonymous terms and biasing speech processing configurations before actual speech recognition occurs. This advance preparation enables faster and more accurate speech recognition without requiring extensive real-time computational resources, as the speech processing system is pre-configured with relevant terminology and language models specific to the application context.
2Adaptability or versatility
If automated assistant interfaces with applications not pre-configured for assistant control, then functionality and versatility are improved, but reliability and control precision deteriorate due to misrecognition errors
Solution Approach 1:
The system introduces an intermediary layer that processes application interface content and generates synonymous term mappings between the application's language and the user's natural language. This intermediary speech processing configuration acts as a bridge, enabling accurate interpretation of user commands even when the application lacks pre-configured assistant control, thereby maintaining high control accuracy across diverse applications.
Solution Approach 2:
The system performs preliminary analysis of the application interface to identify controllable features and generate appropriate speech processing biases before user interaction begins. This advance configuration ensures that the speech recognition system is optimized for the specific application context, improving reliability without requiring pre-configured assistant controls in the application itself.
3Ease of operation
If automated assistant uses synonymous term identification and speech processing biasing, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The system implements a universal speech processing configuration mechanism that can adapt to any application by dynamically generating synonymous term mappings. Rather than requiring application-specific configurations, this multi-functional approach allows the same core mechanism to serve diverse applications, simplifying user interaction while managing complexity through a standardized adaptive framework.
Data Source
AI summary
Implementations set forth herein relate to an automated assistant that can interact with applications that may not have been pre-configured for interfacing with the automated assistant. The automated assistant can identify content of an application interface of the application to determine synonymous terms that a user may speak when commanding the automated assistant to perform certain tasks. Speech processing operations employed by the automated assistant can be biased towards these synonymous terms when the user is accessing an application interface of the application. In some implementations, the synonymous terms can be identified in a responsive language of the automated assistant when the content of the application interface is being rendered in a different language. This can allow the automated assistant to operate as an interface between the user and certain applications that may not be rendering content in a native language of the user.