Multilingual Automated Assistant Query Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated assistants face challenges in communicating effectively in languages other than English, requiring resource-intensive configuration, including generating trigger grammars, recognizing query intents, and producing natural language output, which is hindered by language idiosyncrasies and the scarcity of language experts.
Innovation Solution
Implementing a method that processes user queries through multiple pipelines, translating queries to better-supported languages while preserving slot values and using machine learning models to generate human-like natural language output, which includes colloquialisms, to enhance language coverage and user understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If automated assistants are configured to communicate in new languages using conventional techniques, then language coverage is improved, but resource consumption and system complexity increase significantly
Solution Approach 1:
The patent introduces a translation service as an intermediary component that translates user queries from less-supported languages to better-supported languages. This mediator handles the complexity of language-specific processing externally, allowing the automated assistant to maintain its core functionality while expanding language coverage without proportionally increasing system complexity
Solution Approach 2:
The patent makes the automated assistant multi-functional by enabling it to operate in multiple languages through a unified architecture. Instead of creating separate language-specific systems, a single assistant instance can handle multiple languages by routing queries through the translation service when needed, achieving universal language support without duplicating infrastructure
2Measurement precision
If automated assistants process queries in less-supported languages directly, then response accuracy improves, but the need for language experts and configuration resources increases
Solution Approach 1:
The translation service acts as an intermediary that bridges the gap between less-supported languages and the assistant's core language capabilities. This allows the system to maintain high response accuracy by processing queries in better-supported languages while reducing the need for extensive language-specific configuration resources
Solution Approach 2:
The system creates a translated copy of the user query in a better-supported language, processes this copy through the assistant's proven accurate processing pipeline, and then uses the results to generate responses. This copying approach leverages existing accurate processing capabilities rather than creating new language-specific processing from scratch
3Ease of operation
If colloquialisms are used in natural language output to make it sound more human, then user engagement improves, but translation accuracy to less-supported languages deteriorates
Solution Approach 1:
The system applies different quality levels to different parts of the response generation process. Colloquialisms are used in the assistant's internal processing and in responses to users with well-supported languages, while more formal, translation-friendly language is used when generating output that needs to be translated to less-supported languages. This local differentiation maintains user engagement where possible while preserving translation accuracy where needed
Data Source
AI summary
Techniques described herein relate to facilitating end-to-end multilingual communications with automated assistants. In various implementations, speech recognition output may be generated based on voice input in a first language. A first language intent may be identified based on the speech recognition output and fulfilled in order to generate a first natural language output candidate in the first language. At least part of the speech recognition output may be translated to a second language to generate an at least partial translation, which may then be used to identify a second language intent that is fulfilled to generate a second natural language output candidate in the second language. Scores may be determined for the first and second natural language output candidates, and based on the scores, a natural language output may be selected for presentation.