Multilingual Automated Assistant Query Translation Pipeline
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional automated assistants face challenges in communicating effectively in languages other than English, requiring resource-intensive configuration and lacking support for colloquialisms, which limits their ability to understand and respond to user queries in diverse languages.
Innovation Solution
The implementation of multiple processing pipelines that translate and rank user queries and responses, preserving slot values and colloquialisms, allows automated assistants to handle queries in various languages by translating queries to better-supported languages for intent identification and generating human-like responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If automated assistants are configured to communicate in new languages using conventional techniques, then language coverage is improved, but resource consumption and system complexity increase significantly
Solution Approach 1:
The patent introduces a translation service as an intermediary component that translates user queries from less-supported languages to better-supported languages. This mediator handles the complexity of multiple language support centrally, allowing the automated assistant to maintain its core processing logic in the better-supported language while still serving users in various languages through translation.
Solution Approach 2:
The translation service component serves multiple functions: it translates queries from various languages to the assistant's supported language, handles colloquialisms through translation, and enables the system to serve multiple language groups using a single core processing pipeline, reducing overall system complexity.
2Measurement precision
If automated assistants translate all user queries to better-supported languages, then processing accuracy is improved, but colloquialisms and proper nouns may be lost or mistranslated
Solution Approach 1:
The system applies different processing qualities to different parts of the query. Slot values and proper nouns are preserved in their original form without translation, while the surrounding contextual language is translated to the better-supported language for accurate processing. This local differentiation maintains both accuracy and colloquialism preservation.
Solution Approach 2:
The system extracts slot values and proper nouns from the translated query and preserves them separately in their original form. This extraction ensures that colloquialisms, names, and specific terms are not lost or mistranslated during the translation process, while the rest of the query benefits from translation to the better-supported language.
3Ease of operation
If automated assistants generate responses in multiple languages, then user accessibility is improved, but translation errors and loss of colloquial tone occur
Solution Approach 1:
The translation service acts as a mediator that translates the automated assistant's responses from the better-supported language back to the user's original language. This intermediary translation approach maintains reliability by using the same translation infrastructure that was used for query translation, ensuring consistency while improving accessibility to multiple language users.
4Reliability
If automated assistants process queries in the user's original language, then colloquialism understanding is improved, but language coverage remains limited to well-supported languages
Solution Approach 1:
The translation service serves as an intermediary that bridges less-supported languages with the assistant's core processing capabilities. By translating queries from various languages to the better-supported language, the system extends its language coverage while maintaining the ability to process colloquialisms effectively in the target language.
Data Source
AI summary
Techniques described herein relate to facilitating end-to-end multilingual communications with automated assistants. In various implementations, speech recognition output may be generated based on voice input in a first language. A first language intent may be identified based on the speech recognition output and fulfilled in order to generate a first natural language output candidate in the first language. At least part of the speech recognition output may be translated to a second language to generate an at least partial translation, which may then be used to identify a second language intent that is fulfilled to generate a second natural language output candidate in the second language. Scores may be determined for the first and second natural language output candidates, and based on the scores, a natural language output may be selected for presentation.