In-Call Translation Sentence Segmentation for Real-Time TTS
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing translation technologies in electronic devices fail to provide seamless text-to-speech output during real-time conversations, particularly in multilingual settings, leading to inefficiencies and user experience issues.
Innovation Solution
An electronic device is equipped with a method that performs automatic speech recognition (ASR) to identify sentence endpoints based on pause sections, translates the text into a different language, and generates synthetic speech using text-to-speech (TTS) conversion, allowing for real-time in-call translation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If real-time translation is performed during calls, then user interaction in multilingual conversations is enhanced, but processing time and complexity increase
Solution Approach 1:
The utterance is divided into multiple sentences using pause sections as segmentation points. This allows the translation system to process and translate individual sentences sequentially rather than waiting for the complete utterance, reducing processing complexity while maintaining real-time translation capability
Solution Approach 2:
The system performs automatic speech recognition and sentence segmentation before translation. By identifying pause sections and segmenting sentences in advance, the system prepares the text for translation earlier, enabling smoother real-time translation output without overwhelming processing complexity
2Measurement precision
If complete utterance is processed for translation, then translation accuracy is improved, but response time increases
Solution Approach 1:
The complete utterance is segmented into multiple sentences based on pause sections. Each sentence can be translated and output independently, providing timely responses while maintaining translation accuracy for complete semantic units (sentences) rather than partial phrases
Solution Approach 2:
The system translates sentences partially as they are completed within the utterance rather than waiting for the entire utterance. This partial translation approach provides timely responses while ensuring each translated sentence maintains sufficient accuracy through proper sentence boundary detection
3Ease of operation
If text-to-speech conversion is performed on translated text, then user comprehension is enhanced, but processing time increases
Solution Approach 1:
The translated text is segmented into sentences corresponding to the original utterance segments. TTS conversion is performed on each sentence individually and output in sequence, enhancing user comprehension through audible translation while maintaining processing efficiency by avoiding conversion of entire utterances at once
Solution Approach 2:
The system performs TTS conversion and audio output continuously as sentences are translated, rather than batching all conversions after complete translation. This continuous useful action maintains user comprehension through timely audio feedback while minimizing total processing time
Data Source
AI summary
A method performed by an electronic device during a call is provided. The method includes receiving, via a microphone, an utterance from a user of the electronic device. The method includes performing, by the electronic device, automatic speech recognition (ASR) based on a speech signal corresponding to a portion of the utterance to generate a first text in a first language. The method includes identifying, by the electronic device, an end point of a sentence included in the first text based on at least one pause section associated with the first text. The method includes translating, by the electronic device, a portion of the first text corresponding to the sentence into a second text in a second language, based on the identified end point of the sentence included in the first text. The method includes performing, by the electronic device, a text-to-speech (TTS) conversion on the second text. The method includes generating, by the electronic device, a synthetic speech corresponding to a portion of the utterance before an end of the utterance received from the user, based on the TTS conversion.