On-Device Multi-Speaker AI Interpretation with Selective Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current on-device AI systems struggle to accurately interpret conversations between multiple speakers in real time and identify individual speakers for selective interpretation services.
Innovation Solution
An on-device AI device that preprocesses utterances, classifies them by speaker, extracts the utterance of a specific speaker, and interprets it into a target language, with the option to distribute AI interpretation processing to external devices when needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If on-device AI processes all utterances from multiple speakers, then interpretation completeness is improved, but processing accuracy and speaker identification capability deteriorate
Solution Approach 1:
The system segments the multi-speaker utterance stream into individual speaker components using speaker diarization technology. This segmentation enables the AI to process each speaker's utterance separately, thereby maintaining both interpretation completeness and speaker identification accuracy by treating each speaker as an independent processing unit.
Solution Approach 2:
The system introduces an intermediary speaker classification module between the input utterances and the interpretation engine. This intermediary component identifies and labels each utterance with its corresponding speaker information, enabling accurate speaker identification while maintaining complete interpretation of all utterances through a structured intermediate processing stage.
2Speed
If on-device AI performs real-time speaker classification and interpretation, then service responsiveness is improved, but device power consumption increases
Solution Approach 1:
The system performs partial processing by applying speaker classification only to necessary utterances and using simplified interpretation models for common scenarios. This partial action approach maintains real-time responsiveness for critical functions while reducing overall power consumption by avoiding exhaustive processing of all utterances with full AI models.
Solution Approach 2:
The system dynamically changes processing parameters such as model complexity, processing frequency, and classification thresholds based on operational conditions. During high-power states, more intensive speaker classification is applied, while in low-power states, the system reduces processing intensity, thereby balancing real-time performance with power consumption through adaptive parameter adjustment.
3Adaptability or versatility
If on-device AI handles complex multi-speaker interpretation locally, then service autonomy is improved, but device computational load increases
Solution Approach 1:
The system segments computational tasks by separating speaker classification functions from interpretation functions. This segmentation allows the device to handle classification locally with lower computational requirements while maintaining service autonomy, and enables optional offloading of more complex interpretation tasks to external servers when needed, thereby reducing overall device computational load.
Solution Approach 2:
The system implements a universal architecture that can operate in multiple modes: fully local processing for autonomy, hybrid processing for balanced performance, and cloud-dependent mode for reduced device complexity. This multi-functionality allows the same system to adapt to different computational load conditions while maintaining service autonomy where possible.
Data Source
AI summary
An on-device artificial intelligence (AI) device capable of interpreting a conversation between multiple speakers in real time and a method thereof are provided. The AI device can include an input module where utterance voice of a speaker is input; and a processor configured to perform AI processing to interpret the utterance voice of the speaker into a target language, wherein the processor is configured to when utterance voices are input from a plurality of speakers, preprocess the utterance voices, classify the preprocessed utterance voices by speaker, when a specific speaker is selected from the plurality of speakers, extract utterance voice of the selected specific speaker from the classified utterance voices by speaker, and interpret the utterance voice of the specific speaker into a target language and output it.