Electronic Device Phrase Timing for Voice-Aligned Robot Gestures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to generate robot gestures that align with voice tone changes and require significant user data for training, leading to suboptimal interaction and immersion.
Innovation Solution
An electronic device determines voice tone changes in phrases using pitch contour and utterance time, applying cubic spline interpolation to identify suitable gesture intervals, and trains models for natural interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a gesture is assigned to the entire utterance text, then the gesture can be represented from utterance start to end time point, but this hinders user immersion
Solution Approach 1:
The patent segments the utterance text into multiple phrases based on pause times between word segments. Each phrase is then independently processed to determine voice tone changes and generate corresponding gestures. This segmentation allows the gesture to align with specific semantic units rather than spanning the entire utterance, thereby maintaining user immersion while ensuring accurate gesture assignment to meaningful text portions.
2Ease of manufacture
If technologies generate a gesture based on grammar information or semantic information, then gesture generation is simplified, but connectivity with changes in voice tone is not considered
Solution Approach 1:
The patent introduces voice tone change as a new parameter for gesture generation, in addition to existing grammar and semantic information. The system extracts voice tone change features from the utterance and uses them to adjust gesture parameters, enabling the gesture to reflect the emotional and intonational characteristics of the speech while maintaining the simplicity of automated generation through algorithmic processing.
3Measurement precision
If using a model trained using machine learning method, then gesture generation accuracy can be improved, but requires a process in which a user constructs a huge amount of data, requiring time and effort
Solution Approach 1:
The patent implements a self-service approach where the system automatically extracts training data from existing utterance texts and voice recordings without requiring manual data construction by users. The system automatically pairs text segments with corresponding voice tone changes and generates training datasets ready for model training, eliminating the time-consuming manual data preparation process while maintaining high gesture generation accuracy.
4Productivity
If word segment has short utterance time, then more word segments can be processed, but it is difficult to secure enough time to generate a gesture
Solution Approach 1:
The patent merges adjacent word segments into phrases based on pause time analysis. By combining multiple short word segments into longer phrase units, the system ensures that each phrase has sufficient duration for gesture generation while maintaining efficient processing. This merging strategy allows the system to handle more content overall while providing adequate time for accurate gesture synthesis within each phrase.
Data Source
AI summary
In an electronic device, and control method thereof, at least one processor can identify a first word segment being a first unit on grammar and a second word segment being a second unit on grammar from a target sentence included in a corpus, determine a target phrase including the first word segment and the second word segment, based on comparing a pause time between the first word segment and the second word segment and a threshold time, and determine a change in voice tone in the target phrase, based on information about a pitch contour of voice data obtained by applying the target sentence to a text-to-speech model and an utterance time of each word segment included in the target phrase. Thereby, the device can determine a gesture of a robot while outputting the target sentence.