Phoneme Modification Engine for Speech Accent Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Voice conversations between individuals with different native languages and accents can be difficult to understand, especially in call-center or conference call settings, due to the limitations of automatic speech recognition (ASR) accuracy and Text To Speech (TTS) systems in reflecting speaking patterns.
Innovation Solution
A speech processing system that includes a phoneme recognition engine converting speech data into acoustic data, a phoneme modification engine applying rules to adjust the data, and a phoneme-to-speech engine generating output speech, allowing for real-time modification and normalization of speech to improve understanding across listeners.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a Text To Speech (TTS) engine is used to convert ASR output to speech, then speech output can be generated, but the system cannot accurately reflect the speaking patterns and accents of the original speaker
Solution Approach 1:
The system creates acoustic models by copying and analyzing the speaker's actual speech patterns, accent, and speaking style. These models are then used to synthesize speech that accurately replicates the original speaker's characteristics rather than using generic TTS voices.
Solution Approach 2:
The system modifies acoustic parameters such as pitch, duration, and spectral characteristics to match the original speaker's speaking patterns. By adjusting these parameters based on the acoustic model, the system can reflect different accents and speaking styles accurately.
2Productivity
If automatic speech recognition (ASR) is used to convert speech to text, then text output can be obtained, but the accuracy is limited by vocabulary and recognition capabilities
Solution Approach 1:
The system uses feedback from the acoustic model to continuously improve recognition accuracy. By comparing the acoustic features of the input speech with the stored speaker-specific patterns, the system can better identify and correct recognition errors, especially with proper nouns and speaker-specific terminology.
Solution Approach 2:
The system performs preliminary analysis of the speaker's voice characteristics and creates an acoustic model before the actual speech recognition task. This preliminary action enables the recognition system to adapt to the specific speaker's patterns, improving accuracy before the main recognition process begins.
3Ease of operation
If phoneme modification with rules is applied to adapt speech to different listeners, then communication effectiveness improves, but system complexity increases
Solution Approach 1:
The speech processing system is divided into distinct modules: phoneme recognition engine, phoneme modification engine with rule sets, and phoneme-to-speech engine. Each module handles a specific aspect of the processing, making the overall complex system more manageable and maintainable through functional segmentation.
Data Source
AI summary
A speech processing system includes a multiplexer that receives speech data input as part of a conversation turn in a conversation session between two or more users where one user is a speaker and each of the other users is a listener in each conversation turn. A speech recognizing engine converts the speech data to an input string of acoustic data while a speech modifier forms an output string based on the input string by changing an item of acoustic data according to a rule. The system also includes a phoneme speech engine for converting the first output string of acoustic data including modified and unmodified data to speech data for output via the multiplexer to listeners during the conversation turn.