Personalized Phoneme Synthesis for Speech Intelligibility Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech intelligibility training for hearing instrument users is time-consuming, expensive, and often ineffective due to the difficulty in creating personalized audio material that fits individual user requirements, leading to insufficient training in real-life noisy environments.
Innovation Solution
A method and arrangement that records sound from a user's environment, extracts characteristic voice and speech properties of communication partners, and synthesizes personalized phoneme stimuli to enhance training efficiency and effectiveness by using AI models like WaveNet, Tacotron, or variational autoencoders to create tailored training sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-recorded audio sequences are used for speech intelligibility training, then training material is available, but creating personalized audio material that fits individual user requirements is difficult and time-consuming
Solution Approach 1:
The patent uses voice cloning technology to create synthetic speech that copies the acoustic characteristics of real communication partners. Instead of manually recording personalized training material, the system synthesizes speech by copying voice properties (pitch, timbre, speaking rate) from short audio samples of the user's communication partners, enabling rapid personalization without time-consuming manual recording sessions.
Solution Approach 2:
The system automatically extracts voice characteristics and generates personalized phoneme sequences without requiring manual intervention. The hearing device or computing device autonomously processes audio samples, identifies communication partners, extracts voice properties, and creates tailored training material, eliminating the need for professional audio production services.
2Reliability
If conventional speech intelligibility training is provided, then training can be conducted, but it is often ineffective in real-life noisy environments
Solution Approach 1:
The patent creates training material with locally optimized characteristics by preserving the specific acoustic properties of the user's actual communication partners. Each phoneme sequence is synthesized to match the local voice qualities (pitch contours, timbre, speaking rate) of individual partners, making the training highly adapted to the user's specific listening environment rather than using generic standardized audio.
Solution Approach 2:
The system performs preliminary analysis of the user's acoustic environment by recording and analyzing audio samples to identify communication partners and extract their voice characteristics before generating the personalized training material. This preliminary action ensures that the training content is pre-adapted to the user's specific listening conditions, including background noise characteristics and partner speech patterns.
3Productivity
If personalized phoneme stimuli are synthesized using AI models, then training efficiency is improved, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary computing device that handles the complex AI-based speech synthesis tasks. Instead of embedding complex neural network models directly in the hearing device, the system uses a separate computing device (smartphone, server, or cloud service) to generate personalized phoneme sequences, which are then transmitted to the hearing device for playback. This intermediary approach enables advanced personalization while keeping the hearing device itself relatively simple.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for conducting a speech intelligibility training, and a corresponding arrangement are presented. Herein, a sound from an environment of a participant (36) is recorded. Speech of a speaker (38) different from the participant (36) is extracted from the recorded sound, and at least one characteristic voice property (VP) and/or speech property (SP) of the said speaker (38) is determined from the extracted speech. A plurality of test audio sequences (PS) are created, wherein each of said plurality of test audio sequences (PS) contains synthesized speech of a phoneme or phoneme combination and the speech is synthesized so to conform with said at least one characteristic voice property (VP) and/or speech property (SP) of said speaker (38). A training step is conducted in which one of said test audio sequences (PS) from the plurality is chosen, converted into sound and output to the participant (36). A response (S) of the participant (36) indicating a phoneme or phoneme combination understood by the participant (36) is collected, and a feedback (F) is output to the participant (36) whether or not the phoneme or phoneme combination indicated by the participant (36) as being understood corresponds to the phoneme or phoneme combination output to the participant (36) in the training step.