Speech Enhancement Training with Style Transfer for Hearing Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech enhancement algorithms on hearing devices struggle to improve speech clarity due to the lack of clarity in speech itself, such as mumbling or reverberant speech, which noise removal techniques cannot address, and speech style transfer algorithms require high computational power and long processing times, making them unsuitable for real-time use on hearing devices.
Innovation Solution
A method for training a speech enhancement neural network that incorporates speech style transfer capabilities by using a speech style transfer algorithm as a target for supervised training, allowing the network to apply speech style transfer without the complexity of the original algorithm, and is designed for execution on hearing devices with low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech style transfer algorithms are used to improve speech clarity, then speech intelligibility is improved, but computational power requirements increase and processing time increases
Solution Approach 1:
The speech style transfer algorithm is executed in advance during an offline training phase to generate target speech samples. These pre-processed samples are then used to train a speech enhancement neural network that can be efficiently deployed on hearing devices. This preliminary action transfers the heavy computational burden from the end-use device to an offline processing stage.
Solution Approach 2:
Instead of deploying the complex speech style transfer algorithm directly on the hearing device, the patent creates a simplified copy in the form of a trained neural network model. This model captures the essential speech enhancement capabilities while requiring minimal computational resources, allowing it to run efficiently on resource-constrained hearing devices in real-time.
2Measurement precision
If speech style transfer algorithms are used to improve speech clarity, then speech intelligibility is improved, but processing time increases
Solution Approach 1:
The speech style transfer processing is performed in advance during offline training to create the target speech samples. This preliminary computation allows the actual speech enhancement on the hearing device to use a pre-trained model that operates quickly in real-time, thus reducing processing time during actual use.
Solution Approach 2:
The patent uses a simplified neural network model that serves as a lightweight approximation of the full speech style transfer algorithm. This simplified model requires minimal processing time and computational resources, enabling real-time operation on hearing devices while sacrificing some of the complexity of the original algorithm.
3Object-affected harmful factors
If noise removal techniques are used, then background noise is reduced, but speech clarity issues such as mumbling and reverberation cannot be addressed
Solution Approach 1:
The patent combines noise removal functionality with speech style transfer capabilities in a unified speech enhancement neural network. The network simultaneously performs denoising and speech clarity improvement by learning from target samples that have been processed through speech style transfer, which addresses both background noise and speech quality issues like mumbling and reverberation.
Solution Approach 2:
The speech enhancement neural network is designed to perform multiple functions: it removes background noise, improves speech clarity, and applies speech style transfer. This multi-functional approach allows a single model to address various speech degradation issues that would otherwise require separate processing stages.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A method for training a speech enhancement neural network (18) for being executed on a hearing device comprises the steps: providing a speech enhancement neural network (18), providing a speech style transfer algorithm for converting speech samples with a first speech style into speech samples with a second speech style, obtaining at least one training data set and applying supervised training on the speech enhancement neural network (18). The speech enhancement neural network (18) has a network audio input (28) for receiving an input audio signal, one or more network layers (29) for predicting an enhanced audio signal and/or a filter mask for filtering the input audio signal, and a network output (30) for outputting the enhanced audio signal and/or the filter mask. The at least one training data set comprises a training input audio signal (TI) comprising a speech sample and a target speech sample (TSP), which is used as training target and is obtained by applying the speech style transfer algorithm on the respective speech sample.