Speech Enhancement Neural Network Training for Real-Time Hearing Devices

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing speech enhancement algorithms for hearing devices struggle to improve speech clarity due to lack of clarity in speech itself, such as mumbling or reverberant speech, and require high computational power and long processing times, making them unsuitable for mobile devices and real-time processing.

Innovation Solution

A method for training a speech enhancement neural network that incorporates speech style transfer capabilities by using a speech style transfer algorithm to generate target speech samples, allowing for noise reduction and speech style conversion, which is then trained using knowledge distillation to reduce complexity and enable execution on hearing devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If speech style transfer algorithms are used to improve speech clarity, then speech intelligibility is improved, but computational power requirements increase and processing time increases

Engineering Contradiction:

Improvespeech intelligibilityVSAvoidcomputational power

Core Design Contradiction:

Measurement precisionVSPower

Solution Approach 1:

The speech enhancement task is divided into separate functional modules: a speech style transfer neural network for improving intelligibility and a noise reduction neural network for removing background noise. This segmentation allows each module to be optimized independently for computational efficiency while maintaining overall system performance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A style embedding layer is introduced as an intermediary component that captures speech style characteristics and transfers them between different speech samples. This intermediary enables the speech style transfer function to operate with reduced computational requirements by focusing on extracting and transferring only the essential style features rather than processing entire speech signals.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If speech style transfer algorithms are used to improve speech clarity, then speech intelligibility is improved, but processing time increases

Engineering Contradiction:

Improvespeech intelligibilityVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The speech style transfer neural network is pre-trained on large datasets to learn robust speech style representations. This preliminary training enables the network to perform style transfer operations with reduced processing time during actual speech enhancement tasks, as the complex learning has already been performed during the pre-training phase.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If complex speech enhancement algorithms are used to improve speech clarity, then speech intelligibility is improved, but device complexity increases

Engineering Contradiction:

Improvespeech intelligibilityVSAvoidalgorithm complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The speech enhancement system is designed to perform multiple functions using unified neural network architectures that can handle both speech style transfer and noise reduction tasks. This multi-functionality reduces overall system complexity by avoiding the need for separate specialized algorithms for each enhancement task.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Object-affected harmful factors

If noise reduction algorithms are used to remove background noise, then noise levels are reduced, but speech clarity is not improved due to lack of clarity in speech itself

Engineering Contradiction:

Improvenoise levelsVSAvoidspeech clarity

Core Design Contradiction:

Object-affected harmful factorsVSMeasurement precision

Solution Approach 1:

The system merges noise reduction functionality with speech style transfer capabilities in a unified speech enhancement framework. This combination allows the system to simultaneously remove background noise and improve speech clarity through style transfer, addressing both harmful noise factors and inherent speech clarity issues in an integrated manner.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS20250308514A1Method for training a speech enhancement neural network, speech enhancement neural network and hearing device therewith

Publication Date: 2025.10.02 SONOVA AG

AI summary

A method for training a speech enhancement neural network for being executed on a hearing device comprises: providing a speech enhancement neural network, providing a speech style transfer algorithm for converting speech samples with a first speech style into speech samples with a second speech style, obtaining at least one training data set and applying supervised training on the speech enhancement neural network. The speech enhancement neural network has a network audio input for receiving an input audio signal, one or more network layers for predicting an enhanced audio signal and/or a filter mask for filtering the input audio signal, and a network output for outputting the enhanced audio signal and/or the filter mask. The at least one training data set comprises a training input audio signal comprising a speech sample and a target speech sample.