Neural Transform Audio Feature Fusion for Speech Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing speech recognition systems face challenges in noisy environments due to the inability to effectively combine and filter out noise using current feature extraction methods, particularly in continuous speech recognition applications.

Innovation Solution

A neural network approach that combines robust micro-modulation features with cepstral features using a non-linear multilayer perceptron to enhance noise robustness and improve automatic speech recognition performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional feature extraction methods are used, then the system is simple to implement, but the speech recognition accuracy deteriorates in noisy environments

Engineering Contradiction:

Improvespeech recognition accuracy in noisy environmentsVSAvoidfeature extraction system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent combines two different feature extraction streams (cepstral features and micro-modulation features) into a unified feature representation. This merging allows the system to leverage the complementary strengths of both feature types, achieving robust noise suppression and significantly improved speech recognition accuracy in noisy environments while maintaining a manageable system architecture.

Inventive Principle:

Principle #5Merging (Combining)

2Reliability

If noise robust features are extracted, then speech recognition performance improves, but the feature extraction process becomes more complex

Engineering Contradiction:

Improvenoise robustnessVSAvoidfeature extraction process complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent divides the feature extraction process into two separate, parallel streams: one extracting cepstral features and another extracting micro-modulation features. Each stream processes the audio signal independently using specialized algorithms optimized for its feature type, thereby achieving noise robustness through diversified feature representation without overly complicating any single extraction pathway.

Inventive Principle:

Principle #1Segmentation

3Reliability

If micro-modulation features are used, then noise robustness improves, but the ability to capture formant frequency variations is lost

Engineering Contradiction:

Improvenoise robustnessVSAvoidformant frequency variations

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent merges cepstral features (which capture formant frequency variations) with micro-modulation features (which provide noise robustness) into a unified feature representation. This combination ensures that neither formant information nor noise robustness is lost, as each feature type compensates for the limitations of the other, achieving both goals simultaneously.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS10096318B2System and method of using neural transforms of robust audio features for speech processing

Publication Date: 2018.10.09 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A system and method for processing speech includes receiving a first information stream associated with speech, the first information stream comprising micro-modulation features and receiving a second information stream associated with the speech, the second information stream comprising features. The method includes combining, via a non-linear multilayer perceptron, the first information stream and the second information stream to yield a third information stream. The system performs automatic speech recognition on the third information stream. The third information stream can also be used for training HMMs.