Sensor Enhanced Speech Recognition Acoustic Model Adaptation

Solution Overview

Problem

Current speech recognition technologies face challenges in noisy environments, requiring predefined noise source locations and lacking the use of user and environmental knowledge, leading to inefficient and resource-intensive noise reduction.

Innovation Solution

The system utilizes visual and audio sensors to capture metadata about the user and environment, adapting acoustic models to enhance speech recognition by determining user actions and environmental conditions, ensuring optimal matching between audio conditions and acoustic models.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If current noise reduction technologies are used to separate user audio signals from environmental noises, then speech recognition accuracy can be improved, but the system requires predefined noise source locations and substantial time and resources to implement

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidsystem implementation complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by capturing visual content and extracting metadata about the environment and user before processing audio signals. This allows the system to pre-adapt acoustic models to match the specific environment and user characteristics, reducing the need for complex real-time noise source localization and preprocessing steps during actual speech recognition

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by adapting acoustic models based on metadata extracted from visual content and environmental conditions. Instead of using fixed acoustic models or requiring predefined noise source locations, the system dynamically adjusts model parameters to match the current environment and user characteristics, improving accuracy while reducing implementation complexity

Inventive Principle:

Principle #35Parameter changes

2Reliability

If current noise reduction technologies are used to counteract environmental noise effects, then speech recognition performance can be enhanced, but the system requires significant amounts of time and resources

Engineering Contradiction:

Improvespeech recognition performanceVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by capturing visual content and extracting metadata about the environment and user before processing audio signals. This allows the system to pre-adapt acoustic models to match the specific environment and user characteristics, reducing the need for complex real-time noise source localization and preprocessing steps during actual speech recognition

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses self-service by automatically extracting metadata from visual content and adapting acoustic models without requiring manual configuration or predefined noise source locations. The system serves itself by using the captured visual information to automatically adjust its processing parameters, reducing both time and resource requirements

Inventive Principle:

Principle #25Self-service

3Measurement precision

If current noise reduction technologies are used to separate audio signals, then speech recognition can be improved, but the system fails to use user and environmental knowledge effectively

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidenvironmental adaptation capability

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary actions by capturing visual content and extracting metadata about the environment and user before processing audio signals. This allows the system to pre-adapt acoustic models to match the specific environment and user characteristics, reducing the need for complex real-time noise source localization and preprocessing steps during actual speech recognition

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by adapting acoustic models based on metadata extracted from visual content and environmental conditions. Instead of using fixed acoustic models or requiring predefined noise source locations, the system dynamically adjusts model parameters to match the current environment and user characteristics, improving accuracy while reducing implementation complexity

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10083350B2Sensor enhanced speech recognition

Publication Date: 2018.09.25 AT&T INTELLECTUAL PROPERTY I L P

AI summary

A system for sensor enhanced speech recognition is disclosed. The system may obtain visual content or other content associated with a user and an environment of the user. Additionally, the system may obtain, from the visual content, metadata associated with the user and the environment of the user. The system may also include determining, based on the visual content and metadata, if the user is speaking. If the user is determined to be speaking, the system may obtain audio content associated with the user and the environment. The system may then adapt, based on the visual content, audio content, and metadata, one or more acoustic models that match the user and the environment. Once the one or more acoustic models are adapted and loaded, the system may enhance a speech recognition process or other process associated with the user.