BRDNN Acoustic Score Calculation for Incremental Speech Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Bidirectional recurrent deep neural networks (BRDNNs) are not suitable for incremental speech recognition as they require the entire speech input to calculate pronunciation probabilities, making real-time incremental decoding challenging.

Innovation Solution

A method and apparatus that extract audio frames into windows, add padding frames to both sides of the window, and calculate acoustic scores using a BRDNN-based acoustic model, allowing for incremental recognition by recalculating overlapping frames' scores using statistical methods like arithmetic or geometric mean based on adjacent window scores.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a bidirectional recurrent deep neural network (BRDNN) is used to calculate pronunciation probabilities considering bidirectional information, then speech recognition accuracy is improved, but the ability to perform incremental decoding is lost

Engineering Contradiction:
Improvespeech recognition accuracyVSAvoidincremental decoding capability
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent divides the speech signal into multiple windows and processes each window independently through the BRDNN. By segmenting the speech into manageable frames and processing them in sequence rather than requiring the entire speech signal at once, the system enables incremental decoding while maintaining the accuracy benefits of bidirectional processing within each window context.

Inventive Principle:
Principle #1Segmentation

2Loss of information

If the entire speech is provided as input to BRDNN for calculating pronunciation probabilities, then accurate bidirectional context is obtained, but real-time incremental recognition becomes impossible

Engineering Contradiction:
Improvebidirectional context informationVSAvoidreal-time recognition capability
Core Design Contradiction:
Loss of informationVSLoss of time

Solution Approach 1:

The patent applies padding frames to both sides of each speech window before processing. This preliminary action provides the BRDNN with bidirectional context information in advance for each window, allowing the network to make accurate pronunciation probability calculations without waiting for the entire speech signal. The padding frames contain pre-calculated acoustic scores that provide contextual information about surrounding speech segments.

Inventive Principle:
Principle #10Preliminary action

3Reliability

If padding frames are added to both sides of the window for BRDNN processing, then bidirectional context is provided, but computational complexity increases

Engineering Contradiction:
Improvecontext accuracyVSAvoidcomputational complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent recalculates acoustic scores for overlapping padding frames by combining results from adjacent windows using statistical methods. This approach recovers computational efficiency by reusing previously calculated scores for overlapping regions rather than recalculating them independently, reducing redundant computations while maintaining context accuracy.

Inventive Principle:
Principle #34Discarding and recovering

4Stability of the object's composition

If acoustic scores are recalculated for overlapping frames using statistical methods, then consistency between adjacent windows is improved, but processing time increases

Engineering Contradiction:
Improvescore consistencyVSAvoidprocessing speed
Core Design Contradiction:
Stability of the object's compositionVSProductivity

Solution Approach 1:

The patent uses statistical methods (arithmetic mean or geometric mean) to combine acoustic scores from overlapping regions of adjacent windows. This copying and averaging approach ensures consistency across window boundaries by using the same statistical operation for all overlapping frames, maintaining score reliability while enabling parallel processing of non-overlapping regions.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS10714077B2Apparatus and method of acoustic score calculation and speech recognition using deep neural networks
Publication Date: 2020.07.14 SAMSUNG ELECTRONICS CO LTD
  • US10714077B2 patent drawing
  • US10714077B2 patent drawing
  • US10714077B2 patent drawing

AI summary

An apparatus for calculating acoustic score, a method of calculating acoustic score, an apparatus for speech recognition, a method of speech recognition, and an electronic device including the same are provided. An apparatus for calculating acoustic score includes a preprocessor configured to sequentially extract audio frames into windows and a score calculator configured to calculate an acoustic score of a window by using a deep neural network (DNN)-based acoustic model.