Parallel WFST Token Propagation for Low-Latency ASR

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Automatic speech recognition (ASR) systems on small devices face challenges due to computational complexity and power constraints, leading to delayed and low-quality word recognition, as existing solutions like GPU-based decoding are not feasible for all devices and SIMD parallelization is inefficient due to conditional branching and scattered memory access.

Innovation Solution

The implementation of two-part decoding using parallel processing of equivalent sub-graphs within a Weighted Finite State Transducer (WFST) framework, where sub-graphs are updated in parallel using SIMD instructions like SSE or AVX, allowing for efficient token propagation without altering the WFST structure, thereby enabling ASR on smaller platforms with reduced power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If server-based ASR computations are performed remotely, then word recognition quality is improved, but response delay increases

Engineering Contradiction:

Improveword recognition qualityVSAvoidresponse delay

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The ASR system is segmented into client-side components (acoustic model, feature extraction) and server-side components (language model, vocabulary), allowing critical path computations to occur locally while maintaining recognition quality through distributed processing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from a single-dimension server-client model to a multi-dimensional hybrid architecture where different computational tasks are distributed across different dimensions (local device resources vs. remote server resources), enabling simultaneous local preprocessing and remote language modeling

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Loss of time

If on-board ASR computation is implemented on small devices, then response delay is reduced, but power consumption increases

Engineering Contradiction:

Improveresponse delayVSAvoidpower consumption

Core Design Contradiction:

Loss of timeVSUse of energy by moving object

Solution Approach 1:

The device performs partial ASR computations locally (acoustic scoring, feature extraction) rather than complete processing, sending only intermediate results to the server for language modeling and final decoding, thus reducing local power consumption while maintaining fast response

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system dynamically adjusts computation parameters such as acoustic model complexity, vocabulary size, and decoding depth based on available power resources and performance requirements, optimizing the balance between speed and energy consumption

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If traditional sequential WFST decoding is used, then implementation simplicity is maintained, but processing speed decreases

Engineering Contradiction:

Improveimplementation simplicityVSAvoidprocessing speed

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The WFST decoding process is segmented into independent parallelizable operations (token propagation, score computation, path exploration) that can be executed concurrently using multi-threading and vectorization techniques while maintaining the overall decoding logic

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple decoding paths and candidate hypotheses are merged and processed simultaneously using beam search with parallel score computation, combining the exploration of multiple hypotheses with efficient resource utilization to increase processing throughput

Inventive Principle:

Principle #5Merging (Combining)

4Productivity

If GPU-based decoding is implemented, then processing speed is improved, but device compatibility and power efficiency worsen

Engineering Contradiction:

Improveprocessing speedVSAvoiddevice compatibility

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent implements a universal decoding framework that can operate on various hardware platforms (CPU, GPU, FPGA, mobile processors) by abstracting computation kernels and providing platform-specific optimizations, making the system adaptable to different device types without requiring GPU hardware

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system replaces specialized GPU hardware acceleration with software-based parallel processing techniques (SIMD instructions, multi-threading, vectorized operations) that can run on general-purpose processors, eliminating the need for dedicated graphics hardware while maintaining improved processing speed

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10255911B2System and method of automatic speech recognition using parallel processing for weighted finite state transducer-based speech decoding

Publication Date: 2019.04.09 INTEL CORP

AI summary

A computer-implemented method of speech recognition comprises forming a weighted finite state transducer (WFST) having nodes associated with states and interconnected by arcs, and to identify at least one word or word sequence hypothesis, identifying multiple sub-graphs on the WFST, each sub-graph having the same arrangement of multiple states and at least one arc, and propagating tokens in parallel through the sub-graphs, where each sub-graph is stored as a supertoken each having an array of tokens.