Mobile Speech Recognition Hardware Accelerator Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mobile speech recognition applications face challenges due to limited CPU capabilities in traditional mobile devices, leading to impractical client-side processing and high latency in server-side implementations, especially with inconsistent connection quality and background noise.

Innovation Solution

A mobile speech recognition software application utilizing a client-side implementation of a neural network with a hardware accelerator, such as an ASIC or FPGA, to perform computationally-intensive neural network calculations, including a multiplier-accumulator unit for matrix multiplication and activation functions, and data transceiver circuitry for decompressing and transmitting weight and bias terms.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If client-side processing of speech recognition algorithms is implemented on traditional mobile devices, then processing speed and accuracy are improved, but the limited CPU capability makes the implementation impractical

Engineering Contradiction:

Improvespeech recognition processing speedVSAvoidCPU capability requirement

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system segments the speech recognition processing into two parts: feature extraction is performed on the mobile device CPU, while the computationally intensive neural network acoustic model processing is offloaded to a dedicated hardware accelerator. This segmentation allows the mobile device to achieve high processing speed without requiring its CPU to handle all computational loads alone.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A hardware accelerator serves as an intermediary component between the mobile device CPU and the neural network processing requirements. This intermediary device provides the necessary computational power for acoustic model processing while keeping the mobile device CPU lightweight, thus resolving the contradiction between processing capability and device complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If server-side processing is used for speech recognition, then device capability requirements are reduced, but latency increases due to network transmission and processing delays

Engineering Contradiction:

Improvedevice processing requirementVSAvoidspeech recognition latency

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The system performs preliminary feature extraction and preparation of audio data on the mobile device before transmission to the hardware accelerator. This preliminary action reduces the amount of data that needs to be transmitted and processed, thereby reducing latency while maintaining the benefit of distributed processing architecture.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The hardware accelerator acts as an intermediary that processes speech recognition locally on the device, eliminating the need for round-trip network communication. This local processing intermediary significantly reduces latency compared to pure server-side processing while keeping device complexity manageable through specialized hardware support.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Loss of time

If neural network calculations are performed locally on mobile devices, then latency is reduced, but the computational intensity exceeds the capability of traditional mobile CPUs

Engineering Contradiction:

Improveprocessing latencyVSAvoidcomputational power requirement

Core Design Contradiction:

Loss of timeVSPower

Solution Approach 1:

The computational workload is segmented between the mobile device CPU (for feature extraction) and a dedicated hardware accelerator (for neural network acoustic model processing). This segmentation enables low-latency local processing with high computational power without requiring the mobile CPU to handle all computational demands alone.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A hardware accelerator intermediary is introduced to provide the necessary computational power for neural network processing locally on the device. This intermediary component enables high-performance local processing with reduced latency while keeping the mobile device architecture manageable through specialized hardware support.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Quantity of substance

If compressed weight and bias terms are used in the neural network, then data transmission efficiency is improved, but decompression complexity increases

Engineering Contradiction:

Improvedata transmission sizeVSAvoiddecompression processing complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The weight and bias terms are pre-compressed into a compact format before being loaded into the hardware accelerator. This preliminary compression reduces the data transmission size and storage requirements, while the decompression is performed efficiently during the neural network processing sequence without adding significant complexity to the overall system.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9153230B2Mobile speech recognition hardware accelerator

Publication Date: 2015.10.06 GOOGLE LLC

AI summary

A method for executing a mobile speech recognition software application based on a multi-layer neural network model includes providing to a hardware accelerator in the mobile device to classify one or more frames of an audio signal. The hardware accelerator includes a multiplier-accumulator (MAC) unit to perform matrix multiplication operations involved in computing the neural network output.