Neural Network Accelerator Parallel Processing and Weight Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems are too slow for real-time applications and consume excessive power, making them unsuitable for low-power devices like hearing aids, which require fast and efficient speech recognition.
Innovation Solution
A neural network accelerator that shares resources with a host processor, performs computations in parallel, and uses weight pruning and compression to reduce memory and power consumption, allowing for efficient processing of neural networks in hardware or software.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a processor or DSP is used to perform neural network computations, then the recognition function can be implemented, but the processing speed is too slow for real-time applications
Solution Approach 1:
The patent divides the neural network processing into distinct hardware components: a host processor for control and a dedicated accelerator for computations. The accelerator is further segmented into multiple MAC units that can process different neurons simultaneously, enabling parallel computation to achieve real-time processing speeds.
Solution Approach 2:
The patent introduces a dedicated neural network accelerator as an intermediary component between the host processor and memory. This accelerator includes specialized hardware (MAC units, lookup tables, circular buffers) that offload computation-intensive neural network operations from the general-purpose processor, enabling real-time processing without burdening the host CPU.
2Productivity
If a DSP is used for neural network processing, then real-time processing is achieved, but power consumption is too high for low-power applications
Solution Approach 1:
The patent implements local quality by creating a specialized accelerator with components optimized for specific neural network operations. The MAC units are designed for efficient multiplication and accumulation, lookup tables store pre-computed activation function values, and circular buffers manage data flow locally within the accelerator, reducing the need for high-power general-purpose processing.
Solution Approach 2:
The patent extracts the computation-intensive neural network operations from the power-consuming DSP and places them in a dedicated accelerator. This separation allows the main processor to enter low-power states while the accelerator handles real-time processing with lower overall power consumption, as the accelerator can be optimized for energy-efficient specialized operations.
3Measurement precision
If full precision weights are stored in memory, then computation accuracy is maintained, but memory usage and power consumption increase
Solution Approach 1:
The patent changes the precision parameter of weight storage by implementing a hybrid approach: full-precision weights are kept in memory for accuracy-critical operations, while lower-precision representations are used in the accelerator's registers and computation units. Lookup tables store activation function values at optimized precision levels, balancing memory usage with computational accuracy requirements.
Data Source
AI summary
A neural network implementation is disclosed. The implementation allows the computations for the neural network to be performed on either an accelerator or a processor. The accelerator and the processor share a memory and communicate over a bus to perform the computations and to share data. The implementation uses weight compression and pruning, as well as parallel processing, to reduce computing, storage, and power requirements.