Parallel Node Processors Decompose Matrix Multiplications for High-Throughput DNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural networks face limitations in data throughput due to high computational complexity, especially when dealing with large arrays of input data or numerous layers, leading to reduced accuracy in output data when implemented in fixed-point hardware.
Innovation Solution
A hardware-based deep neural network architecture utilizing parallel node processors that decompose large-scale matrix multiplications into smaller sub-matrix multiplications, reducing hardware complexity and enabling high-throughput processing by storing and processing node values in sub-vectors and sub-matrices within dedicated memories, eliminating the need for external memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed-point hardware architecture is used to improve data throughput, then processing speed increases, but resolution loss occurs leading to reduced output accuracy
Solution Approach 1:
The patent divides the deep neural network computation into multiple processing stages with intermediate rounding. Each stage processes a portion of the computation and rounds results to fixed-point representation, enabling hardware implementation while managing precision loss through controlled segmentation of the computational pipeline
Solution Approach 2:
The patent employs dynamic parameter adjustment by changing the precision parameters (number of bits) at different stages of the neural network. Intermediate layers use lower precision to save hardware resources, while input and output layers maintain higher precision to preserve accuracy, effectively trading precision at different computational stages
2Adaptability or versatility
If software implementation on CPU or GPU is used to maintain flexibility, then adaptability improves, but data throughput becomes limited
Solution Approach 1:
The patent creates a universal fixed-point processor architecture that can handle different neural network configurations and operations. The same hardware structure performs matrix multiplications, activations, and other neural network operations through configurable parameters, providing both hardware speed and software-like adaptability
Solution Approach 2:
The system achieves adaptability through parameter configuration rather than hardware reconfiguration. Different neural network models are implemented by changing computational parameters (weights, biases, activation functions) in the fixed-point processor, maintaining hardware throughput while enabling model flexibility
3Quantity of substance
If large arrays of input data or numerous layers are processed to improve model capability, then computational power increases, but computational complexity increases dramatically
Solution Approach 1:
The patent segments large matrix multiplications into smaller block-wise operations that can be processed in fixed-point arithmetic. By dividing large computations into manageable blocks with intermediate rounding, the system handles large data arrays and deep networks without requiring exponentially increasing hardware complexity
Solution Approach 2:
The patent uses lower-precision fixed-point arithmetic as a computationally inexpensive alternative to high-precision floating-point operations. Each fixed-point operation is cheaper and faster, allowing the system to process larger datasets through many inexpensive operations rather than fewer expensive high-precision operations
Data Source
AI summary
This disclosure describes methods and systems for high-throughput computations in a fully-connected deep neural network. Specifically, a hardware-based deep neural network architecture including a set of parallel node processors is used to process node value transition between layers of the deep neural network, which usually involves a large-scale matrix multiplication. The set of parallel node processors are configured to decompose the large-scale matrix multiplication into sub-matrix multiplications with smaller sizes and thus reducing the hardware-complexity and making feasible direct implementation in hardware. With this implementation deep neural network may achieve a very high throughput and can handle a large number of processing layers.