FPGA Hardware Accelerator for Compressed RNN Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional general-purpose CPUs and GPUs are inefficient in processing and memory usage for large-scale neural networks, limiting further development and requiring compression to reduce computation and memory consumption, while existing solutions fail to fully exploit sparsity in neural networks.
Innovation Solution
A hardware accelerator with a parallelized pipeline design and Compressed Column Storage (CCS) format for sparse neural networks, implemented on an embedded FPGA, which initializes data on-chip, performs matrix-vector multiplication, and iteratively computes activation and output sequences, utilizing processing elements, ALUs, and buffers to optimize computation and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional CPUs or GPUs are used to process large-scale neural networks, then the network can be implemented with general-purpose hardware, but processing efficiency and memory usage become inefficient
Solution Approach 1:
The system divides the neural network processing into multiple processing elements (PEs), each handling specific matrix-vector multiplication operations. The weight matrix W is partitioned across multiple PEs, with each PE processing a subset of rows. This segmentation enables parallel computation while reducing the processing burden on individual units, thereby improving overall processing efficiency.
Solution Approach 2:
The patent transitions from general-purpose sequential processing to specialized parallel processing architecture. By introducing a dimension of parallelism through multiple PEs operating simultaneously, the system achieves higher throughput. Additionally, the compression format transforms the data representation from dense to sparse, exploiting the inherent structure of neural network weights to reduce computational complexity.
2Quantity of substance
If neural networks are compressed to reduce computation and memory consumption, then resource usage decreases, but existing solutions fail to fully exploit sparsity in neural networks
Solution Approach 1:
The system applies different processing strategies to different parts of the neural network based on their sparsity characteristics. Each processing element is configured to handle specific portions of the weight matrix with locally optimized data formats. The compression format preserves the local sparsity patterns by storing only non-zero elements with their indices, allowing each PE to efficiently process its assigned subset without being burdened by zero elements.
Solution Approach 2:
The patent changes the representation parameters of the weight matrix from dense floating-point format to a sparse compressed format that stores only non-zero elements along with their row and column indices. This parameter change reduces memory consumption while the specialized hardware architecture maintains computation efficiency by directly processing the compressed format without requiring full decompression.
3Productivity
If a parallelized pipeline design is implemented on FPGA, then computational efficiency improves, but I/O bandwidth and computational resources must be balanced
Solution Approach 1:
The system performs preliminary data preparation by organizing the compressed weight matrix data into a format suitable for parallel processing before computation begins. The weight matrix is pre-partitioned and loaded into on-chip memory of multiple PEs in advance, reducing the need for complex data transfer operations during computation. This preliminary organization simplifies the I/O bandwidth requirements during the actual computation phase.
Solution Approach 2:
The parallelized pipeline design ensures continuous utilization of computational resources by maintaining a steady flow of data through the processing elements. Multiple PEs operate simultaneously on different portions of the matrix, keeping the computational pipeline full and avoiding idle cycles. This continuous operation maximizes the utilization of FPGA resources while maintaining balanced I/O bandwidth requirements.
Data Source
AI summary
The present invention relates to recurrent neural network. In particular, the present invention relates to how to implement and accelerate a recurrent neural network based on an embedded FPGA. Specifically, it proposes an overall design processing method of matrix decoding, matrix-vector multiplication, vector accumulation and activation function. In another aspect, the present invention proposes an overall hardware design to implement and accelerate the above process.