Multi-Die Neural Network Accelerator Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing neural network accelerators on multiple FPGA dies is challenging due to interconnect bandwidth limitations and complex routing issues, which hinder achieving desired performance levels in deep learning applications.
Innovation Solution
A neural network processing system with a host computer system and multiple RAMs coupled to neural network accelerators, where the host writes input data and work requests to the RAMs, specifying operations and memory locations, allowing each accelerator to perform operations independently and efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If neural network accelerators are implemented on multiple FPGA dies, then device complexity is reduced and manufacturing becomes easier, but interconnect bandwidth is insufficient and routing becomes problematic
Solution Approach 1:
The system divides the neural network processing function into multiple independent accelerators distributed across different FPGA dies. Each accelerator processes a specific subset of operations independently, eliminating the need for complex high-bandwidth interconnects between dies while maintaining overall system functionality through modular segmentation of the computational workload.
Solution Approach 2:
The patent introduces an intermediary memory structure and control mechanism that coordinates between multiple accelerators on separate dies. This intermediary layer manages data flow and synchronization without requiring direct high-bandwidth interconnects, thus resolving the bandwidth limitation while enabling multi-die implementation.
2Adaptability or versatility
If multiple FPGA dies are used for accelerators, then device scalability is improved, but inter-die routing limitations prevent achieving desired performance
Solution Approach 1:
The computational workload is segmented into independent processing units that can be distributed across multiple dies. Each unit processes discrete neural network operations without requiring frequent high-speed communication with other units, thus maintaining high throughput while enabling scalable multi-die configurations.
Solution Approach 2:
The patent transitions from a single-die architecture to a multi-die architecture, adding the spatial dimension of die distribution. This dimensional change allows the system to scale beyond the physical constraints of a single die while maintaining performance through careful architectural design that minimizes inter-die communication requirements.
3Device complexity
If accelerators share common memory, then device complexity is reduced, but access contention increases and processing efficiency decreases
Solution Approach 1:
The memory architecture is segmented into dedicated memory spaces for each accelerator, eliminating access contention while maintaining architectural simplicity. Each accelerator has its own allocated memory region, allowing parallel access without interference, thus preserving high processing efficiency while keeping the overall memory structure manageable.
Data Source
AI summary
A disclosed neural network processing system includes a host computer system, a RAMs coupled to the host computer system, and neural network accelerators coupled to the RAMs, respectively. The host computer system is configured with software that when executed causes the host computer system to write input data and work requests to the RAMS. Each work request specifies a subset of neural network operations to perform and memory locations in a RAM of the input data and parameters. A graph of dependencies among neural network operations is built and additional dependencies added. The operations are partitioned into coarse grain tasks and fine grain subtasks for optimal scheduling for parallel execution. The subtasks are scheduled to accelerator kernels of matching capabilities. Each neural network accelerator is configured to read a work request from the respective RAM and perform the subset of neural network operations on the input data using the parameters.