Hardware Accelerator Circuit Architecture for High Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware accelerators struggle to achieve high data throughput, falling short of modern peripheral devices that operate at rates of millions of operations per second.
Innovation Solution
A circuit architecture is introduced that enhances data throughput for hardware accelerators by reducing transactional overhead, allowing for data rates matching high-speed input/output interfaces like NVMe, and achieving approximately one million operations per second.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional hardware accelerator architecture is used, then device complexity is reduced, but data throughput is insufficient to match modern peripheral devices
Solution Approach 1:
The hardware accelerator is divided into multiple independent compute units (CU0, CU1, CU2, etc.) that can process data packets in parallel. Each compute unit has its own descriptor cache and processing logic, enabling simultaneous handling of multiple data streams and achieving million OPS throughput while maintaining manageable complexity through modular design
Solution Approach 2:
The architecture introduces a new dimension of parallelism by implementing multiple descriptor caches and compute units that operate simultaneously on different data packets. This multi-dimensional processing approach enables the system to achieve high throughput by processing multiple operations in parallel rather than sequentially
2Productivity
If input batching is used to achieve high data rates, then throughput increases, but transactional overhead between CPU and accelerator increases
Solution Approach 1:
Descriptors are pre-loaded into dedicated descriptor caches within the hardware accelerator before processing begins. This preliminary action eliminates the need for continuous CPU intervention during data processing, as the compute units can independently access descriptors from their local caches, significantly reducing transactional overhead time while maintaining high data rates
Solution Approach 2:
Descriptor caches act as intermediary structures between the CPU and compute units, storing descriptor information locally within the accelerator. This intermediary layer reduces the frequency of CPU-accelerator transactions by allowing compute units to independently retrieve descriptors from local cache, thereby reducing transactional overhead while maintaining high throughput
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A hardware acceleration device can include a switch communicatively linked to a host central processing unit (CPU), an adapter coupled to the switch via a control bus, wherein the control bus is configured to convey addresses of descriptors from the host central CPU to the adapter, and a random-access memory (RAM) coupled to the switch through a data bus. The RAM is configured to store descriptors received from the host CPU via the data bus. The hardware acceleration device can include a compute unit coupled to the adapter and configured to perform operations specified by the descriptors. The adapter may be configured to retrieve the descriptors from the RAM via the data bus, provide arguments from the descriptors to the compute unit, and provide control signals to the compute unit to initiate the operations using the arguments.