Adaptive Buffer Sharing Across Multi-Core Stream Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processing is bottlenecked by memory constraints and redundant data copying, leading to inefficient hardware utilization and energy waste in existing neural processing units (NPUs).
Innovation Solution
Implementing a hardware accelerator with shared buffers between processing cores via dedicated output stream links or virtual channels, allowing flexible buffer sharing and reducing redundant data copies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processing cores work in parallel to increase throughput, then processing speed is improved, but data redundancy in each core's buffer wastes energy and memory usage
Solution Approach 1:
The patent merges buffer resources across multiple processing cores by implementing shared buffer memory that can be accessed by multiple cores. Instead of each core having its own dedicated buffer, the buffer is shared among cores, eliminating redundant data storage and reducing energy consumption while maintaining parallel processing throughput.
Solution Approach 2:
The buffer is designed to serve multiple functions and multiple processing cores simultaneously. A single buffer can be accessed by different cores for different operations, making the memory resource universal rather than dedicated to a single core, thereby reducing overall memory requirements and energy usage.
2Productivity
If multiple processing cores work in parallel to increase throughput, then processing speed is improved, but hardware utilization efficiency deteriorates
Solution Approach 1:
The patent introduces a buffer management unit as an intermediary component that coordinates access to the shared buffer among multiple processing cores. This mediator manages buffer allocation, access rights, and data consistency, enabling efficient parallel processing without requiring complex point-to-point interconnections between each core and buffer, thus simplifying the overall hardware architecture.
3Speed
If fixed-size memories are equipped in NPU processing cores, then fast memory access is achieved, but adaptability to different neural network layer constraints deteriorates
Solution Approach 1:
The patent implements dynamic buffer configuration where the buffer size and allocation can be adjusted at runtime based on the specific requirements of different neural network layers. Instead of fixed-size memories, the system can dynamically allocate buffer resources to match the computational needs of each layer, maintaining fast access speeds while adapting to varying constraints.
Solution Approach 2:
The buffer parameters such as size, allocation, and access patterns can be changed based on the specific neural network layer being processed. This allows the memory subsystem to adapt its characteristics to match the requirements of different layers, improving both speed and adaptability.
Data Source
AI summary
A hardware accelerator includes a plurality of functional circuits, a stream switch, and a plurality of neural network processing cores coupled to the plurality of functional circuits via the stream switch to stream data to and from functional circuits of the plurality of functional circuits, wherein the neural network processing cores include at least one sender core having a buffer whose buffer content is sharable with at least one receiver core of the neural network processing cores via at least one of a dedicated output stream link or dedicated virtual channel on a pre-existing output stream link, and wherein the at least one of the dedicated output stream link or dedicated virtual channel is dedicated to sharing the buffer content via the stream switch.