Neural Processor Circular Buffering for Overlap Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems, such as neural networks, rely heavily on central processing units (CPUs) for operations, leading to significant bandwidth consumption and increased power usage due to inefficient data handling and memory access.
Innovation Solution
Implementing circular buffering in a neural processor circuit with data processor and memory access operations, utilizing addressing parameters to control data flow and wrap data around circular buffers, reducing unnecessary copying and re-computation of overlapping regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU and main memory are used to instantiate and execute machine learning systems, then ease of configuration is improved, but bandwidth consumption and power consumption increase
Solution Approach 1:
The system is segmented into a CPU for configuration management and a dedicated neural processing unit for execution. The neural processing unit further segments functionality into multiple neural engines, each with dedicated resources (weights, activations, compute units), allowing parallel processing while reducing CPU involvement in actual computation, thereby lowering power consumption during execution.
Solution Approach 2:
A memory interface and data flow controller act as intermediaries between the CPU and neural processing unit. These intermediaries manage data transfer efficiently, preprocessing data before it reaches the neural engines and managing memory access patterns to minimize bandwidth consumption while maintaining ease of configuration through CPU control.
2Productivity
If circular buffering is implemented to optimize data flow, then data handling efficiency is improved, but device complexity increases
Solution Approach 1:
The circular buffer implementation uses dynamic addressing parameters that can be configured for different neural engine requirements. The flow control circuit dynamically adjusts buffer pointers and addressing modes based on the specific neural network layer being processed, allowing efficient data reuse without requiring hardwired complex control logic for each possible scenario.
Solution Approach 2:
The circular buffer structure serves multiple functions: it acts as an activation buffer for current layer computations, a weight buffer for parameter storage, and an intermediate storage for data reuse across different neural engines. This multi-functionality reduces the need for separate dedicated buffers for each purpose, simplifying the overall device architecture while maintaining high data handling efficiency.
Data Source
AI summary
Embodiments of the present disclosure relate to circular buffers in a neural processor circuit. The neural processor circuit includes multiple neural engine circuits and a data processor circuit coupled to at least one of the neural engine circuits. The at least one neural engine circuit performs at least convolution operations. The data processor circuit includes a circular buffer, and a flow control circuit coupled to the circular buffer. The flow control circuit generates at least one addressing parameter that defines wrapping of data in the circular buffer. The circular buffer controls data flow in the neural processor circuit by storing first data associated with the at least one neural engine circuit so that the first data is wrapped around in the circular buffer. An addressing layout of the first data wrapped around in the circular buffer is defined by the at least one addressing parameter.