Chained Neural Engine Write-Back to Reduce Data Congestion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems rely heavily on central processing units (CPUs) for operations, leading to significant bandwidth consumption and increased power usage, especially when performing extensive computing tasks like convolution neural networks (CNNs).
Innovation Solution
A neural processor circuit with a chained neural engine circuit write-back architecture, where neural engines are pipelined into chains, and output data is aggregated and written back into a buffer memory by a data processor circuit, reducing data congestion and evenly distributing heat across the neural processor circuit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-generated harmful factors
If neural engines are pipelined into chains with fewer chains than neural engines, then data congestion is reduced and heat is evenly distributed, but the number of chains is limited by the smaller of the first number and second number
Solution Approach 1:
The neural engine circuits are segmented into multiple chains, where each chain contains a subset of neural engines. This segmentation divides the data flow into separate paths, reducing data congestion at any single write-back point and distributing thermal load across multiple chains rather than concentrating it at a single location.
2Temperature
If neural engines are pipelined into chains with fewer chains than neural engines, then heat distribution is improved, but the number of chains is constrained by the minimum of available neural engines and channels
Solution Approach 1:
The neural engine circuits are segmented into multiple chains, where each chain contains a subset of neural engines. This segmentation divides the data flow into separate paths, reducing data congestion at any single write-back point and distributing thermal load across multiple chains rather than concentrating it at a single location.
3Device complexity
If a single data processor circuit aggregates data from all chains, then data aggregation is simplified, but data congestion increases before aggregation
Solution Approach 1:
The data aggregation process is segmented into two stages: first, individual chains aggregate their output data locally into intermediate buffers; second, the data processor circuit aggregates data from multiple chains. This segmentation prevents all chains from writing to a single location simultaneously, reducing data congestion during the aggregation process.
Solution Approach 2:
Intermediate buffers are introduced as intermediary storage elements between the neural engine chains and the final data processor circuit. These buffers receive output data from each chain and hold it temporarily, allowing the data processor circuit to aggregate data from multiple chains without creating a congestion bottleneck at a single write-back point.
Data Source
AI summary
Embodiments relate to a neural processor circuit that includes a first number of neural engine circuits, a second number of channels and a data processor circuit. The first number of neural engine circuits are pipelined into the second number of chains smaller than the first number. Each of the chains is configured to generate output data of a first size. Each of the channels is coupled to each of the chains and configured to transmit the output data from each of the neural engine circuits in the chains sequentially. The data processor circuit is coupled to the channels to receive the output data. The data processor circuit aggregates the output data of each of the chains into aggregated data of a second size larger than the first size and writes the aggregated data of the second size into a buffer memory of the data processor circuit.