Distributed Neural Network Chip Processing With Tile Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network chips in devices with limited power supplies, such as ear-worn devices, face challenges in achieving tolerable latencies and efficient processing of matrix-vector operations due to varying completion times of tiles performing sub-operations.
Innovation Solution
A distributed processing scheme across tiles of a neural network chip, where each tile generates data and sends control signals to the preceding tile upon completion, allowing for efficient combination of results through routing circuitry and accumulation, reducing power consumption by storing weights locally and minimizing external memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tiles perform sub-operations independently and finish at different times, then processing parallelism is improved, but coordination complexity and latency increase
Solution Approach 1:
The system performs preliminary actions by having each tile generate its data independently before coordination is needed. The control signal mechanism is pre-established to trigger data transmission only when tiles are ready, avoiding unnecessary coordination overhead and reducing complexity while maintaining parallel processing benefits
Solution Approach 2:
The control signal from subsequent tiles to preceding tiles creates a feedback mechanism that synchronizes data transmission timing. This feedback loop ensures data is only transmitted when both sender and receiver are ready, reducing coordination complexity while maintaining high processing throughput
2Measurement precision
If data is combined at each tile, then processing accuracy is improved, but energy consumption increases
Solution Approach 1:
The system merges the data combination operation into the tile architecture itself, where each tile combines received data with its own generated data locally. This merging approach maintains processing accuracy by ensuring all tile results are aggregated while reducing energy consumption by eliminating the need for external memory access and centralized combination operations
3Adaptability or versatility
If external memory is used for weight storage, then device flexibility is improved, but power consumption and latency increase
Solution Approach 1:
The weight storage function is extracted from external memory and integrated directly into each tile's local memory. This extraction eliminates the need for continuous external memory access during processing, significantly reducing power consumption and latency while maintaining the flexibility to handle different neural network configurations through programmable tile connectivity
Data Source
AI summary
A neural network chip may include a plurality of tiles including a first tile and a second tile. The first tile may be configured to generate first data at least in part by performing first multiply-accumulate operations. The second tile may be configured to generate second data at least in part by performing second multiply-accumulate operations. The second tile may be configured to transmit a control signal to the first tile when the second data has been generated. The first tile may be configured to transmit the first data to the second tile when the control signal has been received and the first data has been generated. The second tile may be configured to combine the second data generated by the second tile with the first data received from the first tile to produce combined first data and second data.