Neural Network Hardware Accelerator Data Parallelism via Segmented Memory Banks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network hardware accelerators with multiple processors and memory banks face inefficiencies due to sequential access and locking mechanisms, leading to delays in inference tasks as only one processor can read or write to a memory bank at a time, limiting parallel processing capabilities.
Innovation Solution
Implementing a neural network hardware accelerator that allows each processor to record resultant values to multiple memory banks simultaneously and read values from multiple banks concurrently, utilizing an external memory interface and data bus to facilitate data parallelism, reducing waiting times and enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sequential access and locking mechanisms are used in neural network hardware accelerators with multiple processors and memory banks, then data integrity is maintained, but processing speed and parallel processing capabilities are limited
Solution Approach 1:
The patent divides the memory system into multiple independent memory banks, each capable of being accessed by multiple processors simultaneously. This segmentation eliminates the need for sequential access and locking mechanisms, as each memory bank operates independently, thereby maintaining data integrity while significantly improving processing speed and parallel processing capabilities.
2Reliability
If only one processor can read or write to a memory bank at a time, then data corruption is prevented, but inference task completion time increases
Solution Approach 1:
The patent segments the memory system into multiple independent memory banks, allowing multiple processors to simultaneously read from or write to different memory banks without data corruption. This parallel access capability dramatically reduces inference task completion time while maintaining data integrity through independent bank operations.
Solution Approach 2:
The patent introduces a new dimension of parallelism by organizing memory access along the memory bank dimension rather than sequentially along the processor dimension. Multiple processors can access multiple memory banks simultaneously, transforming the access pattern from sequential (one processor at a time) to parallel (multiple processors at once), thereby reducing completion time without compromising reliability.
3Productivity
If multiple processors access the same memory bank simultaneously, then processing parallelism is improved, but data corruption risk increases
Solution Approach 1:
The patent segments the memory system into multiple independent memory banks, each capable of handling simultaneous access from multiple processors. This segmentation allows processing parallelism to be improved by distributing access across multiple banks, while data corruption risk is eliminated because each bank operates independently with its own access control, preventing interference between simultaneous operations.
Data Source
AI summary
Neural network hardware acceleration data parallelism is performed by an integrated circuit including a plurality of memory banks, each memory bank among the plurality of memory banks configured to store values and to transmit stored values, a plurality of computation units, each computation unit among the plurality of computation units including one of a channel pipeline and a multiply-and-accumulate (MAC) element configured to perform a mathematical operation on an input data value and a weight value to produce a resultant data value, and a computation controller configured to cause a value transmission to be received by more than one computation unit or memory bank.