Neural Network Acceleration Processor Submatrix Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for performing submatrix operations in multilayer neural networks, such as using general-purpose processors or GPUs, face performance bottlenecks due to low single-processor performance and limited on-chip caching, leading to high power consumption and inefficient data access.
Innovation Solution
A neural network acceleration processor is designed with a controller unit and computation module that receives submatrix instructions including a starting address, width, height, and stride, allowing for efficient selection and operation on submatrices within a matrix, utilizing on-chip caching to improve performance and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a general-purpose processor is used to perform submatrix operations, then the system has high flexibility and ease of operation, but the processing performance is low and cannot meet performance requirements for large submatrices
Solution Approach 1:
The patent divides the processing system into multiple parallel processing units that can simultaneously handle different submatrix operations. Each processing unit is specialized for specific computational tasks, allowing the system to process large submatrices by segmenting them into smaller blocks that can be handled in parallel, thereby achieving both high performance and maintained flexibility through programmable control.
2Productivity
If a GPU is used to perform submatrix operations, then processing performance improves through parallel computation, but on-chip caching is limited causing repeated data movement from off-chip which increases power consumption and creates bandwidth bottlenecks
Solution Approach 1:
The patent implements on-chip caching mechanisms that pre-load and store frequently accessed submatrix data before computational operations are performed. This preliminary action of data preparation and caching on-chip eliminates the need for repeated off-chip data movements during computation, thereby reducing power consumption and bandwidth bottlenecks while maintaining high processing performance.
3Productivity
If a GPU is used to perform submatrix operations, then processing performance improves through parallel computation, but repeated data movement from off-chip creates bandwidth bottlenecks
Solution Approach 1:
The patent introduces on-chip caching memory as an intermediary layer between the processing units and off-chip memory. This intermediary caches frequently accessed submatrix data on-chip, allowing processing units to access data at high speeds without repeatedly querying off-chip memory, thereby eliminating bandwidth bottlenecks while maintaining fast data access speeds for computational operations.
Data Source
AI summary
Aspects for submatrix operations in neural network are described herein. The aspects may include a controller unit configured to receive a submatrix instruction. The submatrix instruction may include a starting address of a submatrix of a matrix, a width of the submatrix, a height of the submatrix, and a stride that indicates a position of the submatrix relative to the matrix. The aspects may further include a computation module configured to select one or more values from the matrix as elements of the submatrix in accordance with the starting address of the matrix, the starting address of the submatrix, the width of the submatrix, the height of the submatrix, and the stride.