ASIC Tile Architecture for Parallel Neural Network Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for neural networks, particularly convolutional neural networks, face inefficiencies in computation due to software-based methods that perform dot products and activation functions sequentially, leading to increased latency and reduced bandwidth in processing large datasets.
Innovation Solution
The implementation of a tile-based architecture within an application-specific integrated circuit (ASIC) that includes a computational array, local memory, and controllable bus lines, allowing for parallel processing and data transfer between tiles, thereby reducing latency and increasing bandwidth by enabling simultaneous computation and data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If software-based methods are used to perform dot products and activation functions sequentially, then ease of implementation is improved, but processing speed and bandwidth deteriorate
Solution Approach 1:
The patent replaces software-based sequential processing with hardware-based parallel processing using computational arrays. Each cell in the computational array performs dot product operations simultaneously, substituting the mechanical/software execution model with a hardware parallel computation model, thereby dramatically increasing processing speed while maintaining ease of implementation through fixed hardware architecture
Solution Approach 2:
The computational array is segmented into multiple independent cells that can operate in parallel. Each cell processes a portion of the input data independently, allowing simultaneous computation of multiple dot products. This segmentation enables the system to achieve high processing speeds while keeping each individual cell simple to implement
2Device complexity
If software-based sequential processing is used, then device complexity is reduced, but latency increases
Solution Approach 1:
The processing pipeline is segmented into distinct stages: data input, computational array processing, local memory storage, and output. This segmentation allows different stages to operate simultaneously and independently, reducing overall latency while keeping each stage relatively simple in design
Solution Approach 2:
The local memory is positioned between the computational array and output stages to perform preliminary data storage and buffering. This preliminary action allows the computational array to continue processing while data is being prepared for output, overlapping computation and data transfer operations to reduce latency
3Productivity
If parallel processing architecture is implemented, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
Each cell in the computational array is designed with identical local functionality, performing the same dot product operation on locally received data. This local quality uniformity simplifies the overall design while enabling parallel processing, as each cell is a simple, replicated unit rather than a complex heterogeneous system
Solution Approach 2:
The computational array cells are designed as universal units that can process any input data through the same dot product operation. This universality allows the same hardware structure to handle different computational tasks by simply changing the input data, maintaining simplicity while achieving high processing efficiency through parallel operation
Data Source
AI summary
A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.