Tile-Based ASIC Accelerator Layout for Low-Latency Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for computing in neural networks, particularly convolutional neural networks, face inefficiencies due to software-based computations that result in high latency and reduced bandwidth, as they rely on software to perform dot products and activations across multiple dimensions, which can lead to increased operational times and data path lengths.
Innovation Solution
The implementation of an application-specific integrated circuit (ASIC) with a tile-based architecture that includes a computational array, local memory, and controllable bus lines, allowing for parallel processing and reduced latency by skipping tiles during data transfer, thereby minimizing the data path length and enhancing bandwidth through dedicated control tiles and vector processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software-based computations are used for convolutional neural network layers, then flexibility and ease of implementation are improved, but latency increases and bandwidth decreases
Solution Approach 1:
The patent replaces software-based computational mechanisms with hardware-based ASIC circuitry. The convolutional layer is implemented using dedicated hardware components including computational arrays, local memory, and controllable bus lines that perform dot products and activations in hardware rather than through software execution, thereby reducing latency while maintaining implementation flexibility through configurable hardware design
Solution Approach 2:
The patent divides the computational array into multiple tiles that can be independently configured and operated. Each tile contains local memory and computational units that can be selectively activated based on the specific convolutional layer requirements, allowing flexible implementation of different kernel sizes and dimensions while enabling parallel processing to reduce overall latency
2Measurement precision
If software-based dot product computations are performed across multiple dimensions, then computational accuracy is improved, but operational time increases
Solution Approach 1:
The patent implements a three-dimensional computational array structure with dimensions corresponding to height, width, and depth (number of dot products). This 3D architecture enables simultaneous execution of multiple dot products across different spatial dimensions and computational depths, maintaining precise calculations while reducing operational time through parallel processing across all dimensions concurrently
Solution Approach 2:
The patent employs pipelined data flow through the computational array where activations continuously flow from input to output across the height and width dimensions. The controllable bus lines enable continuous transfer of data between tiles and local memory without idle cycles, ensuring that computational resources remain continuously utilized and reducing total operational time while maintaining calculation precision
3Area of stationary object
If data is transferred through multiple tiles in sequence, then data path coverage is improved, but data path length increases
Solution Approach 1:
The patent implements dynamically controllable bus lines that can be selectively activated to create optimized data paths between tiles. The controllable bus lines allow the system to adaptively route data through the minimum necessary tiles based on the specific computational requirements, enabling coverage of the entire array area while minimizing the actual data path length by avoiding unnecessary tile traversals
Solution Approach 2:
The controllable bus lines serve multiple functions: they can transfer activations horizontally and vertically between adjacent tiles, connect to local memory within tiles, and be selectively enabled or disabled based on computational needs. This multi-functionality allows a single bus line infrastructure to provide comprehensive data path coverage while reducing the effective data path length by routing data through the most direct available path for each operation
4Ease of manufacture
If computational arrays are divided into multiple tiles, then manufacturing scalability is improved, but device complexity increases
Solution Approach 1:
The patent divides the large computational array into multiple smaller, identical tile units. Each tile contains a complete set of local memory and computational resources needed to perform its designated operations. This segmentation enables modular manufacturing where tiles can be independently fabricated and then assembled into larger arrays, improving manufacturing scalability while the repetition of identical tile designs actually reduces per-unit complexity
Solution Approach 2:
The patent merges multiple functional components within each tile: local memory for storing activations and weights, computational units for performing dot products, and controllable bus line interfaces for data transfer. By integrating these functions into unified tile modules, the design reduces the overall system complexity that would arise from separate independent components, while maintaining manufacturing scalability through the modular tile architecture
Data Source
AI summary
A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.