Tiled Systolic ASIC Architecture for Low-Latency CNN Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems perform computations for convolutional neural network layers in software, which is inefficient and leads to high latency due to the need for repeated dot product calculations and data transfer between layers.
Innovation Solution
An application-specific integrated circuit (ASIC) chip with a systolic array of cells arranged in multiple tiles, each with local memory and controllable bus lines, is used to perform computations. This configuration allows for efficient data transfer and parallel processing, reducing latency and increasing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software is used to perform convolutional neural network computations, then flexibility and ease of implementation are improved, but processing speed and latency are worsened due to repeated dot product calculations and data transfer
Solution Approach 1:
The patent replaces software-based computation with a hardware-based systolic array architecture. The systolic array uses dedicated computational units arranged in a grid pattern with local memory and controllable bus lines, transforming the mechanical/software execution model into a hardware/parallel processing model. This substitution eliminates the overhead of repeated dot product calculations and data transfers between layers, directly resolving the latency issue while maintaining computational flexibility through configurable kernel applications.
2Measurement precision
If software performs repeated dot product calculations with sliding window, then computational accuracy is maintained, but processing efficiency and bandwidth are worsened
Solution Approach 1:
The patent segments the computational array into multiple independent processing units arranged in a grid, where each unit can perform dot product calculations simultaneously. The controllable bus lines divide data pathways into dedicated channels for different computational units, enabling parallel processing of multiple kernels across different regions of the input data. This segmentation maintains computational accuracy through precise local memory storage and retrieval while dramatically improving processing efficiency through concurrent operations.
Solution Approach 2:
The patent transitions from sequential one-dimensional software processing to two-dimensional hardware parallel processing. The systolic array arranges computational units and memory elements in a spatial grid, adding a dimensional aspect to data processing. Multiple kernels can be applied simultaneously across different spatial positions in the input data, transforming the single-threaded sequential execution into multi-threaded parallel execution, thereby maintaining accuracy while boosting productivity.
3Device complexity
If data is transferred between layers in software, then system simplicity is maintained, but bandwidth and latency are worsened due to repeated data movement
Solution Approach 1:
The patent merges the computational units with local memory elements and controllable bus lines into integrated tile structures within the systolic array. This merging eliminates the need for separate data transfer operations between layers, as each tile can perform computations using its locally stored data and exchange information with adjacent tiles through dedicated bus lines. The integration reduces data movement overhead and latency while maintaining system simplicity through a unified hardware architecture that combines storage and computation.
4Productivity
If systolic array with multiple tiles and local memory is implemented, then processing speed and bandwidth are improved, but device complexity and manufacturing difficulty are worsened
Solution Approach 1:
The patent segments the large-scale systolic array into multiple manageable tiles, where each tile contains a subset of computational units, local memory, and controllable bus lines. This segmentation reduces the complexity of individual manufacturing units while enabling parallel assembly and testing. Each tile can be independently fabricated and validated, then combined to form the complete high-performance processing array, thereby achieving high processing speed without proportionally increasing manufacturing difficulty.
Solution Approach 2:
The patent designs the tiles with universal, multi-functional components that can perform multiple operations. The controllable bus lines can dynamically route data between different tiles and memory elements, and the local memory can serve both as input storage and output buffer. This universality reduces the need for specialized components for each function, simplifying the overall device architecture while maintaining high processing speed through flexible resource allocation and parallel operation.
Data Source
AI summary
An application specific integrated circuit (ASIC) chip includes: a systolic array of cells; and multiple controllable bus lines configured to convey data among the systolic array of cells, in which the systolic array of cells is arranged in multiple tiles, each tile of the multiple tiles including 1) a corresponding subarray of cells of the systolic array of cells, 2) a corresponding subset of controllable bus lines of the multiple controllable bus lines, and 3) memory coupled to the subarray of cells.