ASIC Tile Architecture for Parallel Neural Network Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for computing in neural networks, particularly convolutional neural networks, face inefficiencies due to software-based methods that perform computations sequentially, leading to high latency and reduced bandwidth, especially when processing large datasets.
Innovation Solution
The implementation of an application-specific integrated circuit (ASIC) with a tile-based architecture that includes a computational array, local memory, and controllable bus lines, allowing for parallel processing and data transfer between tiles, thereby reducing latency and increasing bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software-based methods are used to perform computations for convolutional neural network layers, then flexibility and ease of implementation are improved, but processing speed and bandwidth deteriorate due to sequential computation
Solution Approach 1:
The patent replaces software-based sequential computation with hardware-based parallel computation using an array of compute units. Each compute unit in the hardware array can simultaneously perform multiplication and accumulation operations, eliminating the sequential bottleneck inherent in software implementations while maintaining the flexibility of programmable hardware through configurable interconnect structures.
Solution Approach 2:
The patent divides the computational array into multiple independently controllable tiles, where each tile contains a subset of compute units. This segmentation allows different tiles to process different portions of the neural network computation simultaneously, enabling parallel processing that dramatically increases throughput compared to sequential software execution while maintaining ease of programming through high-level language support.
2Device complexity
If software-based sequential computation is used, then device complexity is reduced, but latency increases due to sequential processing of activation inputs
Solution Approach 1:
The patent transitions from one-dimensional sequential processing in software to two-dimensional parallel processing in hardware by arranging compute units in an array structure. This dimensional change allows multiple computations to occur simultaneously across the array, dramatically reducing latency while the modular tile architecture keeps device complexity manageable through hierarchical organization.
Solution Approach 2:
The patent implements local memory within each tile that can pre-load and store weight inputs and activation inputs before computation begins. This preliminary action allows data to be staged and prepared in advance, enabling the compute units to immediately begin parallel processing without waiting for sequential data delivery, thereby reducing overall latency without significantly increasing device complexity.
3Adaptability or versatility
If software-based methods process large datasets, then adaptability is maintained, but bandwidth is reduced due to sequential data access patterns
Solution Approach 1:
The patent implements a universal tile architecture where each tile can be configured to process different neural network layer types (convolutional, fully connected, etc.) through programmable interconnect structures. This universality maintains adaptability for processing various dataset types and network architectures, while the parallel hardware structure simultaneously processes multiple data elements, dramatically increasing effective bandwidth compared to sequential software access patterns.
Data Source
AI summary
A tile including circuitry for use with machine learning models, the tile including: a first computational array of cells, in which the computational array of cells is a sub-array of a larger second computational array of cells; local memory coupled to the first computational array of cells; and multiple controllable bus lines, in which a first subset of the multiple controllable bus lines include multiple general purpose controllable bus lines couplable to the local memory.