Data Processing Array Dynamic Overlay Reconfiguration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrated circuits (ICs) face inefficiencies in adapting to different computational workloads, particularly in multi-layered applications like machine learning, where static circuit architectures struggle to efficiently perform matrix multiply operations of varying dimensions, leading to reduced computational efficiency and throughput.
Innovation Solution
A data processing array (DP array) within an IC is configured with a method that loads applications specifying kernels and stream channels, allowing for the sequential implementation of overlays that adapt data movement modes, enabling efficient execution of different workload layers without reconfiguring the entire array, aided by array controllers to manage operations efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a static circuit architecture is used, then device complexity is reduced, but adaptability to different computational workloads deteriorates
Solution Approach 1:
The patent implements dynamic reconfiguration of the data processing array by loading different overlays at runtime. Each overlay contains configuration data that reconfigures compute tiles and stream channels for specific computational patterns. This allows the static hardware architecture to dynamically adapt to different workloads (e.g., GEMM, convolution, FFT) without physical reconfiguration, resolving the contradiction between structural simplicity and operational adaptability.
Solution Approach 2:
The system changes operational parameters by loading different overlay configuration data into the data processing array. Each overlay specifies parameters such as data movement modes, compute tile configurations, and stream channel mappings. By changing these parameters through software-controlled overlay loading, the system adapts to different computational workloads while maintaining the same physical hardware structure.
2Adaptability or versatility
If the entire array is reconfigured for different workloads, then adaptability improves, but reconfiguration time increases
Solution Approach 1:
The patent segments the configuration data into multiple overlays, where each overlay contains configuration information for a specific computational pattern or layer type. Instead of reconfiguring the entire array at once, the system can selectively load only the necessary overlay for the current workload. This segmentation of configuration data enables faster reconfiguration by loading smaller, targeted overlay files rather than complete array configurations.
Solution Approach 2:
The system performs preliminary action by pre-compiling multiple overlays for different computational patterns (GEMM, convolution, transpose, etc.) and storing them in memory before runtime. When a workload needs to be executed, the appropriate pre-compiled overlay is already available for immediate loading, eliminating the need for runtime compilation or extensive reconfiguration. This preliminary preparation significantly reduces reconfiguration time.
3Productivity
If overlays are sequentially implemented, then computational efficiency improves, but runtime overhead increases
Solution Approach 1:
The patent implements continuous useful action by designing overlays that maintain the data processing array in an active computational state throughout execution. Each overlay configures the array for a specific computational pattern, and the array processes data continuously through stream channels without idle reconfiguration periods between layers. The sequential overlay implementation is optimized so that overlay switching occurs at layer boundaries without interrupting the overall computational flow, maintaining high productivity while managing runtime overhead.
Data Source
AI summary
Using multiple overlays with a data processing array includes loading an application in a data processing array. The data processing array includes a plurality of compute tiles each having a processor. The application specifies kernels executable by the processors and implements stream channels that convey data to the plurality of compute tiles. During runtime of the application, a plurality of overlays are sequentially implemented in the data processing array. Each overlay implements a different mode of data movement in the data processing array via the stream channels. For each overlay implemented, a workload is performed by moving data to the plurality of compute tiles based on the respective mode of data movement.