Large Tensor Tiling for Systolic Array Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face inefficiencies when processing large tensors of varying sizes, leading to increased power consumption, complexity, and reduced processing efficiency due to the need for re-reading duplicated data and reorganizing hardware for larger tiles.
Innovation Solution
The implementation of large tensor tiling (LTT) techniques, which divide large input tensors into smaller overlapping or crossover tiles with duplicated edges, allowing for efficient processing by reusing data across shared boundaries within a systolic array, thereby avoiding the need for re-reading duplicated data and reorganizing hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large tensors are processed as single large tiles, then processing accuracy is maintained, but power consumption increases and hardware complexity increases due to re-reading duplicated data and reorganizing hardware
Solution Approach 1:
The patent divides large input tensors into smaller overlapping tiles that can be processed by individual processing elements in the systolic array. Each tile is smaller than the full tensor, allowing parallel processing across multiple PEs without requiring the entire large tensor to be loaded into memory at once, thereby reducing power consumption while maintaining processing accuracy through proper tile overlap and data reuse strategies
2Reliability
If large tensors are processed as single large tiles, then processing accuracy is maintained, but hardware complexity increases due to reorganizing hardware for larger tiles
Solution Approach 1:
The patent segments the large tensor processing task into smaller tile processing units that fit within the capabilities of individual processing elements. This segmentation allows the hardware to process tiles of manageable size using fixed hardware configurations, avoiding the need for complex hardware reorganization when handling different tensor sizes while maintaining processing accuracy
Solution Approach 2:
The patent designs a universal tiling approach where the same hardware architecture can process tensors of various sizes by dynamically adjusting tile dimensions and overlap parameters. This multi-functionality eliminates the need for hardware reconfiguration for different tensor sizes, reducing hardware complexity while maintaining processing accuracy across diverse input dimensions
3Reliability
If large tensors are processed as single large tiles, then processing accuracy is maintained, but processing efficiency decreases due to re-reading duplicated data
Solution Approach 1:
The patent segments the tensor into overlapping tiles where boundary regions are shared between adjacent tiles. This segmentation enables processing elements to reuse data from shared boundaries without re-reading from external memory, significantly improving processing efficiency while maintaining accuracy through proper overlap management
Solution Approach 2:
The patent performs preliminary tile division and overlap identification before processing begins. By pre-computing which data elements will be shared between tiles and optimizing the tiling scheme in advance, the system avoids redundant data reads during processing, improving efficiency while maintaining processing accuracy through careful data reuse strategies
4Adaptability or versatility
If tensors of varying sizes are processed, then adaptability to different input dimensions is improved, but processing efficiency decreases without tiling
Solution Approach 1:
The patent implements dynamic tile sizing and overlap configuration that adapts to the dimensions of the input tensor. The system can adjust tile dimensions, overlap amounts, and data reuse patterns based on the specific tensor size, maintaining high processing efficiency across varying input dimensions while preserving adaptability to different tensor configurations
Data Source
AI summary
Techniques for performing large tensor tiling (LTT) in hardware are enabled. LTT divides a large tensor (e.g., of unsupported size) into overlapping tiles (e.g., having supported tensor size(s)). A tensor may be processed processing the tiles. The output of each processed tile is stored, for example, in a systolic array considering the tile's placement in the large tensor. The output of all processed tiles is identical to the output of processing the large tensor. Tiles may be processed by reusing data overlapping boundaries shared with other tiles. In some examples, overlapping data may be reused (e.g., written once) or partly reused (e.g., written twice). Tiling large tensors with boundary duplication supports dynamic adaptation to a wide variety of tensor sizes, avoids re-reading duplicated data, and avoids reorganizing hardware for large tiles, which reduces power consumption and area, reduces complexity, and increases processing efficiency.