Tensor Duplication and Trimming for CGRA Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Memory allocation in reconfigurable data processors, particularly in coarse-grained reconfigurable architectures (CGRAs), is inefficient and affects the performance of dataflow computing systems due to overlapping memory access patterns and resource conflicts.
Innovation Solution
A data processing system with an array of reconfigurable units, including pattern compute units (PCUs) and pattern memory units (PMUs), utilizes a compiler to create duplicate tensors, assign memory access patterns, trim unnecessary portions, and dispatch contexts to PMUs, ensuring no duplicate tensor exceeds its available ports, thereby optimizing memory allocation and reducing conflicts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a single tensor is shared among multiple memory access patterns, then memory utilization is improved, but resource conflicts and access efficiency deteriorate
Solution Approach 1:
The patent segments a single logical tensor into multiple duplicate physical tensors, where each duplicate is dedicated to specific memory access patterns. This segmentation resolves resource conflicts by assigning exclusive access rights to each duplicate tensor for its designated access patterns, thereby maintaining high access efficiency while still achieving overall memory utilization through shared logical tensor management.
Solution Approach 2:
The patent creates duplicate copies of the logical tensor in physical memory, with each copy assigned to handle specific read or write access patterns. This copying approach allows multiple access patterns to operate simultaneously on different tensor duplicates without interference, resolving the conflict between shared memory utilization and efficient concurrent access.
2Productivity
If multiple duplicate tensors are created for each memory access pattern, then access efficiency is improved, but memory overhead and system complexity worsen
Solution Approach 1:
The patent implements a universal tensor management system where a single logical tensor definition serves multiple functions by being mapped to different physical duplicate tensors. The compiler and memory management system provide universal handling of tensor access patterns, automatically assigning appropriate duplicates to different access patterns without requiring complex manual configuration, thus reducing system complexity while maintaining access efficiency.
Solution Approach 2:
The patent introduces a compiler and memory management system as an intermediary between the logical tensor definition and physical memory allocation. This intermediary automatically handles the complexity of creating and managing duplicate tensors by analyzing access patterns and assigning appropriate duplicates, thereby shielding the user from system complexity while achieving efficient access through multiple tensor copies.
3Adaptability or versatility
If tensor duplication is implemented without trimming, then all access patterns are supported, but memory consumption increases
Solution Approach 1:
The patent applies local quality optimization by trimming each duplicate tensor to retain only the portions necessary for its assigned access patterns. Different duplicate tensors have different retained portions based on their specific read or write access requirements, rather than maintaining complete copies. This localized optimization reduces overall memory consumption while preserving full adaptability to support all required access patterns.
4Ease of manufacture
If static memory allocation is used, then implementation simplicity is improved, but performance optimization and adaptability worsen
Solution Approach 1:
The patent implements dynamic memory allocation where the number and configuration of duplicate tensors are determined at runtime based on actual access patterns, rather than being fixed statically. The system dynamically creates or removes tensor duplicates as needed, allowing performance optimization for different workloads while maintaining relative implementation simplicity through automated compiler and runtime management of the dynamic allocation process.
Data Source
AI summary
A system comprises an array of reconfigurable units including a plurality of pattern compute units (PCUs) and a plurality of pattern memory units (PMUs), and a compiler configured to receive a tensor including a plurality of memory access patterns of a first type (read) and a second type (write) located in a logical memory of the compiler. Each memory access pattern includes a memory access particular to that type. The compiler is configured to create a plurality of duplicates of the tensor and assign one or more contexts of any type to the duplicate tensors, such that no two contexts of the same type are in the same duplicate tensor. The compiler is configured to trim the duplicate tensors to retain portions including its corresponding assigned contexts remove portions that are inconsequential to the assigned contexts and dispatch the assigned contexts from each duplicate tensor to one or more PMUs.