Compiler Constant Memory Optimization for Heterogeneous Parallel Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current heterogeneous parallel architectures face challenges in optimizing memory access for constant memory, particularly due to the reliance on slow global memory and the lack of on-chip constant memory or constant caches in all architectures, leading to inefficient access times for constants.
Innovation Solution
A compiler-based optimizer that determines the scope and access pattern of constants, transforming code to store and access constants in faster memory types such as registers or on-chip memory, even in architectures without on-chip constant memory or constant caches, by selecting appropriate memory types based on constant size and access patterns at compile-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If constants are stored in global memory in heterogeneous parallel architectures, then memory capacity is sufficient, but access speed is slow
Solution Approach 1:
The patent segments constants into different categories based on their access patterns and characteristics. Constants are divided into those that can be passed as implicit pointer arguments and those that remain in global memory, allowing selective optimization without complicating the entire memory system.
Solution Approach 2:
The patent introduces an intermediary mechanism - the implicit pointer argument system - that bridges between global memory and the kernel execution context. This intermediary allows constants to be efficiently accessed without requiring complex on-chip memory structures in every architecture.
2Speed
If on-chip constant memory or constant cache is added to all architectures, then constant access speed improves, but device complexity and cost increase
Solution Approach 1:
The patent creates a universal optimization approach that works across diverse heterogeneous architectures without requiring architecture-specific modifications. The implicit pointer argument mechanism is architecture-agnostic and can be applied to GPUs, FPGAs, and other parallel processors regardless of whether they have on-chip constant memory.
Solution Approach 2:
The patent changes the parameter of constant storage location from a fixed architecture-dependent location to a flexible location determined by code transformation. Constants can be placed in implicit arguments, on-chip memory if available, or remain in global memory, adapting to each architecture's capabilities without requiring modification to the optimization approach.
3Productivity
If code is transformed to pass constants as implicit pointer arguments, then memory access efficiency improves, but code complexity increases
Solution Approach 1:
The patent performs preliminary analysis during compilation to identify constants that are good candidates for implicit pointer argument transformation. By analyzing constant scope and access patterns at compile time, the system prepares the optimal representation before code generation, avoiding runtime complexity.
Solution Approach 2:
The patent implements self-service optimization where the compiler automatically identifies and transforms suitable constants without requiring programmer intervention. The constant memory optimizer autonomously analyzes the code, determines which constants should be passed as implicit pointer arguments, and performs the transformation, reducing the perceived complexity for users.
Data Source
AI summary
The disclosed systems, structures, and methods are directed to optimizing memory access to constants in heterogeneous parallel computers, including systems that support OpenCL. This is achieved in an optimizing compiler that transforms program scope constants and constants at the outermost scope of kernels into implicit constant pointer arguments. The optimizing compiler also attempts to determine access patterns for constants at compile-time and places the constants in a variety of memory types available in a compute device architecture based on these access patterns.