Shared Memory Access via Dual Code Version Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing OpenCL frameworks face inefficiencies in shared memory access, particularly when the work group size is small or the shared local memory is limited, leading to suboptimal performance in computations like matrix multiplication, as they rely on cache-based shared memory without fully utilizing register-based memory access.
Innovation Solution
A new framework compiles two versions of executable binary code: one for register-based shared local memory and another for cache-based shared local memory, allowing the runtime code to choose the optimal version based on work group size and hardware capabilities, thereby improving shared memory access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cache-based shared local memory is used, then compatibility with existing OpenCL frameworks is maintained, but shared memory access performance deteriorates when work group size is small or shared local memory is limited
Solution Approach 1:
The patent changes the memory access parameter by providing two versions of executable binary code: one version accesses shared local memory through cache while another version accesses it through registers. The system dynamically selects between these versions based on runtime parameters such as work group size and shared local memory size, thereby optimizing performance for different scenarios while maintaining compatibility
Solution Approach 2:
The patent introduces dynamic selection between two code versions at runtime based on work group size and hardware capabilities. This dynamic approach allows the system to adapt to different execution contexts, choosing the optimal memory access method (cache-based or register-based) for each specific scenario rather than using a fixed approach
2Productivity
If register-based shared local memory access is implemented, then shared memory access performance improves, but device complexity increases due to need for multiple code versions
Solution Approach 1:
The patent applies preliminary action by compiling two versions of executable binary code in advance during the compilation phase. One version is optimized for cache-based access while the other is optimized for register-based access. This pre-compilation approach eliminates the need for complex runtime compilation decisions, reducing execution-time complexity while maintaining performance benefits
Solution Approach 2:
The patent introduces an intermediary selection mechanism that chooses between two pre-compiled code versions based on runtime parameters. This intermediary layer (the version selection logic) simplifies the overall system by providing a straightforward decision-based approach rather than requiring complex dynamic code generation or interpretation, thereby managing device complexity effectively
Data Source
AI summary
Methods and apparatus to improve shared memory efficiency are described. In an embodiment, a first version of a code to access one or more registers as shared local memory is compiled. A second version of the same code is also compiled to access a cache as the shared local memory. The first version of the code is executed in response to comparison of a work group size of the code with a threshold value. Other embodiments are also disclosed and claimed.