Dynamic Off-Chip Memory Allocation for Unified Shader Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multithreaded processing systems face challenges in efficiently allocating memory for parallel thread processing, leading to increased costs and reduced performance due to the need for large off-chip memory and limited thread parallelism.
Innovation Solution
A system and method for dynamically allocating per-thread memory from a memory pool, allowing fixed size portions to be allocated and deallocated as needed, enabling more threads to execute in parallel without requiring a large memory pool, and allowing different memory allocations for various thread types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large amount of off-chip memory is allocated to accommodate the greatest amount of memory needed by any thread, then all threads can be processed in parallel, but the cost of producing the processing system increases
Solution Approach 1:
The patent implements dynamic memory allocation where memory is allocated to threads on-demand rather than statically pre-allocated. The memory allocation unit dynamically assigns memory regions from a pool to active threads and deallocates when threads complete, allowing the system to support many threads in parallel without requiring memory for all threads simultaneously.
Solution Approach 2:
The system recovers and reuses memory resources by deallocating memory to threads that have completed execution. The memory allocation unit maintains a pool of reusable memory regions that are continuously recycled as threads finish, enabling sustained high thread parallelism without proportionally increasing total memory requirements.
2Quantity of substance
If the amount of memory allocated to threads is limited to reduce cost, then the number of threads that can be processed in parallel is limited by disabling some thread processors, but processing performance is reduced
Solution Approach 1:
The system uses dynamic memory allocation to allow flexible adjustment of memory resources based on actual thread execution needs. Rather than being constrained by fixed memory allocations, the system can dynamically allocate memory to active threads from a shared pool, enabling more threads to run in parallel with limited total memory.
Solution Approach 2:
The memory pool serves multiple functions and multiple thread types simultaneously. A single unified memory pool is shared across all thread processors, allowing the same memory resources to be reused by different threads at different times, maximizing memory utilization efficiency and supporting higher thread parallelism.
3Adaptability or versatility
If a single fixed size portion of memory is allocated to each thread based on the largest amount needed by any thread, then all thread types can be supported, but memory efficiency is reduced
Solution Approach 1:
The system applies local quality by allocating memory based on each thread's specific needs rather than using a uniform allocation for all threads. Each thread receives the appropriate amount of memory from the pool based on its requirements, allowing threads with smaller memory needs to use less memory while still supporting all thread types.
Solution Approach 2:
The system changes the memory allocation parameter dynamically based on thread type and execution state. Rather than using a fixed allocation size, the memory allocation unit adjusts the amount of memory assigned to each thread according to its specific requirements, improving overall memory efficiency while maintaining support for diverse thread types.
Data Source
AI summary
Systems and methods for dynamically allocating memory for thread processing may reduce memory requirements while maintaining thread processing parallelism. A memory pool is allocated to store data for processing multiple threads that does not need to be large enough to dedicate a fixed size portion of the memory pool to each thread that may be processed in parallel. Fixed size portions of the memory pool are dynamically allocated and deallocated to each processing thread. Different fixed size portions may be used for different types of threads to allow greater thread parallelism compared with a system that requires allocating a single fixed portion of the memory pool to each thread. The memory pool may be shared between all of the thread types or divided to provide separate memory pools dedicated to each particular thread type.