Dynamic Contiguous Memory Allocation for Concurrent Threads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer systems with multiple concurrently executing threads, especially in GPU environments, inefficient memory allocation and access occur due to pre-allocation of memory blocks, leading to 'holes' in memory usage, which cause performance decline and complicate memory access.
Innovation Solution
A method of dynamically allocating contiguous memory blocks only to threads that have pending writes, using thread memory counters and atomic operations to ensure efficient and reliable memory allocation and access, avoiding unnecessary pre-allocation and rearrangement of memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory blocks are pre-allocated to threads at launch time, then memory access speed is improved, but memory utilization deteriorates due to holes
Solution Approach 1:
The patent implements dynamic memory allocation where the memory management system continuously monitors thread execution states and reallocates memory blocks based on actual write needs. Threads transition between having memory allocated and not having memory allocated dynamically during execution, rather than being statically pre-allocated at launch. This dynamic approach eliminates holes while maintaining fast access for active writers.
Solution Approach 2:
The system employs self-service mechanisms where threads automatically indicate their write intent through atomic operations on counters, and the memory management system responds by allocating or deallocating blocks without external intervention. This automated feedback loop ensures memory is allocated only when needed and reclaimed when not needed, eliminating waste while maintaining performance.
2Ease of operation
If pre-allocation is used for all threads, then memory access simplicity is improved, but system performance deteriorates due to unused memory blocks
Solution Approach 1:
The patent introduces thread memory counters as intermediary structures that mediate between threads and the memory management system. These counters track which threads need memory, allowing the system to maintain simple atomic operation interfaces for threads while implementing complex dynamic allocation logic in the intermediary layer. This resolves the contradiction by keeping thread operations simple while improving overall system performance through intelligent memory management.
Solution Approach 2:
The system implements feedback mechanisms where thread memory counters provide real-time information about which threads require memory allocation. This feedback enables the memory management system to dynamically adjust allocations based on actual usage patterns, improving system performance without complicating thread access operations. The feedback loop continuously optimizes memory distribution based on observed thread behavior.
3Loss of energy
If a separate process monitors and re-organizes memory blocks, then memory hole elimination is improved, but overhead increases due to re-organization operations
Solution Approach 1:
The patent eliminates the need for separate monitoring processes by implementing self-service mechanisms at the thread level. Each thread's atomic operations on memory counters automatically provide the information needed for memory management, and allocation/deallocation occurs as a natural byproduct of thread execution rather than requiring separate re-organization operations. This reduces overhead while maintaining effective hole elimination.
Solution Approach 2:
The patent extracts the memory monitoring and management functionality from a separate process and integrates it directly into the thread execution model. By embedding memory management logic within the atomic operations that threads already perform, the system eliminates the need for separate monitoring and re-organization processes, reducing overhead while maintaining effective memory hole elimination through the same atomic operation mechanisms.
Data Source
AI summary
A method of allocating a memory to a plurality of concurrent threads is presented. The method includes dynamically determining writer threads each having at least one pending write to the memory; and dynamically allocating respective contiguous blocks in the memory for each of the writer threads. Another method of allocating a memory to a plurality of concurrent threads includes launching the plurality of threads as a plurality of wavefronts, dynamically determining a group of wavefronts each having at least one thread requiring a write to the memory, and dynamically allocating respective contiguous blocks in the memory for each wavefront from the group of wavefronts. A corresponding method of assigning a memory to a plurality of reader threads includes determining a first number corresponding to a number of writer threads having a block allocated in said memory, launching a first number of reader threads, entering a first wavefront of said reader threads from said group of wavefronts to an atomic operation, and assigning a first block in the memory to the first wavefront during the corresponding atomic operation, where the first block is contiguous to a previously allocated block dynamically allocated to another wavefront from said group of wavefronts. Corresponding system embodiments and computer program product embodiments are also presented.