Reordering Buffer for Memory Access Locality in Multi-Core Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-threaded parallel processors, the interleaving of memory read instructions disrupts memory access locality, leading to performance issues in graphics processing as other threads are starved of execution while texture memory read instructions are blocked, resulting in inefficient use of processing resources.
Innovation Solution
A scheduling method that reorders thread execution to ensure localized memory access instructions are executed in sequential cycles, allowing non-batched instructions to be processed between batches, thereby maintaining memory access locality without starving other threads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the instruction scheduler blocks execution of all other instructions while one group of texture memory read instructions is executed by a first thread, then memory access locality is maintained for texture memory reads, but other threads are starved and not processed for many clock cycles
Solution Approach 1:
The instruction stream is segmented into batches, where each batch contains a group of texture memory read instructions that can be executed together to maintain locality. The scheduler processes one batch at a time while allowing other batches from different threads to execute in between, preventing starvation. This segmentation allows the system to maintain locality benefits for texture reads while ensuring fair processing of all threads.
Solution Approach 2:
The instruction scheduler implements periodic switching between different batches of instructions from different threads. Instead of blocking all other threads continuously, the scheduler alternates between batches in a periodic manner, allowing each thread to make progress while still maintaining locality within each batch execution window.
2Productivity
If the instruction scheduler interleaves memory read instructions for two threads, then thread processing continues without starvation, but the locality of memory read instructions is disrupted and graphics processing performance suffers
Solution Approach 1:
By segmenting the instruction stream into batches and executing them sequentially rather than interleaving individual instructions, the system maintains memory access locality within each batch while still providing periodic processing opportunities for all threads. This resolves the conflict between throughput and locality.
3Loss of energy
If texture memory read instructions are executed as a group with blocking of other instructions, then memory bandwidth is reduced for texture map reads, but other threads are effectively starved
Solution Approach 1:
The instruction stream is divided into batches, with each batch containing texture memory read instructions that can be executed together to benefit from locality and reduce memory bandwidth usage. Other batches from different threads are executed in between, ensuring that thread starvation is prevented while still achieving memory bandwidth efficiency within each batch.
Solution Approach 2:
The scheduler periodically switches between different batches, allowing texture memory read batches to execute together for bandwidth efficiency while periodically processing other threads to prevent starvation. This periodic action balances memory efficiency with thread progress.
Data Source
AI summary
Systems and methods for scheduling instructions for execution on a multi-core processor reorder the execution of different threads to ensure that instructions specified as having localized memory access behavior are executed over one or more sequential clock cycles to benefit from memory access locality. At compile time, code sequences including memory access instructions that may be localized are delineated into separate batches. A scheduling unit ensures that multiple parallel threads are processed over one or more sequential scheduling cycles to execute the batched instructions. The scheduling unit waits to schedule execution of instructions that are not included in the particular batch until execution of the batched instructions is done so that memory access locality is maintained for the particular batch. In between the separate batches, instructions that are not included in a batch are scheduled so that threads executing non-batched instructions are also processed and not starved.