Remote Memory Access Scheduling to Reduce Processor Stalls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In systems with shared memory address spaces, remote memory access operations lead to significant performance bottlenecks due to longer access times, causing hardware processors to stall for hundreds of clock cycles, and existing solutions do not effectively manage these delays while maintaining program correctness.
Innovation Solution
Implementing a hardware processor configuration that allows executing a sequence of other instructions after a remote memory access instruction, including completion instructions and non-blocking memory access instructions, to reduce stalling time and maintain program control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the hardware processor executes a synchronous remote memory access instruction, then the remote memory address is accessed correctly, but the hardware processor stalls for hundreds of clock cycles waiting for the access to complete
Solution Approach 1:
The patent introduces a preliminary action by executing a completion instruction before the actual remote memory access instruction. This completion instruction prepares the processor pipeline and initiates background processing tasks that can execute concurrently with the remote memory access, thereby reducing the effective stalling time while maintaining access correctness.
Solution Approach 2:
The patent implements continuity of useful action by introducing background processing that continues executing instructions during the remote memory access period. Instead of leaving the processor idle during the hundreds of clock cycle wait, the system continuously executes background instructions that do not depend on the remote memory outcome, maximizing processor utilization throughout the entire access period.
2Reliability
If the hardware processor stalls waiting for remote memory access to complete, then program correctness is maintained, but system throughput decreases
Solution Approach 1:
The patent segments the instruction execution into foreground and background portions. The foreground execution maintains program correctness by preserving the original synchronous semantics for dependent instructions, while the background execution processes independent instructions concurrently. This segmentation allows the system to maintain reliability for critical path instructions while improving overall throughput through parallel background processing.
Solution Approach 2:
The patent introduces an intermediary mechanism through the completion instruction that bridges the synchronous remote memory access and the background processing. This intermediary allows the processor to initiate background tasks while the remote access is in progress, and properly synchronize when the access completes, thereby maintaining program correctness while enabling concurrent execution to improve throughput.
3Productivity
If the hardware processor executes instructions out of order to fill pipeline slots during stalls, then processor utilization improves, but instruction dependency correctness may be compromised
Solution Approach 1:
The patent applies dynamics by conditionally executing instructions based on their dependency relationships. Instructions that are independent of the remote memory access outcome are executed in the background during the stall period, while instructions that depend on the outcome wait for proper synchronization. This dynamic execution approach maintains instruction dependency correctness while maximizing processor utilization through selective parallel execution.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
An apparatus for executing a software program, comprising at least one hardware processor configured for: identifying in a plurality of computer instructions at least one remote memory access instruction and a following instruction following the at least one remote memory access instruction; executing after the at least one remote memory access instruction a sequence of other instructions, where the sequence of other instructions comprises a return instruction to execute the following instruction; and executing the following instruction; wherein executing the sequence of other instructions comprises executing an updated plurality of computer instructions produced by at least one of: inserting into the plurality of computer instructions the sequence of other instructions or at least one flow-control instruction to execute the sequence of other instructions; and replacing the at least one remote memory access instruction with at least one non-blocking memory access instruction.