Processor Move-In Buffer for Lower Cache Access Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increased latency in memory access due to the addition of multiple functions and miniaturization in processors, particularly in securing cache coherency, leads to decreased processing efficiency and inhibits speculative execution of instructions.
Innovation Solution
A processor design with a move-in buffer that issues both normal and pseudo memory requests to multiple cache levels, allowing parallel processing and speculative execution, while a re-order buffer manages speculative execution completion and rewinds operations upon failure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the procedure for securing cache coherency is performed, then data consistency is ensured, but latency increases and processing efficiency decreases
Solution Approach 1:
The system performs preliminary speculative execution of instructions before cache coherency is fully secured. The move-in buffer stores memory access instructions that result in cache misses, and the re-order buffer allows speculative execution to proceed while coherency procedures are being completed in the background, thus performing the useful action in advance without waiting for coherency confirmation.
Solution Approach 2:
The system dynamically switches between speculative execution mode and normal execution mode. When speculative execution is detected, the re-order buffer enables dynamic rewinding of the arithmetic operation circuit to a state before speculative execution upon coherency failure, allowing the system to adapt its execution strategy based on coherency status without permanently compromising performance.
2Productivity
If speculative execution is performed, then program execution efficiency is improved, but cache coherency cannot be secured
Solution Approach 1:
The move-in buffer acts as an intermediary between the arithmetic operation circuit and the cache system. It stores memory access instructions that result in cache misses and manages the interaction between speculative execution and coherency procedures, allowing speculative execution to proceed while coherency is secured in the background without direct interference between the two processes.
Solution Approach 2:
The system implements feedback mechanisms where the re-order buffer monitors cache coherency status and provides feedback to the arithmetic operation circuit. When coherency failure is detected, the feedback triggers automatic rewinding of the circuit to a state before speculative execution, allowing the system to learn from coherency violations and correct them while maintaining overall execution efficiency.
3Adaptability or versatility
If multiple functions are added and miniaturization is performed, then integration scale is increased, but memory access latency increases
Solution Approach 1:
The system segments the memory access process into multiple independent components: the move-in buffer for storing cache miss instructions, the re-order buffer for managing execution order, and the arithmetic operation circuit for execution. This segmentation allows parallel processing of different memory access operations and reduces the impact of cache coherency procedures on overall latency by isolating them from the critical execution path.
Data Source
AI summary
The processor includes core circuits and a cache unit having L2 to LN caches (N is 3 or more). L1 cache has a move-in buffer including entries in which memory access instruction resulted in cache miss in L1 cache is stored. The move-in buffer, when issuing a normal memory request to L2 cache, issues a pseudo memory request to L3 to LN caches in parallel and receives a pseudo data response that has coherency-unsecured data from any one cache. The re-order buffer executes a normal instruction execution completion process in response to the normal data response, executes a pseudo instruction execution completion process in response to the pseudo data response, and, when the pseudo data response is a failure, rewinds an arithmetic operation circuit that speculatively executed instructions after the memory access instruction in response to the pseudo instruction execution completion process, back to a state before the speculative execution.