Folded Instruction Fetch Pipeline Parallel Sub-Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The lengthy instruction fetch process in processors is due to sequential access to memory structures, leading to high latency and performance issues, particularly exacerbated by long pipelines and branch mispredictions.
Innovation Solution
The introduction of a three-sub-pipeline architecture that allows parallel access to the translation lookaside buffer, tag RAM, and data RAM using predicted set indices and way numbers, effectively shortening the instruction fetch pipeline and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sequential access to memory structures (TLB, tag RAM, data RAM) is used in the instruction fetch process, then the processor can maintain a simple pipeline structure, but the instruction fetch latency becomes excessively long
Solution Approach 1:
The instruction fetch process is segmented into three independent parallel sub-pipelines: (1) TLB access sub-pipeline that translates virtual fetch address to physical fetch address, (2) tag RAM access sub-pipeline that uses predicted set index to read tags and determine hit way, and (3) data RAM access sub-pipeline that uses predicted set index and predicted way number to fetch instruction block. These segmented sub-pipelines operate simultaneously rather than sequentially, resolving the contradiction between simple structure and long latency.
Solution Approach 2:
The branch target buffer (BTB) performs preliminary action by predicting the set index and way number of the instruction cache entry before the actual fetch occurs. This predicted information is used to pre-position the data RAM access, allowing the instruction block to be fetched in parallel with the tag verification process, thereby reducing overall fetch latency without complicating the pipeline structure.
2Device complexity
If a long instruction fetch pipeline is implemented to handle memory access sequentially, then memory structure access can be simplified, but the latency from branch misprediction detection to correct instruction fetch becomes excessive
Solution Approach 1:
The memory access process is segmented into independent parallel sub-pipelines that can be selectively activated. When branch misprediction is detected, the flushed sub-pipelines can rapidly re-fetch instructions from the correct path because the parallel structure allows immediate re-initialization without waiting for sequential completion of previous stages, thereby reducing misprediction recovery time while maintaining simplified memory access logic.
Solution Approach 2:
The instruction fetch pipeline is made dynamic by enabling selective flushing and re-initialization of specific sub-pipelines based on branch prediction outcomes. The parallel sub-pipeline structure allows the system to adaptively respond to mispredictions by quickly redirecting fetch operations without being constrained by fixed sequential dependencies, thus reducing the duration of action during misprediction recovery.
3Reliability
If all ways of the instruction cache are read out for every fetch, then the instruction cache hit rate improves, but the power consumption increases
Solution Approach 1:
The branch target buffer (BTB) performs preliminary action by predicting the correct way number of the instruction cache entry before the actual fetch. This predicted way number is used to directly address and read out only the specific cache way that is likely to contain the required instruction block, rather than reading out all ways. This maintains high hit rates by accurately predicting the correct cache location while significantly reducing power consumption by minimizing unnecessary memory access activity.
Solution Approach 2:
The instruction cache access mode is changed from reading all ways (high power, guaranteed hit) to reading a single predicted way (low power, high probability hit). The BTB prediction mechanism dynamically adjusts the access parameter (number of ways read) based on prediction confidence, optimizing the balance between hit rate and power consumption by changing the readout scope from comprehensive to selective.
Data Source
AI summary
An instruction fetch pipeline includes first, second, and third sub-pipelines that respectively include: a TLB that receives a fetch virtual address, a tag random access memory (RAM) of a physically-indexed physically-tagged set associative instruction cache that receives a predicted set index, and a data RAM that receives the predicted set index and a predicted way number that specifies a way of the entry from which a block of instructions was previously fetched. The predicted set index specifies the instruction cache set that includes the entry. The three sub-pipelines respectively initiate in parallel: a TLB access using the fetch virtual address to obtain a translation thereof into a fetch physical address that includes a tag, a tag RAM access using the predicted set index to read a set of tags, and a data RAM access using the predicted set index and the predicted way number to fetch the block of instructions.