Microprocessor Pipeline Thread Recirculation for Cache Miss Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multithreaded, pipelined microprocessor arrangements face inefficiencies due to blocked threads caused by cache misses, leading to lost execution cycles and the need for complex retry mechanisms, particularly in graphics processing pipelines with many stages and active threads.
Innovation Solution
Implement a method where threads that cannot be processed by a pipeline stage are passed to the next stage without blocking, and uncompleted threads are recirculated from the end of the pipeline to the beginning for retry, allowing continuous processing without the need for complex prioritization or selection mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If threads are blocked in the pipeline until cache data is fetched, then data availability is improved, but execution cycles are lost and productivity decreases
Solution Approach 1:
The patent extracts the blocked thread from the pipeline and places it in a retry pool buffer, separating it from the main execution flow. This allows other threads to continue processing without waiting for the blocked thread's cache data to be fetched, thus maintaining productivity while ensuring data availability for the blocked thread when it is eventually retried.
Solution Approach 2:
The retry pool buffer acts as an intermediary between the pipeline and the blocked thread. Instead of blocking the entire pipeline or permanently removing the thread, the buffer provides a temporary holding area where blocked threads can be stored and later retried, mediating between the need for data availability and the need to maintain execution throughput.
2Reliability
If a retry pool buffer is used to store blocked threads, then thread retry capability is improved, but device complexity increases
Solution Approach 1:
The retry pool buffer is implemented as a simple first-in-first-out (FIFO) structure that can be integrated into existing pipeline stages without requiring complex priority management or selection mechanisms. The buffer serves multiple purposes: storing blocked threads, maintaining their state for retry, and enabling simple retry scheduling, thereby reducing overall device complexity while providing reliable thread retry capability.
3Productivity
If multiple threads are processed simultaneously through the pipeline, then productivity is improved, but the impact of blocked threads on overall performance increases
Solution Approach 1:
The patent segments the thread processing into two distinct paths: the main pipeline for active threads and the retry pool buffer for blocked threads. This segmentation ensures that blocked threads do not hold up the main pipeline, allowing multiple threads to be processed simultaneously with minimal impact on overall performance. The segmentation is achieved through simple buffer storage and FIFO retry scheduling rather than complex thread management.
Data Source
AI summary
A microprocessor pipeline arrangement 1 includes a plurality of functional units 2, 3, 4, 5 and 6. Each functional unit 2, 3, 4, 5, 6 also has access to a respective cache memory 7, 8, 9, 10, 11. Threads for processing are received by the first functional unit 2 from an external source 12, and output by an end functional unit 6 of the pipeline to an output target 13. If a thread encounters a cache-miss on its passage through the pipeline, the thread is allowed to continue to pass through the pipeline in the normal manner. However, when the thread reaches the end of the pipeline, it is sent via a loopback path 14 back to the beginning of the pipeline to be sent through the pipeline again. In this way, any thread that has not completed its processing on passing through the pipeline can be sent through the pipeline again to allow the processing of the thread to be completed.