Stall-free pipelined cache with flexible load latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital signal processors experience significant latency and pipeline stalls due to cache misses, leading to inefficiencies in processing core utilization and increased overhead from stall cycles, which are exacerbated by the assumption that data is always available within a fixed four-cycle latency.
Innovation Solution
Implementing flexible load latency that allows cache misses to be pipelined by adding a miss-under-service bit to each cache line and modifying the memory data controller to accumulate and re-insert pipeline data, enabling reduced or eliminated stall cycles by assuming longer latency periods, such as six or eight cycles, and dynamically adjusting load instructions to specify expected latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed four-cycle latency is assumed for cache accesses, then pipeline scheduling is simplified, but pipeline stalls occur due to cache misses
Solution Approach 1:
The patent implements dynamic load latency values instead of fixed four-cycle latency. Load instructions can specify different latency values (e.g., 6 or 8 cycles) based on whether cache misses are anticipated, allowing the pipeline to adapt to actual memory access patterns and avoid stalls while maintaining scheduling feasibility
Solution Approach 2:
The patent uses miss-under-service bits set during the program (PG) phase to predict cache misses before they occur. This preliminary identification allows the pipeline to prepare for extended latency by inserting appropriate stall cycles in advance, preventing pipeline bubbles and maintaining continuous execution flow
2Productivity
If longer load latency is used to hide cache miss penalties, then stall cycles are reduced, but instruction execution time increases
Solution Approach 1:
The patent changes the latency parameter of load instructions dynamically based on cache hit/miss predictions. By adjusting the latency value (4, 6, or 8 cycles) according to actual cache behavior and pipeline state, the system optimizes the balance between hiding miss penalties and minimizing overall execution time
Solution Approach 2:
The patent ensures continuous pipeline execution by using the extended latency to overlap cache access operations with subsequent instruction execution. While data is being fetched from cache or memory, the pipeline continues processing other instructions, thereby hiding the penalty of cache misses and maintaining continuous useful work
Data Source
AI summary
This invention provides flexible load latency to pipeline cache misses. A memory controller selects the output of one of a set of cascades inserted execute stages. This selection may be controlled by a latency field in a load instruction or by a latency specification of a prior instruction. This invention is useful in the great majority of cases where the code can tolerate incremental increases in load latency for a reduction in cache miss penalty.