Spatial Memory Streaming Prefetch Engine Confidence Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spatial memory streaming (SMS) schemes face challenges such as inability to handle shifting alignment of patterns with respect to cache line boundaries, larger granularity due to spatial bit vectors, lack of temporal order tracking, and inadequate confidence mechanisms, leading to reduced performance and increased power consumption.
Innovation Solution
A spatial memory streaming prefetch engine with a pattern history table (PHT) that includes offset lists and per-offset confidence fields, allowing continuous learning and adaptation to changing program behavior by updating confidence levels based on current accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If spatial bit vectors are used for spatial patterns, then spatial correlation can be tracked, but granularity per access becomes larger and temporal order cannot be tracked
Solution Approach 1:
The patent segments the pattern history table into multiple independent tables, each tracking specific offset values. This segmentation allows the system to maintain fine-grained spatial information while separately tracking temporal order through the activation and updating of individual table entries, thus resolving the contradiction between spatial precision and temporal information retention.
Solution Approach 2:
The patent introduces a new dimension by adding per-offset confidence fields to the existing offset tracking structure. This additional dimension enables the system to track both spatial patterns (through offset values) and temporal order (through confidence levels that accumulate over time), effectively adding temporal tracking capability without sacrificing spatial precision.
2Measurement precision
If SMS schemes use fixed patterns, then spatial correlation can be detected, but they cannot adapt to program phase changes such as dynamic branch behavior
Solution Approach 1:
The patent implements dynamic adaptation by introducing confidence fields that are continuously updated based on actual memory access patterns. When program phase changes occur (such as dynamic branch behavior), the confidence levels are adjusted accordingly, allowing the system to adapt to new patterns while maintaining the core spatial correlation detection capability. This dynamic updating mechanism enables the prefetcher to respond to changing program behavior.
Solution Approach 2:
The patent incorporates feedback mechanisms where the actual memory access outcomes are used to update per-offset confidence values. This feedback loop allows the system to learn from past predictions and adjust future prefetching behavior, enabling adaptation to program phase changes while preserving the ability to detect spatial correlations in stable phases.
3Loss of time
If prefetching accuracy is improved through better patterns, then timeliness is improved, but power consumption increases due to more data movement
Solution Approach 1:
The patent applies local quality by implementing per-offset confidence fields rather than a single global confidence metric. This allows the system to selectively prefetch only those memory locations with high confidence values, improving prefetch accuracy and timeliness while reducing unnecessary data movement and associated power consumption. Each offset can have its own confidence level, enabling fine-grained control over prefetching decisions.
Solution Approach 2:
The patent changes the parameter structure by introducing confidence levels as a new parameter that modulates prefetching behavior. By adjusting confidence thresholds and using confidence values to gate prefetching decisions, the system can optimize the balance between prefetch timeliness and power consumption, prefetching only when confidence indicates high likelihood of actual usage.
Data Source
AI summary
Apparatuses and methods of manufacturing same, systems, and methods for a spatial memory streaming (SMS) prefetch engine are described. In one aspect, the SMS prefetch engine includes a pattern history table (PHT), which has a table in which each entry has an offset list field comprising sub-fields for offset values from a base offset value within a region and a per-offset confidence field comprising sub-fields for per-offset confidence levels corresponding to each offset value. When a PHT entry is activated, the per-offset confidence values corresponding to each offset value in the offset list field of the PHT entry are updated by matching current accesses to the stored offset values in the offset list field of the activated PHT entry. Continuous learning may be provided to the SMS engine at least by the per-offset confidence levels.