Perceptron Off-Chip Predictor Selective Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing off-chip predictors in CPU cache hierarchies suffer from inaccurate predictions that increase DRAM transactions and hinder performance, particularly in bandwidth-constrained scenarios, and existing prefetching techniques are not agile or responsive enough, leading to significant area overheads and suboptimal performance gains.
Innovation Solution
A two-level perceptron-based predictor system, comprising a first-level perceptron (FLP) with selective delay mechanism and a second-level perceptron (SLP) for off-chip prediction and prefetch filtering, using program features and confidence thresholds to selectively delay predictions until L1D lookup is resolved, reducing unnecessary DRAM transactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If off-chip prediction is implemented to reduce cache hierarchy access latency, then the latency of load requests served from DRAM is reduced, but the number of DRAM transactions significantly increases
Solution Approach 1:
The system performs preliminary off-chip prediction using a perceptron-based predictor before the L1D cache lookup is completed. By predicting off-chip loads in advance and initiating DRAM transactions speculatively, the system reduces the critical path latency for correctly predicted off-chip loads while filtering out inaccurate predictions through the L1D lookup result validation.
Solution Approach 2:
The L1D cache lookup acts as an intermediary validation mechanism. The system uses the L1D lookup result to verify off-chip predictions: if the L1D hit is confirmed, the speculative DRAM transaction is aborted; if the L1D miss is confirmed, the DRAM transaction proceeds. This intermediary step filters inaccurate predictions without completely blocking the fast path for accurate predictions.
2Quantity of substance
If off-chip predictions are delayed until L1D lookup is completed to filter inaccurate predictions, then the number of useless DRAM transactions is reduced, but the performance gains from accurate predictions are suboptimal
Solution Approach 1:
The system performs off-chip prediction preliminarily, before the L1D lookup completes, using a perceptron-based predictor that analyzes program features. This allows accurate predictions to proceed through the fast path without waiting for L1D validation, while inaccurate predictions are filtered out by the subsequent L1D lookup result, achieving both low latency and high accuracy.
Solution Approach 2:
The system maintains continuous useful action by allowing the off-chip prediction and L1D lookup to proceed in parallel rather than sequentially. The perceptron predictor continuously evaluates program features to generate predictions, and the system continuously validates these predictions through L1D lookup results, ensuring that accurate predictions never wait and maintain optimal performance.
3Loss of information
If conventional prefetching techniques are used to predict data location, then data location prediction capability is provided, but area overhead and complexity increase significantly
Solution Approach 1:
The system extracts only the essential off-chip prediction functionality from complex prefetching techniques. Instead of implementing full prefetching hardware with large tables and complex logic, the system uses a compact perceptron-based predictor that takes program features as input and outputs off-chip prediction decisions, significantly reducing area overhead while maintaining prediction capability.
Solution Approach 2:
The system replaces mechanical prefetching hardware (large prediction tables, complex comparison logic, multiple cache levels) with a perceptron-based neural network model that uses program features to make predictions. This substitution reduces the mechanical complexity and area overhead while providing adaptive prediction capability that can learn from program behavior patterns.
Data Source
Figure 1~2
Figure 3
AI summary
The present invention relates to a first-level perceptron (FLP) off-chip predictor communicatively connectable to a computing core and to a DRAM, wherein the core and the DRAM are communicatively connected through a multi-level cache hierarchy of levels L1D, L2C, ..., LLC. The FLP is advantageously adapted with an FLP off-chip prediction mechanism comprising two thresholds, Tlow and Thigh. The invention also relates to a two-level perceptron (TLP) off-chip predictor comprising a first-level perceptron (FLP) off-chip predictor according to any of the preceding claims; and a second-level perceptron (SLP) off-chip predictor communicatively connectable to a multi-level cache hierarchy of levels L1D, L2C, ..., LLC through a L1D prefetcher, wherein the multi-level cache hierarchy is communicatively connected to a computing core and to a DRAM.