Compiler Metadata Driven Hardware Prefetch Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data prefetching methods in computer systems are limited by the need for explicit software prefetch instructions, which restrict flexibility and can lead to suboptimal cache hit rates due to inflexible prefetch timing and the requirement for multiple instructions for structure member variables.
Innovation Solution
A method where a compiler determines the start address of functions and global variables, allowing a prefetch engine to prefetch data into the cache based on these addresses without relying on software instructions, thereby improving prefetching flexibility and efficiency by targeting only used structure member variables and optimizing cache line usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If software prefetch instructions are inserted into functions, then data can be prefetched into cache, but the prefetching flexibility is limited and requires multiple instructions for structure member variables
Solution Approach 1:
The patent replaces the software-based prefetch instruction mechanism with a hardware-based automatic prefetch mechanism. The compiler generates metadata containing function start addresses and global variable addresses, and the hardware prefetch engine automatically retrieves data based on this metadata without requiring explicit software prefetch instructions. This substitution eliminates the need for multiple prefetch instructions while improving prefetching flexibility.
Solution Approach 2:
The patent introduces metadata as an intermediary between the compiler and the prefetch engine. The metadata contains the start address of functions and global variables, enabling the hardware prefetch engine to automatically determine what data to prefetch without requiring explicit software instructions. This intermediary structure resolves the contradiction by providing both automation (improving efficiency) and flexibility (through configurable metadata).
2Loss of time
If prefetch instructions are used with specified prefetch times, then data can be fetched in advance, but the prefetch timing is limited and cannot adapt to dynamic requirements
Solution Approach 1:
The patent implements dynamic prefetch timing by using the hardware prefetch engine to automatically determine prefetch moments based on runtime conditions. Instead of fixed prefetch times specified in software instructions, the system dynamically adjusts prefetch timing based on actual program execution flow and data access patterns, reducing data fetching delay while maintaining flexibility.
Solution Approach 2:
The hardware prefetch engine performs self-service by automatically monitoring program execution and initiating prefetch operations without external software instructions. The engine uses the metadata (function start addresses and global variable addresses) to autonomously determine when and what to prefetch, eliminating the need for manual prefetch timing specification while adapting to dynamic execution requirements.
3Reliability
If multiple prefetch instructions are used for structure member variables, then all required data can be prefetched, but the instruction complexity increases
Solution Approach 1:
The patent merges multiple prefetch operations into a single hardware-based prefetch mechanism. Instead of requiring separate prefetch instructions for each structure member variable, the hardware prefetch engine automatically prefetches all required data (functions and global variables) based on the consolidated metadata. This merging reduces instruction complexity while maintaining high cache hit rates through comprehensive prefetching.
Solution Approach 2:
The patent extracts the prefetch logic from software instructions and relocates it to hardware execution. By taking out the complex instruction-based prefetch mechanism and replacing it with hardware-based automatic prefetching driven by simple metadata, the system reduces instruction complexity while ensuring all necessary data is prefetched for reliable cache hits.
Data Source
Figure 1~3
Figure 4~6
Figure 7~8
AI summary
Embodiments of this application disclose a data prefetching method and apparatus that are applied to a computer system, and the computer system includes a prefetch engine, a memory, and a compiler. The method includes: The compiler performs the following operations in a compilation process: obtaining N functions and a first global variable of the N functions, where N is an integer greater than or equal to 1; and determining a start address of the N functions and a start address of the first global variable, so that the prefetch engine can prefetch, into a cache according to the start address of the N functions and the start address of the first global variable, data that is in the memory and that is associated with the first global variable. Therefore, data prefetching flexibility can be improved.