Prefetch Buffer for Multi-Threaded Memory Addressing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-threaded processors, the limited capacity of translation lookaside buffers (TLBs) leads to a high miss rate when handling a large number of hardware threads, resulting in prolonged memory addressing times due to the inability to meet the demand for page table entries.
Innovation Solution
Implementing a method that stores page table entries not queried within a set time in a standby buffer and then transfers them to a prefetch buffer, allowing simultaneous querying of both buffers for virtual addresses to increase hit rates and reduce addressing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the TLB capacity is increased to store more page table entries for multiple hardware threads, then the hit rate for querying physical addresses is improved, but the device complexity and manufacturing cost increase
Solution Approach 1:
The patent divides the TLB into multiple independent TLB units, each dedicated to serving specific hardware threads. This segmentation allows each TLB unit to maintain its own page table entries without competing for space in a shared buffer, thereby maintaining high hit rates for multi-threaded operations without requiring a single oversized TLB that would increase device complexity and cost.
2Device complexity
If the TLB capacity is kept small to reduce device complexity, then the manufacturing cost is reduced, but the hit rate for querying physical addresses deteriorates when handling many hardware threads
Solution Approach 1:
Each TLB unit is designed to serve multiple hardware threads simultaneously, making the TLB structure universal rather than dedicated to a single thread. This multi-functionality allows a limited number of TLB units to collectively support many threads, achieving high hit rates without requiring prohibitively large total capacity.
3Reliability
If software or operating system searches for physical addresses from memory after TLB miss, then the page table entries can be updated, but the memory addressing time is prolonged
Solution Approach 1:
The system proactively loads page table entries into the TLB units before they are actually needed by predicting which threads will be accessed next. This preliminary action ensures that when threads do execute memory operations, their required page table entries are already cached in the TLB, eliminating the time-consuming memory search that would otherwise occur after a TLB miss.
Solution Approach 2:
The patent introduces a standby buffer as an intermediary structure between the TLB and main memory. When the TLB misses, the system first checks the standby buffer which holds recently evicted page table entries, avoiding immediate access to slow main memory. This intermediary layer significantly reduces the time penalty for TLB misses while maintaining accurate page table information.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention provides a method and an apparatus for querying a physical memory address, where the method includes: storing, into a prefetch buffer, page table entries of a second thread that is to perform addressing that are stored in a standby buffer, where the standby buffer stores page table entries that are not queried within a set time in a bypass translation buffer; receiving a memory addressing request message that carries a virtual address and is sent by the second thread; and querying, in the bypass translation buffer and the prefetch buffer, a corresponding physical address according to the virtual address. According to the method and the apparatus for querying a physical memory address provided in the present invention, a bypass translation buffer and a prefetch buffer are queried at the same time according to a to-be-addressed virtual address of a second thread, where the prefetch buffer stores page table entries of the second thread that are not queried within a set time in the bypass translation buffer, thereby increasing a hit rate of querying a physical address corresponding to a virtual address, and shortening an addressing time.