A timing prefetch enhancement method

Abstract

The application discloses a timing prefetch enhancement method, belonging to the technical field of data prefetching, and comprising the following steps: when a conventional address-address association pair cannot be constructed by an existing prefetcher, PC-address association pair metadata constructed based on a prefetch mechanism triggered by PC is used to trigger a prefetch request by PC; based on a metadata management mechanism, an address-address association table is used to contain effective address-address association pair metadata by inserting a strategy and eliminating a strategy; based on a combination mechanism of static foresight and dynamic degree prefetching, a fixed distance foresight is used to select a trigger address constituting the address-address association pair metadata, the prefetch precision of each instruction is monitored to adjust the prefetch degree of the instruction, and the prefetch is performed according to the prefetch degree of the current memory instruction; based on a dual-mode partition adjustment mechanism, the partition mode of the address-address association table is dynamically determined by monitoring the data cache utility and the prefetch utility, and partition adjustment is realized, so that the prefetch is enhanced.

Description

Technical Field

[0001] This invention belongs to the technical field of processor microarchitecture design, caching, and data prefetching, and specifically relates to a timing prefetching enhancement method. Background Technology

[0002] Modern caches typically integrate hardware prefetchers to record and learn the program's historical memory access information and predict future memory addresses. By loading future data from the underlying storage device into the target cache in advance, a cache hit can occur directly when the corresponding data is accessed in the future, thereby improving program performance.

[0003] Timing prefetchers record past access address sequences and prefetch subsequent access addresses based on previous records when the same address is accessed again in the future. For example, if a program previously accessed addresses X and Y, the prefetcher records this address sequence, constructs address-address pair metadata (X, Y), and prefetches address Y based on the metadata when the program accesses address X again in the future. Because timing prefetchers can also predict some complex and irregular access patterns, such as pointer chasing, they have great potential and have been successfully applied in real processor designs (such as the Arm Neoverse N2 series).

[0004] Currently, mainstream timing prefetchers consist of two main components. (1) Training table: A table indexed by the program counter (PC) used to track the last few memory access addresses of each PC and build address-address pairs. An address-address pair consists of two memory addresses in the form of (trigger address, target address). (2) Address-address association table: A table indexed by addresses used to store the address-address pairs generated by the training table. The storage overhead of the address-address association table is very large, reaching the MB level, because it needs to store complete access addresses and contain a sufficient number of entries. Therefore, currently, mainstream timing prefetcher designs allocate a portion of the L3 cache as the address-address association table.

[0005] The training process of the timing prefetcher is as follows: (1) During each memory access, assuming the current access address is Cur, the prefetcher will first look up the training table based on the PC of the current memory access. If the lookup is successful, the memory address Last of the last access of the PC is obtained. If Last ≠ Cur, an address-address pair (Last, Cur) can be constructed, indicating that Last is the trigger address and Cur is the target address. Subsequently, when the address Last is accessed again, a prefetch request for the target address Cur will be triggered. (2) The prefetcher uses Last as the key to insert the newly constructed address-address pair (Last, Cur) into the address-address pair table, so that the table entry can be found based on the address Last in the future. (3) The prefetcher updates the current training table, that is, updates the last access address of the PC to Cur.

[0006] Prefetching process: (1) Each time memory is accessed, the current access address is recorded as Cur. The prefetcher will look up the address-address association table based on the address Cur. (2) If the lookup is successful, a prefetch request is issued based on the found table entry; if the lookup fails, the memory access will not trigger a prefetch request.

[0007] To improve the timeliness of prefetching, existing technologies typically employ a multi-degree prefetching mechanism. This mechanism alters the prediction process of the prefetcher. Suppose that a prefetch address P1 is found based on the current address Cur. Multi-degree prefetching then searches the address-address association table again based on address P1 to find the next prefetch address P2, and so on, until no address is found or the degree limit is reached. If the degree is D, then the address-address association table is searched at most D times per operation.

[0008] In current processor designs, the size of the address-address association table is adjustable. Taking a 16-way L3 cache as an example, typically 0-8 ways are allocated to the address-address association table. The size of the address-address association table can be adjusted at a fine-grained level for each way. Some techniques employ dynamic adjustment methods during program execution, deciding to add or remove a way based on prefetch metadata hits and cache hits. Because each partition switch causes a change in the index's hash function, the same lookup key may map to different ways. Therefore, each switch requires rebuilding the index for all table entries and moving related entries. For example, when switching from 4 ways to 5 ways, the index, which previously used a modulo 4, now uses a modulo 5.

[0009] In summary, the existing technology has the following problems: (1) Existing technologies often fail to construct address-address pairs in certain workload scenarios, leading to ineffective prediction. There are two situations in which the prefetcher cannot construct address-address pairs: the number of memory access instructions in the program is much larger than the size of the training table, which causes some PCs to be frequently removed from the training table. During training, if the search for the training table fails, that is, if there is no record for the corresponding PC, the Last of that PC cannot be found, and thus the address-address pair cannot be constructed, and subsequent effective prefetching is also impossible; during training, if the record for the corresponding PC is found in the training table, but the Last is found to be Cur, the address-address pair cannot be constructed either. These scenarios are widespread in workloads such as cloud computing. These workloads have a large instruction footprint and contain a large number of instructions that repeatedly access the same address.

[0010] (2) The timeliness of prefetching in existing technologies is generally poor. Specifically, existing technologies often issue prefetch requests too late, and the data has not been returned when the program needs it, resulting in reduced benefits from prefetching.

[0011] (3) Existing technologies face metadata capacity pressure, and there are a large number of useless entries in the address-address association table.

[0012] (4) Existing technologies cannot quickly adopt a more suitable L3 partitioning scheme, and the partition adjustment overhead is large. Specifically, if the allocated address-address association table is too small, it may not be able to fully utilize the benefits of prefetching, resulting in suboptimal performance. If the allocated address-address association table is too large, and the benefits obtained by the program from prefetching cannot offset the performance loss caused by the compression of L3 data cache, the program's performance will actually decrease. Existing technologies adjust the allocation of only one way at a time. In addition, since the number of ways participates in the indexing process, each adjustment requires remapping and moving every metadata table entry. Summary of the Invention

[0013] In view of the above, the purpose of this invention is to provide a timing prefetch enhancement method, which enhances the prefetching of memory access addresses by the timing prefetcher from four aspects: a prefetching mechanism triggered by a PC, a combination of static look-ahead and dynamic degree prefetching, a metadata management mechanism, and a dual-mode partitioning adjustment mechanism.

[0014] To achieve the above-mentioned objectives, an embodiment provides a timing prefetch enhancement method, comprising the following steps: When the existing prefetcher cannot build regular address-address association pairs, PC-address association pair metadata is built based on the prefetch mechanism triggered by the PC, and the PC is used to trigger the prefetch request; Based on the metadata management mechanism, the address-address association table is enhanced to accommodate valid address-address association pair metadata through insertion and removal strategies. Based on a combination of static look-ahead and dynamic degree prefetching, a fixed-distance look-ahead is used to select the trigger address that constitutes the metadata of the address-address association pair. At the same time, the prefetching precision of each instruction is monitored to adjust the prefetching degree of the instruction, and prefetching is performed based on the prefetching degree of the current memory access instruction. Based on the dual-mode partition adjustment mechanism, the partition mode of the address-address association table is dynamically determined and partition adjustment is implemented by monitoring the data cache utility and prefetch utility.

[0015] Preferably, the step of constructing PC-address association metadata based on a PC-triggered prefetching mechanism and using the PC to trigger a prefetch request includes: Introduction and Update of PC Queue: A PC queue is introduced to store the sequence of PCs for the most recent non-repeating memory access instructions. When the currently accessed PC is not in the queue, the currently accessed PC is inserted at the end of the queue, and a first-in-first-out strategy is adopted to achieve queue management. When to build the PC-address association metadata: If a lookup in the training table fails, or if Last=Cur is found in the training table, then the construction of the association metadata is enabled. The method for constructing PC-address association pair metadata is as follows: the head position in the PC queue is selected as the trigger PC, the current memory access address Cur is used as the target address, the PC-address association pair metadata is constructed and inserted into the PC-address association table, and the trigger PC is used as the key. Prefetching process: The lookup of the PC-address association table is triggered by the event that the currently accessed PC successfully updates the PC queue. If the lookup is successful, a prefetching request is issued based on the target address stored in the found table entry.

[0016] Preferably, the PC-address association table is implemented by partitioning a portion of the L3 cache into a way, and the cache size is adjusted in the following ways: Define the sampling period length as L. The program will enter the sampling period in stages and count the number of times N that the prefetcher cannot build regular address-address association pairs. Calculate the density ρ=N / L of the address-address association pair construction failure and adjust the size allocation of the PC-address association table according to the relationship between ρ and a preset threshold.

[0017] Preferably, the insertion strategy filters out metadata entries that cannot generate useful prefetches. Specifically, after a certain instruction has consecutive cache hits, the address-address association metadata generated by the instruction is filtered out.

[0018] Preferably, the elimination strategy adopts a fine-tuned SHiP strategy, which adjusts the way the replacement strategy updates the state: the promotion operation is only performed when the exact same address-address association pair metadata is repeatedly inserted.

[0019] Preferably, the step of using fixed-distance look-ahead to select the triggering address constituting the address-address association pair metadata includes: Based on a fixed distance N, N additional Last fields are added to the training table entries to record historical access addresses. Based on a fixed distance N, an earlier access address is selected from the training table as the trigger address for address-address association of metadata.

[0020] Preferably, the step of monitoring the prefetch accuracy of each instruction to adjust the prefetch degree of the instruction includes: Add three fields to each training table entry: two counters for issuing prefetch and exact prefetch counts respectively, and one degree field; When the issued prefetch or the accurate prefetch is saturated, a field update is triggered: the prefetch precision is calculated by dividing the accurate prefetch by the issued prefetch; if the prefetch precision is greater than the high precision threshold, the degree field is increased; if the prefetch precision is less than the low precision threshold, the degree field is decreased; then both counters are reset.

[0021] Preferably, the prefetching based on the prefetch degree of the current memory access instruction includes: A filter table is introduced. Each time a prefetch request is made for the metadata of an address pair found in the address-address association table, the metadata of the address-address association pair is simultaneously inserted into the filter table. Before each lookup of the address-address association table, the filter table is searched first to find the current leading prefetch address. Then, the address-address association table is searched based on the leading address. When the total number of successful searches of the two tables equals the prefetch degree or the lookup of the address-address association table fails, the table lookup and prefetching are stopped.

[0022] Preferably, the step of dynamically determining the partitioning pattern of the address-address association table by monitoring data cache utility and prefetch utility includes: Based on the monitoring data caching utility and prefetch utility during the sampling period, and by calculating the utility value through a utility function, if the utility value is greater than 0, it indicates that the data caching utility is higher, and a conservative mode partitioning strategy is adopted, allocating less space to the address-address association table; if the utility value is less than or equal to 0, it indicates that the prefetch utility is higher, and an aggressive mode partitioning strategy is adopted, allocating more space to the address-address association table.

[0023] Preferably, the partition adjustment includes: When switching from aggressive mode to conservative mode, you only need to clear the multiple ways that originally stored metadata and allocate them to the data cache. The metadata of the one way that is retained does not need to be re-indexed, and all table entries in it can still be indexed normally in the future. When switching from conservative mode to aggressive mode, multiple more ways are directly allocated to metadata, and metadata remapping and relocation are not performed.

[0024] Compared with the prior art, the beneficial effects of the present invention include at least the following: The prefetching mechanism triggered by a PC proposed in this invention overcomes the problem that traditional time-series prefetchers cannot construct prefetch metadata when encountering training table lookup failures or Last=Cur cases. By capturing the correlation between the PC and the target address, and using the PC as a trigger, it effectively captures and predicts access patterns that traditional techniques cannot capture, reducing cache misses and improving application performance. This mechanism's optimization effect is particularly significant on some cloud computing workloads with very large instruction footprints.

[0025] The proposed mechanism combining static look-ahead and dynamic degree effectively improves the timeliness of the prefetcher. Static look-ahead ensures basic timeliness, while dynamic degree further enhances timeliness and improves performance on high-prediction-accuracy loads, while avoiding performance loss on low-prediction-accuracy loads. Improved timeliness reduces performance loss caused by delayed prefetching, enhances prefetching benefits, and improves program performance.

[0026] This invention proposes an efficient metadata management mechanism that filters out metadata that is prone to cache hits during insertion, effectively reducing the capacity pressure on associated tables. With the same capacity budget, more space can be used to store more useful metadata entries. For metadata replacement, a fine-tuned SHiP strategy is adopted, further leveraging the PC-based traffic splitting characteristic in time-series prefetching to improve the effectiveness of metadata management and retain useful metadata. Furthermore, by adjusting the update mechanism of the replacement state, the hit rate of useful metadata is maximized, and the retention of inaccurate metadata is reduced.

[0027] This invention proposes a dual-mode partition adjustment mechanism that enables the system to quickly adopt a more suitable partitioning strategy without requiring fine-grained adjustments one way at a time. Furthermore, it supports low-overhead partition adjustments without the need to remap and move all metadata entries.

[0028] Overall, this invention enhances time-series prefetching from multiple dimensions: (1) the PC-triggered prefetch path effectively broadens the prediction range of existing technologies, resulting in performance improvement; (2) the prefetching mechanism combining static look-ahead and dynamic degree effectively improves prefetching timeliness and enhances prefetching performance benefits; (3) the proposed metadata management mechanism allows the prefetcher to store more metadata that can bring useful prefetching under the same storage budget, improving prefetching performance; (4) the proposed dual-mode partitioning mechanism allows the system to quickly and effectively make trade-offs between L3 data cache and prefetch metadata cache, avoiding excessive cache occupation and performance degradation. By integrating the above-mentioned multiple optimization mechanisms, this invention effectively solves various performance problems faced by time-series prefetchers under different workloads, achieving performance optimization effects on various complex workloads; at the same time, the low-overhead insertion filtering mechanism and the dual-mode partitioning mechanism design also effectively reduce the system's storage overhead and hardware complexity. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of the timing prefetch enhancement method provided in the embodiment; Figure 2 This is a schematic diagram of the prefetcher and cache system provided in the embodiment; Figure 3 This is a schematic diagram of the data flow during the prefetcher training phase provided in the embodiment; Figure 4 This is a schematic diagram of the data flow during the prediction stage of the prefetcher provided in the embodiment. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0032] like Figure 1 As shown, the embodiment provides a timing prefetch enhancement method, including the following steps: S1, when the existing prefetcher cannot build a regular address association pair, construct PC-address association pair metadata in the form of (triggering PC, target address) based on the prefetch mechanism triggered by PC, and use PC to trigger the prefetch request.

[0033] In this embodiment, a PC-triggered prefetch mechanism (PCTP) is proposed. The core of this mechanism is a novel prefetch metadata format: (Trigger PC, Target Address). Specifically, when the timing prefetcher cannot construct address-address pair metadata, it instead constructs association pair metadata in the form of (Trigger PC, Target Address). This metadata means selecting the PC of a certain instruction as the prefetch trigger for a certain memory address. This metadata format opens up a new prefetch path, realizing the use of the PC to trigger prefetch requests. This effectively solves the problem of existing timing prefetchers failing when encountering scenarios where address-address pair construction is impossible, effectively covering access patterns that existing timing prefetchers cannot predict, and represents an enhancement to existing timing prefetchers.

[0034] In the specific implementation, two additional main components are introduced on the basis of the existing timing prefetcher. (1) PC Queue (PCQ): A first-in-first-out queue used to store the PC sequence of the most recently non-repeating memory access instructions. (2) PC-Address Table (PAT): A table used to store the metadata of PC-address pairs constructed by the PCTP mechanism. The PC-address table is similar to the address-address table in the existing prefetcher, and is also implemented by dividing a part of the L3 cache into a way. The difference is that the stored content is PC-address pairs in the form of (trigger PC, target address), with the trigger PC as the key. Each cache line is used to store 12 PAT entries, that is, 12 metadata in the form of (trigger PC, address). Lookup process: Hash the trigger PC, calculate the Set and Way of the L3 cache, obtain a certain cache line, and then read the corresponding cache line, and compare the Tag of the 12 pairs with the corresponding field of the trigger PC.

[0035] like Figure 3 As shown, the working principle of the PC-triggered prefetching mechanism includes: (1) PCQ Update: PCQ is updated on every memory access. An update only occurs if the currently accessed PC is not in the queue. This update rule effectively avoids conflicts between triggering PCs for different addresses in the subsequently constructed metadata, preventing different target addresses from sharing the same triggering PC. The update inserts the currently accessed PC at the tail of the queue. The queue uses a first-in, first-out (FIFO) strategy to track the PCs of recent memory access instructions. If the queue is full, the oldest PC at the head of the queue is removed first.

[0036] (2) Timing of PCTP metadata construction: During the training process of the existing prefetcher, if the search for the training table fails (i.e., the currently accessed PC is not recorded in the training table), or if Last=Cur is found in the training table, then the construction of PCTP metadata will be enabled.

[0037] (3) Construction method of PCTP metadata: Select the oldest PC in PCQ (i.e. the head position) as the trigger PC, and then use the current memory access address Cur as the target address to construct (trigger PC, Cur) association pair as metadata and insert it into the PC-address association table. In this way, the prefetch advance can be easily controlled by adjusting the depth of PCQ.

[0038] (4) PCTP prefetching process: It is not necessary to look up the PAT based on the current PC during every memory access. This invention uses the event that the currently accessed PC successfully updates the PCQ to trigger the PAT lookup. This event indicates that the currently accessed PC is different from all PCs in the PCQ. Using this event to trigger the PAT lookup can filter out many duplicate PAT lookups, because the loop structure in the program will cause a certain memory access instruction to be executed repeatedly in a short period of time, resulting in many duplicate PAT lookups. The prefetcher looks up the PAT based on the currently accessed PC. If the lookup is successful, it will issue a prefetch request based on the target address stored in the found entry.

[0039] In this embodiment, the depth of the PCQ determines the prefetch lead. A deeper PCQ results in better timeliness of the prefetch issued by the PCTP. However, a deeper PCQ also means a weaker correlation between the selected trigger PC and the target address, leading to lower accuracy in the prefetch issued by the PCTP. Through testing with a large number of workloads, this invention demonstrates that a PCQ depth of 8-12 achieves better optimization results.

[0040] In this embodiment, the PAT is similar to the address-address association table in existing prefetchers, and it is also implemented by allocating a portion of the L3 cache, such as... Figure 2 As shown. Since this reduces the available data cache capacity for the program, it needs to be allocated on demand to ensure that the prefetching benefits of PAT outweigh the cost of reduced data cache capacity. This invention proposes a runtime adaptive PAT size allocation adjustment scheme.

[0041] This PAT size allocation adjustment scheme is a sampling-based method, using the address-address pair construction failure density as the primary basis for adjustment. The sampling period length L is defined, measured in terms of demand accesses, such as 100,000 demand accesses. During the sampling period, the number N of times the prefetcher fails to construct regular address-address pairs is counted, i.e., cases where training table lookup fails or Last = Cur occur. ρ is defined as the address-address pair construction failure density, ρ = N / L.

[0042] The program periodically enters a sampling period and counts N. After each sampling period, ρ is calculated, and then the size allocation of PAT is adjusted according to the relationship between ρ and a preset threshold. For example, if ρ > 30%, a 2-way L3 cache is allocated to PAT; if 15% < ρ <= 30%, a 1-way L3 cache is allocated to PAT; if ρ <= 15%, no PAT is allocated, i.e., the PCTP prefetch path is disabled. In actual processor design, the specific threshold setting needs to be adjusted based on the microarchitecture design space exploration.

[0043] S2, based on the metadata management mechanism, enhances the address-address association table to accommodate valid address-address association pair metadata through insertion and removal strategies.

[0044] In this embodiment, a simple and efficient metadata management mechanism is provided to enhance the address-address association table (AAT)'s ability to accommodate metadata for valid address-address association pairs. This mechanism mainly consists of two parts: an insertion strategy and a removal strategy.

[0045] The insertion strategy filters out metadata entries that cannot generate useful prefetches. Specifically, if an instruction has consecutive cache hits (i.e., if the "consecutive hits" field of an instruction is saturated), the associated metadata pairs generated by that instruction are filtered out. If the cache is not saturated, no filtering is performed. This effectively reduces the insertion of useless metadata, alleviates the pressure on metadata capacity, and allows the prefetcher to store more useful metadata within the same budgeted memory space.

[0046] Specifically, a 3-bit consecutive hit field is added to each entry in the training table to detect consecutive memory access hits for each PC. Filtering is implemented using the following field update and filtering rules.

[0047] Field update rules: This field is updated with each memory access based on whether the current access hits the cache. If the memory access hits and it's not due to useful prefetching, the consecutive hit field corresponding to that PC will be incremented (unless it's already saturated). Otherwise, the field will be cleared to 0. Note that if the cache hit is due to prefetching, it should be cleared to 0, because without prefetching, this access would have been a miss.

[0048] Filtering rule: Prefetched metadata generated by PCs that have consecutively hit field saturation will be filtered and will not be added to any associated table (PAT or AAT). This is because consecutive field saturation hits mean that the PC's recent memory accesses have all been cache hits, indicating that its recent memory accesses are cache-friendly. Therefore, it can be predicted that future accesses to the same address will also likely result in cache hits, eliminating the need for prefetching and storing this metadata. This mechanism effectively reduces the insertion of useless metadata, alleviates metadata capacity pressure, and allows the prefetcher to store more useful metadata within the same budgeted memory space.

[0049] The eviction strategy adopts a fine-tuned SHiP strategy. Based on the characteristics of time-series prefetching scenarios, the state update method is adjusted: the promotion operation is only performed when the same identical association pair metadata is repeatedly inserted, avoiding the erroneous promotion of invalid metadata to a high-priority position and effectively ensuring the residence time of useful metadata.

[0050] SHiP (Signature-based Hit Predictor) is a replacement strategy primarily used for L3 cache. Its core idea is to use "signatures" to associate and predict cache line re-reference behavior. SHiP typically uses instruction probabilities (PCs) as signatures because they can effectively associate specific data access patterns. This aligns with the idea and behavior of using PCs to partition instruction access flows in the training table of time-series prefetchers, resulting in higher performance. Therefore, this invention adopts the SHiP strategy as the replacement strategy for the address-address association table. Based on this, the following fine-tuning optimizations were made for the characteristics of time-series prefetching scenarios: The traditional SHiP strategy aims to maximize cache hit rate by performing a boost operation (setting the rereference prediction value to 0) when a cache hit occurs and setting an initial value based on the signature table when a cache miss occurs. Directly applying this to prefetch metadata management results in maximizing metadata hit rate. This is not optimal because metadata hits do not necessarily produce useful prefetches and may even produce erroneous prefetches that harm performance. Therefore, a better time-series prefetch replacement strategy should aim to maximize the hit rate of useful metadata.

[0051] This invention cleverly achieves this by adjusting the replacement strategy to update the state. Specifically, this method uses a deferred update approach, not performing any updates during the lookup but delaying them until metadata insertion, i.e., only then performing the promotion operation. This is because in a time-series prefetching scenario, the current accessed address will inevitably be used as a key to be inserted into the address-address association table when the same instruction accesses it again in a subsequent instance.

[0052] Specific update rules: When inserting metadata, if the exact same metadata is inserted repeatedly (i.e., the metadata already exists in the table), it indicates that the resulting prefetch is accurate, and a promotion operation is performed. Otherwise, the initial value is set according to the signature table. Note that inserting metadata with the same key but different target addresses (this metadata will result in a lookup hit and would be promoted under traditional methods) indicates that the resulting prefetch is incorrect and should not be promoted.

[0053] This method effectively avoids erroneously promoting invalid metadata to high-priority positions, effectively ensuring the residence time of useful metadata, and maximizing the hit rate of useful metadata.

[0054] S3, based on a combination of static look-ahead and dynamic degree prefetching, uses a fixed-distance look-ahead to select the trigger address that constitutes the metadata of the address-address association pair, while monitoring the prefetching precision of each instruction to adjust the prefetching degree of the instruction.

[0055] In this embodiment, a combined mechanism of static look-ahead and dynamic degree prefetching is proposed. By adopting static look-ahead, the most basic prefetch timeliness is effectively guaranteed. At the same time, dynamic degree prefetching is introduced to improve prefetch accuracy. This enables the prefetcher to achieve better prefetch timeliness on some workloads with high prefetch accuracy and avoids performance loss on workloads with low prefetch accuracy.

[0056] In this context, lookahead refers to using an earlier access address as the trigger address instead of the last access address. The lookahead mechanism alters the prefetcher's training process. For example, with a lookahead distance of 2, each entry in the training table stores the addresses of the last three accesses of the instruction: Last0, Last1, and Last2. Instead of using Last0 as the trigger address, Last2 is used, and the constructed association pair metadata is (Last2, Cur). Each time an access is updated, these addresses are also updated sequentially (Last2=Last1, Last1=Last0, Last0=Cur).

[0057] Simulation analysis revealed that look-ahead effectively improves prefetch timeliness without reducing prefetch accuracy. Therefore, this invention employs a fixed look-ahead distance, which improves prefetch timeliness while avoiding the metadata learning cost associated with changing the look-ahead distance. The specific look-ahead distance setting needs to be determined based on the budget, as increasing the look-ahead distance increases the implementation overhead of the prefetcher (requiring the storage of more historical access addresses). In this embodiment, a static look-ahead distance of 2 is used, resulting in two additional Last fields added to each training table entry to record historical access addresses.

[0058] Furthermore, simulation analysis revealed that multi-degree prefetching effectively improves prefetch timeliness but leads to a significant decrease in prefetch accuracy. This invention proposes a method to dynamically adjust the prefetch degree based on the prefetch accuracy of each instruction. Specifically, three fields are added to each training table entry: two 6-bit counters (issued prefetch number and accurate prefetch number), and a 3-bit degree field. The issued prefetch number increments each time an instruction triggers a prefetch request, and the accurate prefetch number increments each time an accurate prefetch occurs. When either the issued or accurate prefetch number is saturated, a field update is triggered: the prefetch accuracy is calculated by dividing the accurate prefetch number by the issued prefetch number; if the prefetch accuracy is greater than a high-precision threshold (e.g., 0.75), the degree field is increased; if the prefetch accuracy is less than a low-precision threshold (e.g., 0.15), the degree field is decreased; subsequently, both counters are reset.

[0059] During each prediction, prefetching is performed based on the degree field of the current memory access instruction. In traditional schemes, multi-degree prefetching requires continuous lookups of the address-address association table located in the L3 cache, which introduces a large amount of redundant metadata traffic. For example, when the degree is 3, suppose at time T1 the program accesses address Cur and prefetches addresses P1, P2, and P3. At time T2, the program accesses address P1, and the prefetcher must again look up P2, P3, and P4 based on P1 and prefetch them. To reduce redundant metadata traffic and prefetch requests, this invention introduces a filter table to store the address-address association pair metadata obtained from the AAT when a prefetch request is recently issued, effectively reducing redundant metadata traffic. Figure 4 As shown, Each time a prefetch request is sent to the metadata based on the address found by AAT, the address-address association metadata is simultaneously inserted into the filter table. Before each lookup of the address-address association table, the filter table is searched first to find the current leading prefetch address, and then the address-address association table is searched based on the leading address. When the total number of successful searches of the two tables equals the prefetch degree or the lookup of the address-address association table fails, the table lookup and prefetching are stopped.

[0060] In the example above, at time T1, the filter table stores (P1, P2) and (P2, P3). At time T2, the prefetcher does not directly look up the address-to-address association table based on P1. Instead, it recursively looks up the filter table based on P1 to find the leading address P3, and then looks up the address-to-address association table based on P3. Because the filter table is very small (implemented as a 256-entry 8-way set-associative cache in this embodiment), and the address-to-address association table is relatively large and located in the L3 cache, the lookup latency of the filter table is much lower than that of the address-to-address association table. In this way, the filter table not only effectively improves the timeliness of prefetching but also effectively reduces L3 cache traffic caused by metadata lookups.

[0061] S4, based on a dual-mode partitioning adjustment mechanism, dynamically determines the partitioning mode for the address-address association table by monitoring data cache utility and prefetch utility.

[0062] In the embodiments, analysis of a large number of workloads revealed that the partition state at which the program achieves optimal performance is often at the boundary, i.e., under-allocation or over-allocation, depending on whether the workload is prefetch-friendly. Therefore, this invention proposes an efficient dual-mode partition adjustment mechanism. This mechanism abandons fine-grained partition adjustments for each way and sets only two modes for the size allocation of the address-address association table: aggressive mode (allocating 8-way) and conservative mode (allocating 1-way). This allows the system to quickly adopt better partitions without undergoing a lengthy adjustment process.

[0063] In this embodiment, a utility function is proposed to monitor data cache utility and prefetch utility to determine which partitioning mode to use. This enables the system to quickly adopt a more suitable L3 cache partitioning scheme and achieves low-overhead partition switching. Specifically, the utility function is represented as follows: .in, Represents relative utility score. η represents the demand access hit rate of the L3 data cache, and η represents the proportion of useful prefetches, which is the number of useful prefetches divided by the number of issued prefetches. κ is an introduced scaling factor, a constant used to reconcile the overhead and benefits of L3 data caching and prefetching. In the implementation, κ is set to 1.5.

[0064] In this embodiment, the system is initially set to aggressive mode, and then partition adjustments are performed periodically. Before each adjustment, a sampling period is initiated, which is synchronized with the sampling period for PAT size adjustment mentioned above. During the sampling period, the total number of L3 cache access requests, the number of access hits, the number of prefetches issued, and the number of useful prefetches are counted. After the sampling period ends, S is calculated, and then partition adjustments are performed according to the following adjustment rules: Adjustment rules: If S > 0, it indicates that the data cache has higher utility, so the system adopts a conservative partitioning strategy, allocating only a small amount of space (1 way) to AAT. If S ≤ 0, it indicates that prefetching has higher utility, so the system adopts an aggressive partitioning strategy, allocating more space (8 ways) to AAT. Furthermore, this partitioning adjustment mechanism is a positive feedback mechanism; each partitioning adjustment essentially allocates more space to the more efficient side and suppresses the less efficient side. This positive feedback mechanism effectively prevents jitter, ensuring that the next partitioning change is caused by a change in access patterns.

[0065] Specific implementation of partition adjustment: When switching from aggressive mode to conservative mode, simply clear the seven ways that originally stored metadata and allocate them to the data cache. The metadata of the one way that is retained does not need to be re-indexed, and all entries within it can still be indexed normally. When switching from conservative mode to aggressive mode, simply allocate seven more ways to metadata, again without remapping or moving the metadata. In this way, some entries in the metadata of the original one way will not be indexed, and these entries will be evicted over time by the eviction policy. This avoids the high overhead of remapping all metadata by sacrificing a small portion of metadata entries.

[0066] The entire partition switching process only involves simple capacity allocation and release, without rebuilding metadata table entry indexes or moving data.

[0067] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A time-series prefetch enhancement method, characterized in that, Includes the following steps: When the existing prefetcher cannot build regular address-address association pairs, PC-address association pair metadata is built based on the prefetch mechanism triggered by the PC, and the PC is used to trigger the prefetch request; Based on the metadata management mechanism, the address-address association table is enhanced to accommodate valid address-address association pair metadata through insertion and removal strategies. Based on a combination of static look-ahead and dynamic degree prefetching, a fixed-distance look-ahead is used to select the trigger address that constitutes the metadata of the address-address association pair. At the same time, the prefetching precision of each instruction is monitored to adjust the prefetching degree of the instruction, and prefetching is performed based on the prefetching degree of the current memory access instruction. Based on the dual-mode partition adjustment mechanism, the partition mode of the address-address association table is dynamically determined and partition adjustment is implemented by monitoring the data cache utility and prefetch utility. The step of dynamically determining the partitioning mode of the address-address association table by monitoring data cache utility and prefetch utility includes: calculating the utility value based on the monitored data cache utility and prefetch utility during the sampling period and using a utility function; when the utility value is greater than 0, it indicates that the data cache utility is higher, and a conservative partitioning strategy is adopted, allocating less space to the address-address association table; when the utility value is less than or equal to 0, it indicates that the prefetch utility is higher, and an aggressive partitioning strategy is adopted, allocating more space to the address-address association table.

2. The timing prefetch enhancement method according to claim 1, characterized in that, The method of constructing PC-address association metadata based on a PC-triggered prefetching mechanism and using the PC to trigger prefetch requests includes: Introduction and Update of PC Queue: A PC queue is introduced to store the sequence of PCs for the most recent non-repeating memory access instructions. When the currently accessed PC is not in the queue, the currently accessed PC is inserted at the end of the queue, and a first-in-first-out strategy is adopted to achieve queue management. When to build the PC-address association metadata: If a lookup in the training table fails, or if Last=Cur is found in the training table, then the construction of the association metadata is enabled. The method for constructing PC-address association pair metadata is as follows: the head position in the PC queue is selected as the trigger PC, the current memory access address Cur is used as the target address, the PC-address association pair metadata is constructed and inserted into the PC-address association table, and the trigger PC is used as the key. Prefetching process: The lookup of the PC-address association table is triggered by the event that the currently accessed PC successfully updates the PC queue. If the lookup is successful, a prefetching request is issued based on the target address stored in the found table entry.

3. The timing prefetch enhancement method according to claim 2, characterized in that, The PC-address association table is implemented by allocating a portion of the L3 cache as a way, and the cache size is adjusted in the following ways: Define the sampling period length as L. The program will enter the sampling period in stages and count the number of times N that the prefetcher cannot build regular address-address association pairs. Calculate the density ρ=N / L of the address-address association pair construction failure and adjust the size allocation of the PC-address association table according to the relationship between ρ and a preset threshold.

4. The timing prefetch enhancement method according to claim 1, characterized in that, The insertion strategy filters out metadata entries that cannot generate useful prefetches. Specifically, after a certain instruction has consecutive cache hits, the address-address association metadata generated by the instruction is filtered out.

5. The timing prefetch enhancement method according to claim 1, characterized in that, The elimination strategy adopts a fine-tuned SHiP strategy, which adjusts the way the replacement strategy updates the state: the promotion operation is only performed when the exact same address-address association pair metadata is repeatedly inserted, where the promotion operation means setting the rereference prediction value to 0.

6. The timing prefetch enhancement method according to claim 1, characterized in that, The method of using fixed-distance lookahead to select the triggering address that constitutes the metadata of the address-address association includes: Based on a fixed distance N, N additional Last fields are added to the training table entries to record historical access addresses. Based on a fixed distance N, an earlier access address is selected from the training table as the trigger address for address-address association of metadata.

7. The timing prefetch enhancement method according to claim 1, characterized in that, The method of adjusting the prefetch degree of each instruction by monitoring the prefetch accuracy includes: Add three fields to each training table entry: two counters for issuing prefetch and exact prefetch counts respectively, and one degree field; When the issued prefetch or the accurate prefetch is saturated, a field update is triggered: the prefetch precision is calculated by dividing the accurate prefetch by the issued prefetch; if the prefetch precision is greater than the high precision threshold, the degree field is increased; if the prefetch precision is less than the low precision threshold, the degree field is decreased; then both counters are reset.

8. The timing prefetch enhancement method according to claim 1, characterized in that, The prefetching based on the prefetch degree of the current memory access instruction includes: A filter table is introduced. Each time a prefetch request is made for the metadata of an address pair found in the address-address association table, the metadata of the address-address association pair is simultaneously inserted into the filter table. Before each lookup of the address-address association table, the filter table is searched first to find the current leading prefetch address. Then, the address-address association table is searched based on the leading address. When the total number of successful searches of the two tables equals the prefetch degree or the lookup of the address-address association table fails, the table lookup and prefetching are stopped.

9. The timing prefetch enhancement method according to claim 8, characterized in that, The partition adjustment includes: When switching from aggressive mode to conservative mode, you only need to clear the multiple ways that originally stored metadata and allocate them to the data cache. The metadata of the one way that is retained does not need to be re-indexed, and all table entries in it can still be indexed normally in the future. When switching from conservative mode to aggressive mode, multiple more ways are directly allocated to metadata, and metadata remapping and relocation are not performed.

Images