Data prefetching method and apparatus, storage medium, and SMS prefetcher

Abstract

The application relates to the pre-fetching technical field and discloses a data pre-fetching method, a data pre-fetching device, a storage medium and an SMS pre-fetcher. The data pre-fetching method comprises the following steps: when it is monitored that a primary cache sends a memory access address to a secondary cache, extracting a memory access footprint, an access frequency and a confidence degree corresponding to the memory access address from a mode history table; when the access frequency is 0, storing a virtual pre-fetch address generated according to the memory access footprint into a training table, calculating and updating the confidence degree corresponding to the memory access address in the mode history table based on the training table and the memory access address, and meanwhile, the access frequency is increased by 1; when the access frequency is greater than 0 and the confidence degree is greater than or equal to a preset confidence threshold, sending an actual pre-fetch address generated according to the memory access footprint to a tertiary cache to complete an actual pre-fetching operation, meanwhile, storing the actual pre-fetch address into the training table, calculating and updating the confidence degree corresponding to the memory access address in the mode history table based on the training table and the memory access address.

Description

Technical Field

[0001] The present invention relates to the field of prefetching technology, and in particular to a data prefetching method, apparatus and storage medium, and SMS prefetcher. Background Technology

[0002] The development of processor and memory technologies has been driven by different goals: processor development primarily pursues increased speed and frequency, while memory development focuses more on increasing capacity. This imbalance in the scale of technological development has led to the so-called "memory wall" phenomenon, where the performance gap between processors and memory is widening. To mitigate this trend, data prefetching technology has emerged as a key means to minimize the impact of the performance gap.

[0003] Spatial Memory Streaming (SMS) is an advanced spatial prefetcher designed for servers and scientific applications. Its core principle is to record the program's access patterns to memory regions to guide future prefetching. An ideal prefetching scheme accurately captures the program's memory access patterns and loads relevant data into the processor's cache before it is needed, reducing memory access latency and improving system performance. All prefetcher designs are based on a trade-off between two core metrics: prefetch coverage and prefetch accuracy. Some SMS prefetchers perform poorly in scenarios requiring high-precision prefetching and in certain test scenarios, such as cloud computing services, due to their low prefetch accuracy and the issuance of a large number of invalid prefetches. Furthermore, when bandwidth is limited, invalid prefetches can preempt bandwidth from normal access requests, causing the SMS prefetcher's performance to decrease rather than improve. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. To this end, one object of this invention is to provide a data prefetching method that can effectively control the number of prefetches issued by the prefetcher, prevent the issuance of a large number of invalid prefetch requests, and improve the performance of the SMS prefetcher.

[0005] The second objective of this invention is to provide a data prefetching device.

[0006] A third objective of this invention is to provide a computer-readable storage medium.

[0007] The fourth objective of this invention is to provide an SMS prefetcher.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a data prefetching method for an SMS prefetcher. The SMS prefetcher is coupled to a multi-level storage structure, which includes a main memory and a Level 1 cache, a Level 2 cache, and a Level 3 cache connected between the main memory and the processor core. The SMS prefetcher stores a pattern history table and a training table, including: when the Level 1 cache sends a memory access address to the Level 2 cache, extracting the memory access footprint, access count, and confidence level corresponding to the memory access address from the pattern history table; when the access count is 0, retrieving the data based on the... The virtual prefetch address generated by the memory access footprint is stored in the training table. Based on the training table and the memory access address, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated, and the access count is incremented by 1. When the access count is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold, the actual prefetch address generated based on the memory access footprint is sent to the L3 cache to complete the actual prefetch operation. At the same time, the actual prefetch address is stored in the training table, and based on the training table and the memory access address, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated.

[0009] According to the data prefetching method of the present invention, access counts and confidence levels are introduced into the pattern history table PHT, and the access counts and confidence levels in the pattern history table PHT are updated using a training table and memory access addresses. The prefetching operation corresponding to the pattern history table entry corresponding to the memory access address is determined, thereby controlling the number of prefetches issued by the prefetcher, preventing the issuance of a large number of invalid prefetch requests, and improving the performance of the SMS prefetcher.

[0010] In addition, the data prefetching method proposed in the above embodiments of the present invention may also have the following additional technical features:

[0011] According to an embodiment of the present invention, the SMS prefetcher stores a filter table and a cumulative table. Before extracting the memory access footprint, access count, and confidence level corresponding to the memory access address from the pattern history table, the method further includes: determining that there is an event identifier matching the memory access address in the pattern history table, wherein, when there is no event identifier matching the memory access address in the pattern history table, it is detected whether the cache block corresponding to the memory access address is accessed for the first time; if it is the first access, the cache block corresponding to the memory access address and the event identifier are stored in the filter table; if it is not the first access, a memory access footprint is generated according to the memory access address, and the memory access footprint and event identifier corresponding to the memory access address are stored in the cumulative table, wherein, when the memory access of the cache block corresponding to the memory access address ends, the information stored in the cumulative table is transferred to the pattern history table, and the access count and confidence level corresponding to the memory access address in the pattern history table are initialized.

[0012] According to one embodiment of the present invention, the method further includes: terminating prefetching when the number of accesses is greater than 0 and the confidence level is less than a preset confidence threshold.

[0013] According to an embodiment of the present invention, the entries in the training table include region codes and offset streams. The step of storing the virtual prefetch address generated based on the memory access footprint into the training table includes: determining a target region code based on the high-order bits of the virtual prefetch address; if the region code entry in the training table does not have a corresponding target region code, then adding a target region code to the region code entry in the training table; determining a target offset based on the low-order bits of the virtual prefetch address; and storing the target offset in the offset stream entry corresponding to the target region code; if the region code entry in the training table has a corresponding target region code, then determining the target offset based on the low-order bits of the virtual prefetch address; and storing the target offset in the offset stream entry corresponding to the target region code.

[0014] According to an embodiment of the present invention, the entries in the training table further include prefetch invalid values, prefetch valid values, and remaining training iterations. The step of calculating and updating the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address includes: when it is determined that a region code matching the high-order part of the memory access address exists in the training table, extracting the offset stream corresponding to the matching region code, the prefetch invalid value, the prefetch valid value, and the remaining training iterations; detecting whether there is an offset matching the low-order part of the memory access address in the offset stream corresponding to the matching region code; if so, the prefetch valid... The value is incremented by 1, the prefetched invalid value is decremented by 1, and the remaining training count is decremented by 1. Simultaneously, the matched offsets are cleared from the offset stream corresponding to the matched region encoding. The initial value of the prefetched valid value is 0, the initial value of the prefetched invalid value is the number of offsets in its corresponding offset stream, and the initial value of the remaining training count is the preset training count. If no such value exists, the remaining training count is decremented by 1. The remaining training count is monitored, and when the value of the remaining training count is 0, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated based on the prefetched valid value and the prefetched invalid value.

[0015] According to one embodiment of the present invention, the step of calculating and updating the confidence level corresponding to the memory access address in the pattern history table based on the prefetch valid value and the prefetch invalid value includes: calculating the ratio of the prefetch valid value to the total number of prefetch addresses, wherein the total number of prefetch addresses is the sum of the prefetch valid value and the prefetch invalid value; if the ratio is greater than a preset ratio threshold, the confidence level corresponding to the memory access address in the pattern history table is incremented by 1; if the ratio is less than the preset ratio threshold, the confidence level corresponding to the memory access address in the pattern history table is decremented by 1, wherein the minimum value of the confidence level is 0.

[0016] According to one embodiment of the present invention, when the processor core requests data from the L1 cache and a cache miss occurs in the L1 cache, the L1 cache sends the memory access address to the L2 cache.

[0017] To achieve the above objectives, a second aspect of the present invention provides a data prefetching device for an SMS prefetcher. The SMS prefetcher is coupled to a multi-level storage structure, which includes a main memory and a Level 1 cache, a Level 2 cache, and a Level 3 cache connected between the main memory and the processor core. The SMS prefetcher stores a pattern history table and a training table. The device includes: an extraction module, used to extract the memory access footprint, access count, and confidence level corresponding to the memory access address from the pattern history table when the Level 1 cache sends a memory access address to the Level 2 cache; and a virtual prefetching module, used to extract the memory access footprint, access count, and confidence level corresponding to the access address from the pattern history table when the access count is 0. When the access footprint is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold, the virtual prefetch address generated based on the access footprint is stored in the training table. Based on the training table and the access address, the confidence level corresponding to the access address in the pattern history table is calculated and updated. At the same time, the access count is incremented by 1. The actual prefetch module is used to send the actual prefetch address generated based on the access footprint to the L3 cache to complete the actual prefetch operation when the access count is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold. At the same time, the actual prefetch address is stored in the training table. Based on the training table and the access address, the confidence level corresponding to the access address in the pattern history table is calculated and updated.

[0018] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the data prefetching method as proposed in the first aspect of the present invention.

[0019] To achieve the above objectives, a fourth aspect of the present invention provides an SMS prefetcher, including a storage unit and a logic unit. The storage unit stores a computer program, and when the computer program is executed by the logic unit, it implements the data prefetching method as proposed in the first aspect of the present invention.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a prefetcher according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart of a data prefetching method according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of a filter table, an accumulation table, and a pattern history table according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart of generating a prefetch address according to a specific embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of a training table according to an embodiment of the present invention;

[0026] Figure 6 This is a flowchart illustrating the training process of a training table according to an embodiment of the present invention.

[0027] Figure 7 This is a schematic diagram of a data prefetching device according to an embodiment of the present invention. Detailed Implementation

[0028] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0029] It's important to note that prefetch coverage refers to the percentage of cache misses a prefetcher successfully covers. Prefetch accuracy refers to how much of the data successfully loaded by the prefetcher is actually used by the processor. These two metrics often present a contradictory relationship: improving one may come at the expense of the other. For example, when a program first accesses a new memory region, a simple prefetcher might predict that all data in that region will be used. While this strategy can achieve 100% coverage, its accuracy can be very low because most of the prefetched data may not actually be accessed. This not only wastes cache capacity but also consumes bandwidth resources, potentially negatively impacting performance. On the other hand, an overly conservative prefetcher might not prefetch any data at all, thus avoiding wasted cache and bandwidth, but this also results in 0% prefetch coverage, offering no performance optimization. Therefore, prefetcher design needs to strike a delicate balance between coverage and accuracy to maximize its positive impact on performance.

[0030] The core working principle of the SMS prefetcher is to record the access patterns of programs to memory regions to guide future prefetching. When a program first accesses a memory region, the SMS prefetcher begins monitoring and recording the access behavior to that region. Once the program stops using the region (i.e., when the block associated with that region in the cache is evicted), the SMS prefetcher stores the recorded access information in a metadata table called the PHT (Pattern History Table). The information in the PHT is stored in the form of "events" and "patterns." An "event" refers to the condition that triggers the access pattern; the SMS prefetcher combines the PC (Program Counter) and Offset as an event identifier. A "pattern" reflects the usage of each cache block in the memory region and is stored as a bit vector. Each bit in the vector indicates whether the corresponding cache block has been accessed recently (1 indicates access, 0 indicates no access). When the event occurs again (i.e., when the same PC+Offset occurs), the SMS prefetcher uses pattern information in the Pattern History Table (PHT) to predict which cache blocks will be used, and only issues prefetch requests for blocks marked "1" in the vector, thereby improving efficiency and reducing unnecessary data loading. This design significantly improves system performance by recording access history and making efficient use of it when triggered in the future, while reducing the waste of cache capacity and bandwidth.

[0031] However, the SMS prefetcher has a serious problem: when dealing with data structures that heavily rely on pointer chain tracing, using only PC+Offset as the trigger event is clearly too simplistic and imprecise. This design cannot accurately capture complex memory access patterns, leading to the prefetching of many invalid addresses. Furthermore, for some programs, their access patterns to specific memory regions may lack sufficient regularity and repeatability, further exacerbating the problem. In both cases, the SMS prefetcher exhibits high coverage but extremely low accuracy. The root cause is its inability to control the number of prefetches issued, resulting in a large number of invalid prefetch requests. This not only wastes the processor's valuable bandwidth resources but also occupies limited on-chip cache space, leading to cache pollution. The result is not only a failure to improve performance but may also significantly reduce overall system efficiency.

[0032] To address the aforementioned problems, embodiments of the present invention provide a data prefetching method, apparatus, storage medium, and SMS prefetcher. The data prefetching method, apparatus, storage medium, and SMS prefetcher of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] The data prefetching method in this embodiment of the invention is used in an SMS prefetcher. For example... Figure 1 As shown, the SMS prefetcher is coupled to a multi-level memory structure, which includes main memory and L1, L2, and L3 caches connected between main memory and processor core.

[0034] It's important to note that the processor core (CPU core) accesses data in a "nearest to furthest" order. Specifically, the processor core first checks the L1 cache. If the data is found (cache hit), it uses it directly, which is the fastest method. If the L1 cache is missed, it checks the L2 cache. If the L2 cache is missed, it uses it; if the L2 cache is missed, it uses it; this is faster than main memory. Only if the L3 cache is missed will the processor core access main memory.

[0035] The SMS prefetcher in this embodiment of the invention stores a pattern history table and a training table.

[0036] The core structure of the data prefetching method in this invention consists of two main parts: a prefetching part and a training part. The prefetching part captures memory access patterns based on a pattern history table, while the training part trains the data using a training table and updates the confidence level in the pattern history table based on memory access addresses.

[0037] This invention introduces confidence level and access count into the Pattern History Table (PHT). Therefore, in this invention, the entries in the Pattern History Table (PHT) include an event identifier (PC+Offset), a memory access footprint (Pattern), access count, and confidence level. This invention updates the confidence level and access count in the Pattern History Table (PHT) based on the training table and memory access addresses.

[0038] To prevent the prefetcher from performing a large number of invalid prefetches, this invention establishes a training table within the prefetcher. The confidence level is based on this training table and the historical table of memory access addresses used for dynamic training patterns. Based on this training table, this invention can accurately record the actual usage of memory access addresses, providing the prefetcher with a precise basis for confidence level adjustment, dynamically evaluating and providing timely feedback on the adjusted confidence level.

[0039] The SMS prefetcher in this embodiment of the invention also stores a filter table (FT) and an accumulation table (AT). The filter table (FT) records the cache block (4KB cache area) accessed for the first time, along with its corresponding program counter (PC) and the offset of the first access. When this cache block is accessed a second time, it is added to the accumulation table (AT). The accumulation table (AT) accumulates the memory access patterns generated by this cache block. When multiple accesses occur within the cache block accessed for the first time, the accumulation table (AT) begins to accumulate these accesses, forming the memory access patterns within the cache block. When the prefetcher determines that the memory access of this cache block has ended, it transfers the entries from the accumulation table (AT) to the pattern history table (PHT).

[0040] Figure 2 This is a flowchart of a data prefetching method according to an embodiment of the present invention. Figure 2 As shown, data prefetching methods may include:

[0041] S101, when it is detected that the L1 cache sends a memory access address to the L2 cache, extract the memory access footprint, access count and confidence level corresponding to the memory access address from the pattern history table;

[0042] S102, when the number of accesses is 0, the virtual prefetch address generated based on the memory access footprint is stored in the training table, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated based on the training table and the memory access address, and the number of accesses is incremented by 1.

[0043] S103, when the number of accesses is greater than 0 and the confidence level is greater than or equal to the preset confidence threshold, the actual prefetch address generated based on the memory access footprint is sent to the level 3 cache to complete the actual prefetch operation. At the same time, the actual prefetch address is stored in the training table. Based on the training table and the memory access address, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated.

[0044] The data prefetching method in this embodiment of the invention generates a prefetch address stream by monitoring the memory access address stream sent from the level 1 cache to the level 2 cache.

[0045] In one embodiment of the present invention, when the processor core requests data from the L1 cache and a cache miss occurs in the L1 cache, the L1 cache sends the memory access address to the L2 cache.

[0046] Specifically, when the CPU core requests data from the L1 cache and a cache miss occurs in the L1 cache, the L1 cache sends the memory access address to the L2 cache.

[0047] In this embodiment of the invention, when capturing the memory access address stream sent from the Level 1 cache to the Level 2 cache, each access address is simultaneously transmitted to both the pattern history table and the training table. The invention queries the pattern history table (PHT) to check if there exists an event identifier corresponding to the access address, and the training table to check if there exists a region code corresponding to the high-order part of the access address.

[0048] Specifically, when an event identifier corresponding to the access address exists in the pattern history table PHT, the memory access footprint, access count, and confidence level corresponding to the memory access address are extracted from the pattern history table PHT.

[0049] If the access count is 0, it means that the entry corresponding to the memory access address has just been transferred from the cumulative table AT to the pattern history table PHT, and no actual prefetch observation has been performed yet. At this time, the confidence level of the memory access address is unknown. This embodiment of the invention addresses the problem of unknown confidence level during the first prefetch of the SMS prefetcher by using a virtual prefetch method. Specifically, a virtual prefetch address is generated based on the memory access footprint. The virtual prefetch address with unknown confidence is stored in the training table and not directly sent to the L3 cache. After verifying the effectiveness of the virtual prefetch address by observing the actual access situation based on the training table and the memory access address, the confidence level corresponding to the memory access address in the pattern history table is updated, reducing invalid operations in the initial prefetch. When updating the confidence level corresponding to the memory access address in the pattern history table, the access count corresponding to the memory access address extracted from the pattern history table is incremented by 1.

[0050] If the number of accesses is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold, an actual prefetch address is generated based on the memory access footprint and sent to the L3 cache to complete the actual prefetch operation. Simultaneously, the actual prefetch address is stored in the training table, and the confidence level corresponding to the memory access address in the Pattern History Table (PHT) is calculated and updated based on the training table and the memory access address. The confidence level in the Pattern History Table (dynamically updated) is evaluated in real time to determine whether to initiate a prefetch. When the confidence level is high (greater than or equal to the preset confidence threshold), the prefetch operation is performed. When the confidence level is insufficient (less than the preset confidence threshold), the prefetch operation is reduced or suspended, effectively avoiding invalid prefetching and improving system performance.

[0051] It should be noted that when calculating and updating the confidence level corresponding to the memory access address in the pattern history table PHT based on the training table and the memory access address after storing the actual prefetch address in the training table, the access count corresponding to the memory access address extracted from the pattern history table can be incremented by 1, or it can be left unincremented. The comparison of the embodiments of the present invention is not limited.

[0052] To prevent the SMS prefetcher from generating a large number of invalid prefetches, the data prefetching method in this embodiment of the invention adopts a dynamic confidence adjustment mechanism. The access count and confidence are introduced into the pattern history table PHT, and the access count and confidence in the pattern history table PHT are updated using the training table and the memory access address. The corresponding prefetch operation is determined according to the pattern history table entry corresponding to the memory access address. This can effectively control the number of prefetches issued by the prefetcher, prevent a large number of invalid prefetch requests, and improve the performance of the SMS prefetcher.

[0053] In one embodiment of the present invention, the SMS prefetcher stores a filter table FT and a cumulative table AT. Before extracting the memory access footprint, access count, and confidence level corresponding to the memory access address from the pattern history table PHT, the data prefetching method may further include:

[0054] Determine if there is an event identifier in the pattern history table PHT that matches the memory access address. If there is no event identifier in the pattern history table PHT that matches the memory access address, check whether the cache block corresponding to the memory access address is being accessed for the first time.

[0055] If it is the first access, the cache block and event identifier corresponding to the memory access address are stored in the filter table FT;

[0056] If it is not the first access, a memory access footprint is generated based on the memory access address. The memory access footprint and event identifier corresponding to the memory access address are stored in the cumulative table AT. When the memory access of the cache block corresponding to the memory access address ends, the information stored in the cumulative table AT is transferred to the pattern history table PHT, and the access count and confidence level corresponding to the memory access address in the pattern history table are initialized.

[0057] Specifically, when a memory access address stream is captured from the Level 1 cache to the Level 2 cache, the pattern history table PHT is queried to see if there is an event identifier corresponding to the access address.

[0058] If an event identifier (PC+Offset) matching the memory access address exists in the schema history table PHT, then the memory access footprint, access count, and confidence level corresponding to the memory access address are extracted from the schema history table PHT, and the above prefetching process is executed based on the access count and confidence level.

[0059] If there is no event identifier (PC+Offset) in the schema history table PHT that matches the memory access address, then check whether the cache block corresponding to the memory access address is being accessed for the first time.

[0060] If the cache block corresponding to the memory access address is being accessed for the first time, then the cache block corresponding to the memory access address and the event identifier are stored in the filter table FT (see [link]). Figure 3 ).

[0061] If the cache block corresponding to the memory access address is not being accessed for the first time, a memory access pattern is generated based on the lower-order bits of the memory access address. The memory access pattern and event identifier (PC+Offset) corresponding to the memory access address are then stored in the cumulative table AT (see [link to table]). Figure 3 When the access to the cache block corresponding to the access address ends, the access pattern and access history corresponding to the access address stored in the cumulative table AT are transferred to the pattern history table PHT, and the access count and confidence level corresponding to the access address in the pattern history table are initialized to 0.

[0062] In this embodiment of the invention, the primary function of the filter table FT is to filter cache blocks that will only be accessed once. When the SMS prefetcher captures the memory access address sent from the L1 cache to the L2 cache, the filter table FT records the new cache block accessed for the first time, and also records the program counter PC and the offset of the first access. When this 4KB area is accessed a second time, it is added to the accumulation table AT.

[0063] In this embodiment of the invention, the accumulator table AT is responsible for accumulating memory access footprints. When multiple accesses occur within the same 4KB region, the accumulator table AT begins to accumulate these accesses, forming the memory access footprints within that region. When the prefetcher determines that the memory access in that region has ended, the entries in the accumulator table AT are transferred to the mode history table PHT.

[0064] In this embodiment of the invention, the Pattern History Table (PHT) is responsible for storing the event identifier (PC+Offset), memory access footprint (4KB Pattern), access count, and confidence level (see [link to relevant documentation]). Figure 3 Based on the confidence level, determine whether to generate a specific prefetch address stream.

[0065] In this embodiment of the invention, the access count is a 1-bit flag used to indicate whether an entry has just been transferred from the cumulative table AT. Since entries newly transferred to the pattern history table PHT have not yet undergone real observation, their confidence is unknown and they may contain a large number of invalid prefetches. Therefore, further observation and training are needed to confirm their reliability. Confidence is used to quantify the confidence of each entry in the pattern history table PHT, representing whether the prefetch addresses contained in that entry exhibit a high hit rate in historical accesses. When the prefetch footprint is transferred from the cumulative table AT to the pattern history table PHT, both the access count and confidence are initialized to 0.

[0066] In one embodiment of the present invention, the data prefetching method may further include:

[0067] Prefetching will terminate when the number of accesses is greater than 0 and the confidence level is less than the preset confidence threshold.

[0068] Specifically, if the number of accesses is greater than 0, it means that the entry has been observed to a certain extent, and the process proceeds to the next step of confidence level determination. If the confidence level is less than the preset confidence threshold, it means that the confidence level is insufficient, and the prefetching process is terminated to avoid generating invalid prefetch addresses.

[0069] The process of generating the prefetch address by the prefetcher in this embodiment of the invention is as follows: Figure 4 As shown. Specifically, after capturing the memory access address sent from the L1 cache to the L2 cache, the prefetcher accesses the entry in the Pattern History Table (PHT) based on the PC and Offset corresponding to the memory access address to check if the PHT table has been hit. If it has, the prefetcher extracts the memory access footprint, access count, and confidence level from the entry as the basis for subsequent prefetch address generation. It checks if the extracted access count is 0. If the access count is 0, it means the entry has just been transferred from the Accumulation Table (AT) to the Pattern History Table (PHT) and has not yet undergone actual prefetching observation. At this time, the prefetcher generates a "virtual prefetch," i.e., generates a virtual prefetch address. This virtual prefetch address is not directly sent to the L3 cache but is only stored in the training table. Based on the training table and the memory access address, an actual access observation is performed to calculate the confidence level. If the access count is greater than 0, it means the entry has undergone a certain amount of observation and proceeds to the next step of confidence level determination. If the confidence level is greater than or equal to 1 (the preset confidence threshold), it indicates that the entry has high reliability. The prefetcher will generate the actual prefetch address stream and send it to the L3 cache to complete the actual prefetch operation. If the confidence level is less than 1, the prefetch process will be terminated to avoid generating invalid prefetch addresses.

[0070] In one embodiment of the present invention, the entries of the training table include region encoding and offset stream, and the virtual prefetch address generated based on the memory access footprint is stored in the training table, which may include:

[0071] The target region code is determined based on the high-order part of the virtual prefetch address;

[0072] If the region encoding entry in the training table does not have a corresponding target region encoding, the target region encoding is added to the region encoding entry in the training table, and the target offset is determined based on the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region encoding.

[0073] If a corresponding target region code exists in the region coding entry of the training table, the target offset is determined based on the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region code.

[0074] In this embodiment of the invention, the main function of the training table is to store the virtual prefetch address and the actual prefetch address, and to update the confidence level corresponding to the memory access address in the pattern history table by observing the future actual access of the prefetch address (virtual prefetch address and actual prefetch address) based on the memory access address.

[0075] Specifically, when storing the virtual prefetch address into the training table, the target region encoding is determined based on the high-order bits of the virtual prefetch address.

[0076] If a region encoding entry in the training table does not have a corresponding target region encoding, then the target region encoding is added to the region encoding entry in the training table. The target offset is determined based on the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region encoding.

[0077] If the region encoding entry in the training table has a corresponding target region encoding, then the target offset is determined based on the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region encoding.

[0078] It should be noted that the process of storing the actual prefetch address into the training table is the same as the process of storing the virtual prefetch address into the training table, and will not be described again here.

[0079] In one embodiment of the present invention, the entries in the training table further include prefetch invalid values, prefetch valid values, and remaining training iterations. Calculating and updating the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address may include:

[0080] When it is determined that there is a region code in the training table that matches the high-order part of the memory access address, the offset stream, prefetch invalid value, prefetch valid value and remaining training times corresponding to the matching region code are extracted.

[0081] Check if there is an offset in the offset stream corresponding to the matching region encoding that matches the low-order part of the memory access address;

[0082] If it exists, increment the prefetch invalid value by 1, decrement the prefetch valid value by 1, and decrement the remaining training count by 1. At the same time, clear the matched offset from the offset stream corresponding to the matching region encoding. The initial value of the prefetch valid value is 0, the initial value of the prefetch invalid value is the number of offsets in its corresponding offset stream, and the initial value of the remaining training count is the preset training count.

[0083] If it does not exist, then the remaining training attempts are reduced by 1;

[0084] Monitor the value of the remaining training iterations. When the value of the remaining training iterations is 0, calculate and update the confidence level corresponding to the memory access address in the pattern history table based on the invalid and valid prefetch values.

[0085] In this embodiment of the invention, the training table is responsible for storing the virtual prefetch address stream and tracking the usage of the actual prefetch address stream. Its usefulness is determined by observing whether the actual prefetch address is accessed in the future, and the confidence of the pattern history table is updated by calculating the proportion of useful prefetches.

[0086] The training table in this embodiment of the invention is as follows: Figure 5 As shown, the training table entries include region code (RegionNumber), offset stream (Offset1, Offset2, ...), prefetch invalid value (USELESS), prefetch valid value (USEFUL), and remaining training iterations (ACCESS_Num).

[0087] The region code represents the high-order bits of all prefetch addresses in a prefetch operation (excluding the address bits in the lower 4KB range of the virtual prefetch address), used to indicate the 4KB region to which the prefetch operation belongs. Since the prefetcher of this invention limits each generated prefetch address to a 4KB range, prefetch addresses issued in a single operation share the same region code, allowing indexing of a previous prefetch stream through this field. For example, the high-order bits of all prefetch addresses in a single prefetch operation are identical because all prefetch addresses fall within this 4KB range. For instance, assuming the prefetch region is 100 (decimal), the prefetch addresses could be 1105, 1188, and 1199, with the high-order bits of each prefetch address being 11.

[0088] Offset Stream: Represents the low-order offset of the prefetch address within a 4KB region. The complete memory address can be reconstructed by concatenating the region code with a given offset. Invalid Prefetch Value: Represents the number of offsets in the current prefetch stream that have not yet been accessed. When a new prefetch stream is stored in the training table, the initial value of the invalid prefetch value is the number of addresses (offsets) contained in that prefetch stream. Valid Prefetch Value: Represents the number of offsets in the current prefetch stream that have been accessed, reflecting the validity of the prefetch address. Remaining Training Counts: Represents the remaining training counts for the current entry. The initial value of the remaining training counts is the preset training count. The preset training counts can be 100, 500, etc., and this embodiment of the invention does not limit the value of the preset training counts.

[0089] In this embodiment of the invention, when a memory access address stream is captured from the Level 1 cache to the Level 2 cache, the training table is searched to see if there is a region code that matches the high-order part of the access address.

[0090] If no matching region code is found, the training ends immediately. If a matching region code is found, it means that the memory access address is associated with a previous prefetch stream. Then, a further check is performed to see if there is a matching offset. If both the region code and offset match, it means the prefetch address completely predicted the future access behavior. In this case, the USEFUL counter is incremented by 1 (valid prefetch value), the USELESS counter is decremented by 1 (invalid prefetch value), and the ACCESS counter is also decremented by 1 (remaining training iterations). If only the region code matches but the offset does not, only the remaining training iterations are decremented by 1, indicating that the 4KB region to which the prefetch stream belongs was accessed but not hit.

[0091] Monitor the remaining training iterations. When the remaining training iterations are 0, calculate and update the confidence level corresponding to the memory access address in the pattern history table based on the invalid and valid prefetch values. If the remaining training iterations are not reduced to 0, it indicates that the prefetch stream has not yet completed training and continues to wait for subsequent memory access addresses to observe it.

[0092] In one embodiment of the present invention, calculating and updating the confidence level corresponding to the memory access address in the mode history table based on the prefetch invalid value and the prefetch valid value may include:

[0093] Calculate the ratio of valid prefetched values ​​to the total number of prefetched addresses, where the total number of prefetched addresses is the sum of invalid prefetched values ​​and valid prefetched values;

[0094] If the ratio is greater than the preset ratio threshold, the confidence level of the memory access address in the pattern history table is incremented by 1;

[0095] If the ratio is less than the preset ratio threshold, the confidence level of the memory access address in the pattern history table is reduced by 1, where the minimum confidence level is 0.

[0096] Specifically, the ratio of USEFUL to the total number of prefetched addresses (the sum of invalid and valid prefetched values) is calculated. This ratio is compared to a preset ratio threshold. If the ratio is greater than the preset ratio threshold, the confidence level corresponding to the memory access address in the PHT table entry is incremented by 1. If the ratio is less than the preset ratio threshold, the confidence level corresponding to the memory access address in the PHT table entry is decremented by 1 (the minimum confidence level is 0).

[0097] The specific training process of the training table in this embodiment of the invention is as follows: Figure 6 As shown, the high-order bits of the memory access address are compared with the region code in the training table. If no match is found, the training ends immediately. If a match is found, the corresponding offset stream and the values ​​of the USEFUL, USELESS, and ACCESS_Num counters are extracted. Offset matching is checked: If the offset matches successfully, it means the complete address match is successful. At this time, the USEFUL counter is incremented by 1, the USELESS counter is decremented by 1, and the ACCESS counter is also decremented by 1, while the offset is cleared. If the offset does not match successfully, only the ACCESS counter is decremented by 1, indicating that the prefetched address was not hit. The ACCESS counter status is checked: When the ACCESS counter reaches 0, it means that all addresses in the prefetch stream have been observed. At this time, the ratio of the USEFUL counter value to the total number of prefetched addresses (the sum of the USEFUL counter value and the USELESS counter value) is calculated and compared with a preset ratio threshold. If the confidence level is greater than the preset ratio threshold, the confidence level of the PHT entry in the prefetcher is incremented by 1; if it is less than the preset ratio threshold, the confidence level is decremented by 1 (the minimum confidence level is 0). If the ACCESS counter does not decrease to 0, it means that the prefetch stream has not yet completed training and continues to wait for subsequent memory access addresses to observe it.

[0098] The data prefetching method of this invention introduces a dynamic confidence mechanism based on the SMS prefetcher. Combining access count marking and confidence quantification, it dynamically evaluates and updates each entry in real time. Compared to the limitations of the SMS prefetcher in controlling invalid prefetches, this invention significantly reduces the generation of invalid prefetches and substantially improves prefetch accuracy. In bandwidth-constrained scenarios, this invention effectively reduces prefetch requests from prefetching requests that preempt normal memory access requests, thereby reducing power consumption, optimizing on-chip cache space utilization, and achieving a significant performance improvement within a limited area.

[0099] The data prefetching method of this invention proposes a virtual prefetching mechanism to accurately improve prefetching reliability. Addressing the problem of blind prefetching due to insufficient confidence in the initial stage of traditional prefetchers, this invention proposes a virtual prefetching mechanism. By designing a "virtual prefetch" function for entries newly transferred to the PHT (Programmable Data Set), the risk of entries with unknown confidence directly triggering actual prefetching is avoided, thus achieving a good balance between prefetching accuracy and system efficiency.

[0100] The data prefetching method of this invention proposes a complete and efficient prefetcher training mechanism, solving the problem of the lack of a method for evaluating the confidence of the prefetcher. Through efficient matching based on region coding, it can accurately associate historical prefetch streams, and achieve flexible training control by dynamically adjusting the preset number of training iterations, thus improving training efficiency and shortening training time. Simultaneously, the training table can monitor the prefetch stream in real time and promptly provide feedback on the confidence optimization results, significantly improving the prefetcher's adaptability in complex memory access scenarios, thereby achieving a comprehensive improvement in the overall system performance.

[0101] This invention provides a data prefetching device.

[0102] The data prefetching device of this invention is used for an SMS prefetcher, which is coupled to a multi-level storage structure, including a main memory and a level 1 cache, a level 2 cache and a level 3 cache connected between the main memory and the processor core.

[0103] The SMS prefetcher in this embodiment of the invention stores a pattern history table and a training table.

[0104] Figure 7 This is a schematic diagram of a data prefetching apparatus according to an embodiment of the present invention. Figure 7 As shown, the data prefetching device 100 may include:

[0105] Extraction module 10 is used to extract the memory access footprint, access count and confidence level corresponding to the memory access address from the pattern history table when it is detected that the L1 cache sends a memory access address to the L2 cache.

[0106] The virtual prefetch module 20 is used to store the virtual prefetch address generated based on the memory access footprint into the training table when the number of accesses is 0, calculate and update the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address, and increment the number of accesses by 1 at the same time.

[0107] The actual prefetch module 30 is used to send the actual prefetch address generated based on the memory access footprint to the level 3 cache to complete the actual prefetch operation when the number of accesses is greater than 0 and the confidence level is greater than or equal to 1. At the same time, it stores the actual prefetch address in the training table, calculates and updates the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address.

[0108] It should be noted that other specific embodiments of the data prefetching device provided in the embodiments of the present invention can be found in other specific embodiments of the data prefetching method of the above embodiments of the present invention.

[0109] To prevent the SMS prefetcher from generating a large number of invalid prefetches, the data prefetching device of this embodiment of the invention adopts a dynamic confidence adjustment mechanism. It introduces access counts and confidence into the pattern history table PHT, and uses a training table to train and update the access counts and confidence in the pattern history table PHT with memory access addresses. It determines the corresponding prefetch operation based on the pattern history table entry corresponding to the memory access address, thereby controlling the number of prefetches issued by the prefetcher and preventing a large number of invalid prefetch requests from being issued.

[0110] This invention provides a computer-readable storage medium.

[0111] In this embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, it implements the data prefetching method as described above.

[0112] This invention provides an SMS prefetcher.

[0113] In this embodiment, the SMS prefetcher may include a storage unit and a logic unit. The storage unit stores a computer program, and when the computer program is executed by the logic unit, it implements the data prefetching method described above.

[0114] The computer-readable storage medium and SMS prefetcher in this embodiment of the invention, based on the above-described data prefetching method, control the number of prefetches issued by the prefetcher to prevent the issuance of a large number of invalid prefetch requests.

[0115] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0116] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0117] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0118] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0120] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0121] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0122] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A data prefetching method, characterized in that, For an SMS prefetcher, the SMS prefetcher is coupled to a multi-level storage structure, the multi-level storage structure including main memory and L1 cache, L2 cache and L3 cache connected between the main memory and the processor core, the SMS prefetcher storing a pattern history table and a training table, the method including: When it is detected that the first-level cache sends a memory access address to the second-level cache, the memory access footprint, access count and confidence level corresponding to the memory access address are extracted from the pattern history table; When the number of accesses is 0, the virtual prefetch address generated based on the memory access footprint is stored in the training table. The confidence level corresponding to the memory access address in the pattern history table is calculated and updated based on the training table and the memory access address, and the number of accesses is incremented by 1. When the number of accesses is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold, the actual prefetch address generated based on the memory access footprint is sent to the level 3 cache to complete the actual prefetch operation. At the same time, the actual prefetch address is stored in the training table. Based on the training table and the memory access address, the confidence level corresponding to the memory access address in the pattern history table is calculated and updated.

2. The data prefetching method according to claim 1, characterized in that, The SMS prefetcher stores a filter table and a cumulative table. Before extracting the memory access footprint, access count, and confidence level corresponding to the access address from the pattern history table, the method further includes: Determine that an event identifier matching the memory access address exists in the pattern history table, wherein, if no event identifier matching the memory access address exists in the pattern history table, check whether the cache block corresponding to the memory access address is being accessed for the first time; If it is the first access, the cache block and event identifier corresponding to the memory access address are stored in the filter table; If it is not the first access, an access footprint is generated based on the access address, and the access footprint and event identifier corresponding to the access address are stored in the cumulative table. When the access to the cache block corresponding to the access address ends, the information stored in the cumulative table is transferred to the pattern history table, and the access count and confidence level corresponding to the access address in the pattern history table are initialized.

3. The data prefetching method according to claim 1, characterized in that, The method further includes: Prefetching is terminated when the number of accesses is greater than 0 and the confidence level is less than a preset confidence threshold.

4. The data prefetching method according to claim 1, characterized in that, The entries in the training table include region codes and offset streams. Storing the virtual prefetch addresses generated based on the memory access footprints into the training table includes: The target region encoding is determined based on the high-order part of the virtual prefetch address; If the region encoding entry of the training table does not have a corresponding target region encoding, then a target region encoding is added to the region encoding entry of the training table, the target offset is determined according to the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region encoding. If a corresponding target region code exists in the region coding entry of the training table, the target offset is determined based on the low-order part of the virtual prefetch address, and the target offset is stored in the offset stream entry corresponding to the target region code.

5. The data prefetching method according to claim 1, characterized in that, The training table entries also include prefetched invalid values, prefetched valid values, and remaining training iterations. The step of calculating and updating the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address includes: When it is determined that there is a region code in the training table that matches the high-order part of the memory access address, the offset stream, prefetch invalid value, prefetch valid value and remaining training times corresponding to the matching region code are extracted. Detect whether there is an offset in the offset stream corresponding to the matching region encoding that matches the low-order part of the memory access address; If it exists, the prefetched valid value is incremented by 1, the prefetched invalid value is decremented by 1, and the remaining training counts are decremented by 1. At the same time, the matched offsets are cleared from the offset stream corresponding to the matched region encoding. The initial value of the prefetched valid value is 0, the initial value of the prefetched invalid value is the number of offsets in its corresponding offset stream, and the initial value of the remaining training counts is the preset number of training counts. If it does not exist, then the remaining training attempts are reduced by 1; Monitor the remaining training counts. When the value of the remaining training counts is 0, calculate and update the confidence level corresponding to the memory access address in the pattern history table based on the prefetch valid value and the prefetch invalid value.

6. The data prefetching method according to claim 5, characterized in that, The step of calculating and updating the confidence level corresponding to the memory access address in the pattern history table based on the prefetched valid value and the prefetched invalid value includes: Calculate the ratio of the valid prefetched values ​​to the total number of prefetched addresses, where the total number of prefetched addresses is the sum of the valid prefetched values ​​and the invalid prefetched values; If the ratio is greater than a preset ratio threshold, the confidence level corresponding to the memory access address in the pattern history table is incremented by 1; If the ratio is less than a preset ratio threshold, the confidence level corresponding to the memory access address in the pattern history table is reduced by 1, wherein the minimum value of the confidence level is 0.

7. The data prefetching method according to claim 1, characterized in that, When the processor core requests data from the L1 cache and a cache miss occurs in the L1 cache, the L1 cache sends the memory access address to the L2 cache.

8. A data prefetching device, characterized in that, For an SMS prefetcher, the SMS prefetcher is coupled to a multi-level storage structure, the multi-level storage structure including main memory and L1 cache, L2 cache and L3 cache connected between the main memory and the processor core, the SMS prefetcher storing a pattern history table and a training table, the device comprising: The extraction module is used to extract the memory access footprint, access count and confidence level corresponding to the memory access address from the pattern history table when the first-level cache sends a memory access address to the second-level cache. The virtual prefetch module is used to store the virtual prefetch address generated based on the memory access footprint into the training table when the access count is 0, calculate and update the confidence level corresponding to the memory access address in the pattern history table based on the training table and the memory access address, and increment the access count by 1. The actual prefetch module is used to send the actual prefetch address generated based on the memory access footprint to the level 3 cache to complete the actual prefetch operation when the number of accesses is greater than 0 and the confidence level is greater than or equal to a preset confidence threshold. At the same time, the actual prefetch address is stored in the training table, and the confidence level corresponding to the memory access address in the pattern history table is calculated and updated based on the training table and the memory access address.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data prefetching method as described in any one of claims 1-7.

10. An SMS prefetcher, comprising a storage unit and a logic unit, wherein the storage unit stores a computer program, characterized in that, When the computer program is executed by the logic unit, it implements the data prefetching method as described in any one of claims 1-7.

Images