Data prefetching methods, apparatus, electronic devices, electronic equipment, and media

Abstract

Embodiments of the present disclosure provide a data prefetching method, apparatus, electronic device, electronic apparatus and medium. A data prefetching method according to an embodiment of the present disclosure is applied to an electronic device including internal memory, a cache and a prefetcher. The method includes the steps of: acquiring performance data that displays the usage rate of the internal memory (S101); adjusting the assertiveness level of the prefetcher based on the performance data (S102), wherein the assertiveness level is used to control confidence conditions and the number of prefetches for the prefetcher to generate prefetches; and performing a prefetch operation based on the adjusted assertiveness level (S103).

Description

Technical Field

[0001] This application claims the priority of Chinese Patent Application No. 2024103914997 filed on April 1, 2024, and the entire content disclosed in the above Chinese patent application is incorporated herein by reference as part of this application.

[0002] Embodiments of the present disclosure relate to a data prefetching method, apparatus, electronic device, electronic apparatus, and medium.

Background Art

[0003] Modern multi-issue high-performance CPUs (Central Processing Units) include at least one processor core (referred to as a CPU Core, or CPU core, or processor core), and each CPU core may include multiple execution units for executing instructions. In existing CPU architectures, program instructions and data are generally stored in an internal DRAM memory (Dynamic Random Access Memory). Since the execution frequency of the CPU core is much higher than that of the internal DRAM memory, it takes hundreds of CPU core clock cycles to obtain data and instructions from the internal memory, which inevitably causes the CPU core to be idle while waiting for related instructions and data, resulting in a performance degradation. In contrast, modern high-performance CPUs install a multi-level cache architecture to store recently accessed data, and at the same time, utilize a prefetcher to detect the rules by which the CPU accesses data, and thereby prefetch the data and instructions to be accessed into the cache in advance.

[0004] However, too many inaccurate prefetch requests can lead to problems such as increased memory access latency and cache contamination. These problems significantly impact CPU performance, ultimately degrading execution speed. This issue is even more pronounced in CPUs that support SMT (Synchronous Multithreading). [Overview of the Initiative] [Means for solving the problem]

[0005] Embodiments of this disclosure provide a data prefetching method, apparatus, electronic device, electronic apparatus, and medium used to adjust the level of prefetching activity of a prefetcher based on performance data displaying the usage rate of internal memory, thereby enabling the prefetcher to dynamically control prefetch generation conditions based on the set level of activity, thereby improving the accuracy and coverage of prefetching and further improving the overall operational performance of the electronic device.

[0006] According to one aspect of the present disclosure, a data prefetching method is provided. The method is applied to electronic equipment including internal memory, a cache, and a prefetcher, and the method includes the steps of: obtaining performance data indicating the usage rate of the internal memory; adjusting the assertiveness level of the prefetcher based on the performance data, the assertiveness level being used to control confidence conditions and the number of prefetches that the prefetcher generates prefetches; and performing prefetch operations based on the adjusted assertiveness level.

[0007] According to some embodiments of this disclosure, performance data includes one or more of the following: average access delay of internal memory, occupancy of the electronic device's misstatus handling registers, bandwidth utilization of internal memory, and queue occupancy in the electronic device's on-chip internet.

[0008] According to some embodiments of the present disclosure, if the performance data includes the average access delay of internal memory, the step of obtaining performance data that displays the utilization of internal memory includes the steps of obtaining the historical average access delay time of internal memory prior to a time window, obtaining the current average access delay time of internal memory within the time window, and calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of internal memory.

[0009] According to some embodiments of the present disclosure, an electronic device is equipped with a time window counter, a delay counter, a backfill counter, and a history delay counter, and the step of acquiring performance data that displays the usage rate of internal memory includes the step of recording a time window using a time window counter, triggering the calculation of the average access delay when the time window counter reaches a set threshold, and adjusting the aggressiveness level based on the average access delay; recording the total delay time of missed caches within the time window using a delay counter; recording the number of access backfills of missed caches within the time window using a backfill counter; recording the historical average access delay time prior to the time window using a history delay counter; calculating the current average access delay time based on the total delay time and the number of access backfills of missed caches within the time window, and calculating the average access delay time by taking a weighted average of the current average access delay time and the historical average access delay time.

[0010] According to some embodiments of the present disclosure, the step of adjusting the prefetcher aggressiveness level based on performance data includes the steps of increasing the prefetcher aggressiveness level to increase the amount of prefetching to internal memory when the average access delay is less than a first threshold, and decreasing the prefetcher aggressiveness level to decrease the amount of prefetching to internal memory when the average access delay is greater than a second threshold.

[0011] According to some embodiments of the present disclosure, an electronic device processes multiple threads synchronously, and the step of obtaining performance data that displays the usage of internal memory includes the step of obtaining performance data that displays the usage of internal memory for each of the multiple threads, and the step of adjusting the assertiveness level of a prefetcher based on the performance data includes the step of adjusting the assertiveness level of a prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an assertiveness level for each thread.

[0012] According to some embodiments of the present disclosure, the step of recording a time window using a time window counter includes one of the following steps: for processor clock cycles of an electronic device, incrementing the count by 1 for each clock cycle; for cache accesses of an electronic device, incrementing the count by 1 for each access; for cache access miss events of an electronic device, incrementing the count by 1 for each cache access miss; or for prefetch requests of a prefetcher, incrementing the count by 1 for each prefetch request sent.

[0013] According to some embodiments of this disclosure, the cache includes a multilevel cache, and the missed cache is the last level cache of the missed multilevel cache.

[0014] Another aspect of the present disclosure provides an electronic device comprising a processor core, internal memory, a cache, and a prefetcher, wherein the processor core is configured to acquire performance data representing the utilization of the internal memory and to adjust the aggressiveness level of the prefetcher based on the performance data, the aggressiveness level being used to control the confidence conditions and the number of prefetches that the prefetcher generates. The prefetcher is configured to perform prefetch operations based on the adjusted aggressiveness level.

[0015] A further aspect of the present disclosure provides a data prefetching device comprising: an acquisition unit configured to acquire performance data displaying the usage rate of internal memory; an adjustment unit configured to adjust the assertiveness level of a prefetcher based on the performance data, the assertiveness level comprising an adjustment unit used to control confidence conditions and the number of prefetches that the prefetcher generates prefetches; and a prefetching unit configured to perform prefetch operations based on the adjusted assertiveness level.

[0016] Another aspect of the present disclosure provides an electronic device comprising a processor and a memory in which computer-readable code is stored, and when the computer-readable code is executed by the processor, the processor is made to perform a step of the data prefetching method according to an embodiment of the present disclosure.

[0017] According to yet another aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, in which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the processor is instructed to perform a step of the data prefetching method according to an embodiment of the present disclosure.

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings of the embodiments are briefly introduced below, and it is clear that the drawings described below relate only to some embodiments of this disclosure and do not limit this disclosure. [Brief explanation of the drawing]

[0019] [Figure 1] Figure 1 is a flowchart showing how the data prefetcher performs training and prefetching. [Figure 2] Figure 2 is a schematic flowchart of a data prefetching method according to some embodiments of the present disclosure. [Figure 3] Figure 3 is a schematic diagram illustrating the adjustment of the prefetcher activity level in some embodiments of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram for adjusting the aggressiveness level of a prefetcher for SMT according to some embodiments of the present disclosure. [Figure 5] FIG. 5 is a schematic block diagram of an electronic device according to some embodiments of the present disclosure. [Figure 6] FIG. 6 is a schematic block diagram of a data prefetching device according to some embodiments of the present disclosure. [Figure 7] FIG. 7 is a schematic block diagram of an electronic device according to some embodiments of the present disclosure. [Figure 8] FIG. 8 is a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure. **DETAILED DESCRIPTION OF THE INVENTION**

[0020] Hereinafter, the technical solutions in the embodiments of the present disclosure will be clearly and completely described with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative labor shall fall within the protection scope of the present disclosure.

[0021] Also, as shown in the present disclosure and the claims, unless the context clearly indicates otherwise, terms such as "one", "a", "one kind" and / or "the" do not particularly refer to the singular form and can include the plural form. The terms "first", "second" and similar terms used in the present disclosure do not indicate any order, number or importance, but are only used to distinguish different components. Similarly, similar terms such as "include" or "comprise" mean that the elements or members described before such terms cover the elements or members listed after such terms and their equivalents, and do not exclude other elements or members. Similar terms such as "connect" or "couple" are not limited to physical or mechanical connections, and may include electrical connections such as direct or indirect connections.

[0022] In the present disclosure, flowcharts are used to illustrate the steps of the methods according to the embodiments of the present disclosure. As can be understood, the steps before and after are not necessarily performed exactly in order. On the contrary, various steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes.

[0023] As can be understood, the explanations of the technical terms and nouns mentioned in this specification have meanings known to those skilled in the art.

[0024] With the development of more extensive general computing scenarios such as artificial intelligence, higher requirements are demanded for the computing capabilities, processing efficiencies, and bandwidths of processor chips. How to provide high-performance electronic devices from software and hardware has become an important research direction.

[0025] The terms that may be mentioned in at least some embodiments of the present disclosure are interpreted as follows.

[0026] An electronic device usually includes one or more processor cores (CPU Cores) and a multi-level cache. In the architecture of the multi-level cache, the access speed of the first-level (L1) cache memory is the fastest, but the capacity is the smallest, and it is usually installed within the processor core. The capacity of the last-level (LLC, generally the third level) cache memory is the largest, but the access speed is the slowest, and it is usually shared by multiple processor cores. The access speed and capacity of the second-level (L2) cache memory are both between the L1 cache memory and the LLC cache memory, and it is usually also installed within the processor core.

[0027] Regarding data prefetching, CPU architectures allow program instructions and data to be stored in Dynamic Random Access Memory (DRAM). Because the execution frequency of the processor core is much higher than that of the DRAM's internal memory, hundreds of processor core clock cycles are required to retrieve data and instructions from internal memory. This results in the processor core becoming idle while waiting for the relevant instructions and data, causing performance loss. Therefore, modern high-performance electronic devices all include a multi-level cache architecture to store recently accessed data, and simultaneously utilize prefetchers to detect the CPU's data access rules and prefetch the data and instructions to be accessed into the cache in advance.

[0028] Regarding prefetchers, those that prefetch instructions are called instruction prefetchers, and those that prefetch data are called data prefetchers. For example, a data prefetcher can be divided into a first-level data prefetcher (i.e., a data prefetcher that prefetches target data into the first-level cache memory (L1 Cache)), a second-level data prefetcher (i.e., a data prefetcher that prefetches target data into the second-level cache memory (L2 Cache)), and a last-level data prefetcher (i.e., a data prefetcher that prefetches target data into the last-level cache memory (LLC Cache)).

[0029] As an example, Figure 1 shows a flowchart illustrating the training and prefetching processes of a data prefetcher. The prefetcher shown in Figure 1 is a first-level prefetcher, meaning it prefetches target data or instructions into first-level cache memory. Furthermore, the prefetcher shown in Figure 1 is trained using a virtual address.

[0030] As shown in Figure 1, the prefetcher performs training and prefetching according to the following steps S11 to S16.

[0031] In step S11, the prefetcher receives the virtual addresses and other attributes of at least some (e.g., all) of the access requests (e.g., historical access requests) from the CPU cores, uses the virtual addresses and other attributes of the access requests (e.g., historical access requests) to train and obtain the CPU core access rules, and forms prefetch requests based on the access rules.

[0032] In step S12, the address translator 101 converts the prefetch virtual address of the prefetch request to a prefetch physical address. Based on the prefetch physical address, it is determined whether the target data or instruction of the prefetch request exists in the first-level cache memory. If the target data or instruction of the prefetch request exists in the first-level cache memory, the prefetch request is discarded. In response to this, it is not necessary to perform the following steps S13 to S16 for the prefetch request.

[0033] Step S13, if the target data or instruction for the prefetch request is not present in the first-level cache memory, the prefetch object (e.g., target data or instruction) that needs to be prefetched is obtained from the second-level cache memory based on the prefetch physical address. For example, a storage entry is requested in the misaddressed cache memory 102, and this storage entry is assigned to the prefetch request.

[0034] In step S14, the misaddressed cache memory 102 requests the target data or instruction from a lower-level cache memory (e.g., a second-level cache memory) based on the stored entry.

[0035] In step S15, the lower-level cache memory retrieves the target data or instruction and returns the target data or instruction to the misaddressed cache memory 102.

[0036] As shown in Figure 1, if the target data or instruction of a prefetch request is stored in the second-level cache memory, the first-level cache memory retrieves the target data or instruction of the data prefetch request from the second-level cache memory. If the target data or instruction of a prefetch request is not stored in the second-level cache memory, the second-level cache memory can retrieve the target data or instruction of the prefetch request from memory at a lower level than the second-level cache memory (e.g., the last-level cache memory or internal memory). For example, the memory at a lower level than the second-level cache memory may be the third-level cache memory (e.g., the last-level cache memory, Last Level Cache) or internal memory (e.g., DRAM).

[0037] In step S16, the misaddressed cache memory 102, after receiving the target data or instruction, loads the target data or instruction into the first-level cache memory to complete the prefetch operation.

[0038] As needs to be explained, while the prefetcher is training and prefetching, the CPU core can also perform data or instruction read operations to process the associated threads. Accordingly, for the sake of clarity, Figure 1 further illustrates some of the processes by which the CPU core reads data or instructions. For example, the process by which the CPU core reads data or instructions includes the following steps:

[0039] In step S10, the CPU core outputs the virtual address of the target data / instruction of the access request to the address translator 101 (for example, the address translation pipeline), the address translator 101 translates the virtual address to a physical address, and then the CPU core determines whether or not the target data / instruction of the access request is in the first level cache memory.

[0040] For example, if the target data or instruction for an access request exists in the first-level cache memory, the target data or instruction for the access request is fetched from the first-level cache memory. If the target data or instruction for the access request does not exist in the first-level cache memory, a storage entry is requested in the misaddressed cache memory, and the storage entry is assigned to the access request. The misaddressed cache memory requests the target data or instruction for the access request from a lower-level cache memory (e.g., the second-level cache memory, L2 Cache) based on the storage entry. The lower-level cache memory retrieves the target data or instruction for the access request and returns it to the misaddressed cache memory 102. The misaddressed cache memory then writes the target data or instruction for the access request to the first-level cache memory. The first-level cache memory then provides the target data or instruction for the access request to the CPU core so that the CPU core can complete the operation of reading the data or instruction.

[0041] In high-performance CPUs, prefetching data / instruction virtual addresses often shares the address translator / address translation pipeline with normal data / instruction reads. In high-frequency and high-performance CPUs, this address translation pipeline can be multi-level, and because the importance and priority of prefetch operations are lower than the importance and priority of the CPU reading data or instructions, a single prefetch operation must wait multiple clock cycles before entering the address translation pipeline. Simultaneously, the multiple pipeline clock cycles of address translation mean that multiple clocks are required to send such a prefetch to a lower-level cache. Often, prefetch operations are timely, and the corresponding CPU core access requests occur rapidly, and the delay caused by the address translation often results in prefetched data arriving late in the first-level cache memory, thus making it impossible to completely avoid cache misses.

[0042] In many high-performance CPUs, misaddressed cache memory is a scarce resource because its data storage capacity determines the number of requests that simultaneously miss the first-level cache memory. When misaddressed cache memory becomes full, new requests are rejected, and the address translation pipeline must be run again to re-determine which memory entries can be allocated to the misaddressed cache memory. This further increases the delay of prefetching cache memory, reducing system efficiency.

[0043] Prefetch Metrics are standards for evaluating the prefetch performance of a prefetcher and may include Accuracy, Coverage, and Lateness. Accuracy is the ratio of the number of correct prefetches to the total number of prefetches; Coverage is the ratio of the number of Demand requests (requests generated by memory access instructions in the program) that hit prefetched data to the total number of Demand requests; and Lateness is the ratio of the number of prefetches that were correct but sent too late to the total number of prefetches. It is often difficult to achieve both high Accuracy and high Coverage simultaneously.

[0044] Regarding the Prefetcher Aggressiveness level, a higher Prefetcher Aggressiveness level results in looser prefetch generation conditions and a larger number of simultaneous prefetches. Conversely, a lower Prefetcher Aggressiveness level results in stricter prefetch generation conditions and a smaller amount of simultaneous prefetch data. Generally, for the same prefetcher, a higher set Aggressiveness level results in looser prefetch conditions and a larger number of items that can be prefetched at once, thereby ensuring prefetch coverage. However, this consumes more processor resources and internal memory bandwidth, inevitably reducing prefetch accuracy. On the other hand, for the same prefetcher, a lower set Aggressiveness level results in stricter prefetch conditions and a smaller number of items that can be prefetched at once, thereby ensuring prefetch accuracy and reducing internal memory bandwidth usage, but inevitably reducing prefetch coverage.

[0045] Simultaneous multithreading (SMT) technology (also known as synchronous multithreading) is a crucial technique for improving the overall performance of a CPU. It utilizes mechanisms such as multi-issue and out-of-order execution on high-performance CPU cores to execute instructions for multiple threads simultaneously. Such physical CPU cores are presented to software and the operating system as multiple virtual CPU cores. When a modern multi-issue high-performance CPU core runs a single thread, its internal execution units and hardware resources are often not fully utilized. If the thread stops execution for any reason (e.g., an L2 cache miss), the hardware execution units become idle, wasting hardware resources and reducing the performance-to-power ratio. In comparison, SMT mode improves hardware resource utilization because other threads can continue execution even if one thread stops, thereby improving the multithreaded throughput, overall performance, and performance-to-power ratio of the CPU core. It should be noted that, because CPU core resources are shared with other threads, the performance of a single thread running in SMT is often lower than its performance in single-threaded mode. Based on the maximum number of active threads supported, SMT may be referred to as SMT2 (maximum 2 active threads), SMT4 (maximum 4 active threads), etc. For ease of explanation, electronic devices capable of implementing SMT technology may be described as SMT electronic devices or SMT processors, etc.

[0046] In SMT, the memory access subsystem, each level of cache (L1, L2, LLC), and internal memory all need to handle demand and prefetch requests from multiple threads simultaneously. This significantly reduces the amount of cache available per thread and doubles the pressure to access internal memory. This becomes even more severe in SMT4. During the prefetching process, too many inaccurate prefetch requests can lead to increased memory access latency and cache contamination, among other problems. These issues, including a decrease in prefetcher performance, have a significant negative impact on CPU performance, ultimately resulting in a performance degradation. This problem is even more pronounced in CPUs that support SMT.

[0047] Therefore, how to dynamically adjust the prefetcher's aggressiveness level to improve prefetch performance while simultaneously enhancing prefetcher accuracy and coverage becomes particularly important.

[0048] To address the above problems, embodiments of the present disclosure propose a data prefetching method, apparatus, electronic device, electronic equipment, and medium, which can adjust the level of prefetching activity of a prefetcher based on performance data displaying the usage rate of internal memory, thereby dynamically adjusting the prefetch generation conditions based on the set level of activity of the prefetcher, improving the accuracy and coverage of prefetching, and further improving the overall operational performance of the electronic device.

[0049] The data prefetching methods according to embodiments of this disclosure are applied to electronic devices, which may include internal memory, a cache, and a prefetcher. For example, the internal memory may refer to the DRAM internal memory. The cache may refer to a multilevel cache architecture, for example, a cache architecture including a level 3 cache memory. The prefetcher may refer to any type of prefetcher, for example, a stride prefetcher, and the prefetcher may be a first-level cache prefetcher or a cache prefetcher of any level, but is not limited herein. Furthermore, the number of prefetchers may be one or more, and the data prefetching methods according to embodiments of this disclosure may set different aggressiveness levels for different prefetchers, or the same aggressiveness level may be set for two or more prefetchers. Also, although expressed as a data prefetching method for ease of understanding, the methods referred to herein are applicable to both data prefetchers and instruction prefetchers.

[0050] Specifically, the method according to the embodiment of the present disclosure includes the steps of: acquiring performance data that displays the usage rate of internal memory; adjusting the assertiveness level of a prefetcher based on the performance data, wherein the assertiveness level is used to control the confidence conditions and the number of prefetches that the prefetcher generates; and performing a prefetch operation based on the adjusted assertiveness level.

[0051] In one embodiment, the electronic device may be a synchronous multithreaded (SMT) processor, that is, it can simultaneously execute instructions from multiple threads within a single operation cycle.

[0052] The data prefetching method of the present disclosure will be described below with reference to the drawings. In order to maintain clarity and brevity in the following description of embodiments of the present disclosure, detailed descriptions of known functions and known components may be omitted. If any component of an embodiment of the present disclosure is shown in one or more drawings, such component will be represented by the same or similar reference numerals in each drawing.

[0053] Figure 2 is a schematic flowchart of a data prefetching method according to some embodiments of the present disclosure, the method comprising the following steps S101 to S103.

[0054] Step S101: Obtain performance data that displays the usage of internal memory.

[0055] In step S102, the aggressiveness level of the prefetcher is adjusted based on performance data, and the aggressiveness level is used to control the confidence conditions and the number of prefetches that the prefetcher generates.

[0056] Step S103: Perform a prefetch operation based on the adjusted aggressiveness level.

[0057] To ensure accuracy, prefetchers within electronic devices can be controlled to send fewer prefetch requests by setting strict prefetch conditions, for example. However, this inevitably leads to a loss of prefetch coverage. If the settings generate many prefetch requests to improve coverage, accuracy decreases, memory access pressure inevitably increases, and prefetch latency increases. In particular, with SMT, the memory access subsystem, each level of cache (L1, L2, LLC), and internal memory all need to process demand and prefetch requests from multiple threads simultaneously. This significantly reduces the amount of cache available per thread, doubling the pressure to access internal memory, a situation that becomes even more severe in SMT4. Too many inaccurate prefetch requests lead to problems such as longer memory access latency and increased cache contamination. These problems have a significant negative impact on CPU performance, ultimately degrading performance. Dynamically adjusting the aggressiveness of prefetchers is especially important in SMT.

[0058] In relation to the above prefetch characteristics, this disclosure proposes dynamically adjusting the prefetcher's aggressiveness level based on performance data that displays internal memory usage, since the internal memory usage performance data can reflect the prefetch state. When internal memory usage is high, i.e., when access to internal memory and prefetch operations are at a high level, the bandwidth occupancy of internal memory increases, prefetch accuracy decreases, and prefetch delay increases. In such cases, the prefetch conditions can be adjusted to be stricter, reducing access to internal memory for prefetch operations. When internal memory usage is low, i.e., when access to internal memory and prefetch operations are at a low level, the bandwidth occupancy of internal memory decreases, prefetch delay is reduced, but prefetch coverage decreases. In such cases, the prefetch conditions can be adjusted to be looser, increasing access to internal memory for prefetch operations.

[0059] According to some embodiments of this disclosure, the performance data includes one or more of the following: average access delay of internal memory, occupancy rate of the electronic device's misstatus handling registers, bandwidth utilization rate of internal memory, and occupancy rate of queues in the electronic device's on-chip internet.

[0060] For example, the average access latency of internal memory can be embodied as the average access latency within a certain time window when performing data prefetching against the cache. A high level of access latency indicates an increased bandwidth utilization of internal memory, while a low level of access latency indicates a low bandwidth utilization of internal memory and the ability to perform more prefetch operations. The specific steps for obtaining the average access latency of internal memory will be described in detail below with reference to the diagrams.

[0061] As an example, Miss-status Handling Registers (MSHRs) in electronic devices are used to record incomplete Cache Miss events, and the recorded information typically includes information such as addresses and thread identifiers. The MSHR occupancy rate can indicate the usage rate of internal memory. A high MSHR occupancy rate can reflect a high number of prefetcher Cache Miss events, meaning that the accuracy of prefetching is reduced and internal memory usage is at a high level. In one embodiment, the average MSHR occupancy rate over a certain period can be statistically recorded by adding new hardware to the electronic device, or by adding a counter to the MSHR Entry.

[0062] For example, the bandwidth utilization rate of the DRAM internal memory can be used by the internal memory controller within the electronic device to provide performance data for adjusting the prefetcher aggressiveness level. Alternatively, statistical utilization rates can be provided via an on-chip internet as performance data for adjusting the prefetcher aggressiveness level.

[0063] In the methods according to the embodiments of this disclosure, the confidence conditions and number of prefetches generated by the prefetcher can be controlled by dynamically adjusting the prefetcher's assertiveness level based on acquired performance data. The logic for adjusting the assertiveness level is as follows: If the performance data indicates high internal memory utilization, i.e., high levels of access to internal memory and prefetch operations, the bandwidth occupancy of internal memory increases, prefetch accuracy decreases, and prefetch delay increases. In such cases, the prefetch conditions can be adjusted to be stricter to reduce access to internal memory for prefetch operations. If the performance data indicates low internal memory utilization, i.e., low levels of access to internal memory and prefetch operations, the bandwidth occupancy of internal memory decreases, prefetch delay decreases, but prefetch coverage decreases. In such cases, the prefetch conditions can be adjusted to be looser to increase access to internal memory for prefetch operations.

[0064] As one embodiment, the following describes the specific process for obtaining the average access delay of internal memory when the performance data includes the average access delay of internal memory.

[0065] According to some embodiments of this disclosure, the step of obtaining the average access delay of internal memory includes the steps of obtaining the historical average access delay time of internal memory prior to a time window, obtaining the current average access delay time of internal memory within the time window, and calculating a weighted average of the historical average access delay time and the current average access delay time. The calculated weighted average is used as the average access delay of internal memory. Specifically, the prefetcher aggressiveness level is adjusted based on the weighted average.

[0066] According to some embodiments of the present disclosure, several improvements can be made to the CPU architecture to obtain average access delay, for example, by arranging several counters in the electronic device. According to some embodiments of the present disclosure, the electronic device is arranged with a time window counter, a delay counter, a backfill counter and a history delay counter, and specifically, the step of obtaining performance data that displays the usage of internal memory includes the step of recording a time window using a time window counter, triggering the calculation of average access delay when the time window counter reaches a set threshold, and adjusting the aggressiveness level based on the average access delay; the step of recording the total delay time of missed caches within the time window using a delay counter; the step of recording the number of access backfills of missed caches within the time window using a backfill counter; the step of recording the historical average access delay time prior to the time window using a history delay counter; and the step of calculating the current average access delay time based on the total delay time and the number of access backfills of missed caches within the time window, and calculating the average access delay by weighting the current average access delay time and the historical average access delay time.

[0067] For example, a time window counter can be added within the CPU to count CPU clock cycles, for instance, incrementing the count by 1 for each clock cycle. When a time interval threshold is reached, the prefetcher's aggressiveness level is adjusted, and the time window counter is cleared. To understand this, the time window counter is used as a trigger condition to adjust the prefetcher's aggressiveness level. Each time the set time window threshold is reached, it triggers the calculation of the average access delay, and the prefetcher's aggressiveness level is adjusted based on the calculated average access delay.

[0068] To make it clear, adjusting the assertiveness level of the prefetcher as described herein may involve increasing the assertiveness level, decreasing the assertiveness level, or maintaining the current assertiveness level.

[0069] Other implementations include the time window counter being counted using Cycles, the number of Cache Accesses (i.e., incrementing the count by 1 for each access), or the Cache Miss event (i.e., incrementing the count by 1 for each cache miss), or the Prefetch Sent event (i.e., incrementing the count by 1 for each prefetch request sent). As you can see, the time window counter may use other count trigger conditions, which will not be listed one by one here.

[0070] For example, a delay counter can be added within the CPU to statistically track the average delay of accesses within a single time window. Additionally, a timestamp field can be added to the MSHR entry. In the event of a cache miss, the timestamp at that moment is written to the newly added timestamp field in the MSHR. When the request data for a cache miss is backfilled, the time at that moment is used to subtract the timestamp in the MSHR to obtain the delay interval for that access. This interval is then accumulated in the delay counter, and the delay counter data is cleared whenever the time window counter reaches a threshold.

[0071] According to some embodiments of this disclosure, the cache in the computer equipment referred to above may be a multilevel cache architecture, for example, including an L1 Cache, an L2 Cache, and an LLC Cache, and for example, a missed cache (Cache Miss) recorded by a delay counter is the last level of the multilevel cache that has been missed, i.e., an LLC Miss. As another example, it may be an L1 Miss or an L2 Miss. As can be seen, since the LLC Cache is closest to the internal memory, recording an LLC Miss can better reflect the usage status of the internal memory compared to an L1 Miss or an L2 Miss.

[0072] For example, a new backfill counter could be added within the CPU to track the total number of Cache Miss access backfills within a single time window. When a Cache Miss request is backfilled, the backfill counter would be incremented by 1, and the data in the backfill counter would be cleared whenever the time window counter reaches a threshold.

[0073] As an example, a historical delay counter can be added to the CPU to store the average delay calculated within each past time window.

[0074] Specifically, when the time window counter reaches a count threshold (for example, when the record reaches N CPU clock cycles), the current average access delay time for the current window can be obtained by dividing the backfill counter using the delay counter. Then, the average access delay can be calculated by weighting the current average access delay time and the historical average access delay time recorded by the history delay counter. For example, the weight values ​​for the current average access delay time and the historical average access delay time can be set to be the same. Furthermore, the newly calculated average access delay time is used to update the history delay counter as the historical average access delay time to be used in the next calculation process.

[0075] Based on the above process, the average access delay within a fixed time window can be obtained, and then the prefetch aggressiveness level of the prefetcher can be adjusted based on the average access delay. Based on the aggressiveness level, the prefetcher adjusts the prefetch generation process to improve the accuracy / coverage of prefetches and further improve the overall CPU performance.

[0076] According to some embodiments of the present disclosure, the step of adjusting the prefetcher aggressiveness level based on performance data includes the steps of increasing the prefetcher aggressiveness level to increase the amount of prefetching to internal memory when the average access delay is less than a first threshold, and decreasing the prefetcher aggressiveness level to decrease the amount of prefetching to internal memory when the average access delay is greater than a second threshold.

[0077] For example, the first threshold may be called the rising threshold and may be set to equal to, for example, 80 ns, and the second threshold may be called the falling threshold and may be set to equal to, for example, 120 ns.

[0078] Figure 3 is a schematic diagram illustrating the adjustment of the prefetcher's aggressiveness level according to some embodiments of this disclosure. First, the time window counter is incremented by 1 each cycle, and when the time window counter reaches a set threshold, the adjustment of the prefetcher's aggressiveness is initiated. Subsequently, the current average access delay of cache misses within the current time window can be calculated based on the delay counter and backfill counter, and the value of (average access delay of the current window + history access delay counter) / 2 is updated in the history access delay counter and compared with the set increase and decrease thresholds for adjusting the prefetcher's aggressiveness level. If it is greater than the decrease threshold, the prefetcher's aggressiveness level is reduced, and if it is less than the increase threshold, the prefetcher's aggressiveness level is increased.

[0079] In summary, the process of adjusting the level of prefetcher activity may include the following: On the one hand, if the calculated average access delay time for Cache Misses is greater than the lower threshold (120 ns), it is shown that the current prefetch access delay increases, and the accuracy of prefetching decreases. To avoid problems such as increased memory access delay and increased corrupted cache data due to too many inaccurate prefetch requests, the prefetcher aggressiveness level is reduced, thereby making the prefetcher generation conditions stricter, for example, requiring higher confidence conditions, and appropriately reducing the number of one-time prefetches, thereby reducing the access pressure on internal memory and improving prefetch accuracy. On the other hand, if the calculated average access delay time for Cache Misses is less than the upper threshold (80 ns), it is shown that the current prefetch access delay is low, meaning that internal memory utilization is low and more access operations can be performed. This improves the prefetcher aggressiveness level, makes the prefetcher generation conditions looser, for example, requiring lower confidence conditions, and appropriately increasing the number of one-time prefetches, thereby improving prefetch coverage. In another aspect, if the calculated average access delay for Cache Miss falls between the rising and falling thresholds, the current aggressiveness level can be maintained.

[0080] Table 1 is an illustrative table showing the level of assertiveness of a Stride prefetcher, using Stride prefetcher as an example.

[0081] [Table 1]

[0082] For example, the initial assertiveness level for the Stride prefetcher may be set to 2, meaning the confidence conditions for generating prefetches are a confidence level of 4 or higher and a maximum of two one-time prefetches. Subsequently, the assertiveness level of the Stride prefetcher can be adjusted by referring to the process described in conjunction with Figure 3, and the assertiveness level, confidence conditions, and number of prefetches can be adjusted using the data shown in Table 1.

[0083] In some embodiments, the adjustment of prefetcher parameters based on the assertiveness level includes, but is not limited to, the confidence conditions for generating prefetches and the number of prefetches generated each time; parameters such as the address distance of the prefetches may also be adjusted, and are not limited thereto.

[0084] As mentioned above, in the case of multi-threaded SMT, compared to single-threaded, the access pressure on internal memory is inevitably further increased. Too many inaccurate prefetch requests can lead to problems such as longer memory access delays and increased cache contamination. These problems have a significant negative impact on CPU performance and ultimately degrade execution performance.

[0085] The data prefetching method according to the embodiments of this disclosure is applied to an SMT processor. Specifically, in the case of an SMT, the step of acquiring performance data that displays the usage rate of internal memory includes the step of acquiring performance data that displays the usage rate of internal memory for each of a plurality of threads, and the step of adjusting the assertiveness level of the prefetcher based on the performance data includes the step of adjusting the assertiveness level of the prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an assertiveness level for each thread.

[0086] In other words, in a multithreaded scenario, the aggressiveness level for each thread can be set by statistically calculating the average internal memory access delay for each different thread.

[0087] Figure 4 is a schematic diagram of adjusting the prefetcher aggressiveness level for an SMT according to some embodiments of this disclosure. As shown in Figure 4, in the process of dynamically adjusting the aggressiveness level described in combination with Figure 3, aggressiveness levels are set for multiple threads. For example, a CPU can process instructions from four threads simultaneously, i.e., n is equal to 3, and the aggressiveness level at which it performs prefetching can be adjusted for threads 0, 1, 2, and 3, respectively. As an example, since the information recorded in the MSHR includes information such as the thread number, prefetch requests from different threads can be distinguished and their average access delays can be statistically calculated.

[0088] As for several implementations, the adjustment process is not limited to individually statistically measuring the average access delay of each thread; it is also possible, and is not limited, to using the amount of access delay shared between threads to adjust the shared aggressiveness level among threads in the prefetcher.

[0089] Embodiments of this disclosure provide a data prefetching method that can dynamically adjust the prefetcher aggressiveness level, monitoring memory access latency by MSHR and dynamically adjusting the prefetcher aggressiveness level based on the latency. The above method is particularly applicable in the case of SMT, where, when SMT is enabled and access latency is high, the prefetch aggressiveness is reduced, thereby reducing the number of prefetches and improving prefetch accuracy, mitigating internal memory bandwidth constraints and cache contamination issues. When SMT is enabled and access latency is not high, the prefetch aggressiveness is increased, and the number of prefetches is increased to improve prefetch coverage.

[0090] According to the methods provided in the embodiments of this disclosure, monitoring access delay and dynamically adjusting the prefetcher's aggressiveness is not limited to throttling the prefetcher, but may also involve increasing the prefetcher's aggressiveness, thereby making the adjustment method more real-time and accurate. Furthermore, multiple prefetchers may be integrated simultaneously, and the aggressiveness of multiple prefetchers may be dynamically controlled, allowing for finer adjustments based on aggressiveness levels for different prefetchers to achieve optimal prefetcher performance.

[0091] In summary, the data prefetching method provided in the embodiments of this disclosure allows the prefetcher to adjust its prefetching aggressiveness level based on performance data displaying internal memory usage. In one embodiment, the prefetcher's aggressiveness level can be dynamically adjusted by comparing the average access delay of internal memory with a set threshold. This allows the prefetcher to dynamically adjust prefetch generation conditions based on the set aggressiveness level, improving prefetch accuracy and coverage, and further enhancing the overall operational performance of the electronic device.

[0092] Another aspect of the present disclosure further provides electronic devices. An electronic device according to an embodiment of the present disclosure may include a processor core (referred to as a CPU core), internal memory, a cache, and a prefetcher. The processor core may be configured to perform the steps of: acquiring performance data representing the usage of internal memory; and adjusting the aggressiveness level of a prefetcher based on the performance data, the aggressiveness level being used to control confidence conditions and the number of prefetches that the prefetcher generates. The prefetcher may be configured to perform prefetch operations based on the adjusted aggressiveness level.

[0093] In one embodiment, the electronic device may be a synchronous multithreaded (SMT) processor, that is, it can simultaneously execute instructions from multiple threads within a single operation cycle.

[0094] Figure 5 is a schematic block diagram of an electronic device according to some embodiments of the present disclosure. As shown in Figure 5, the electronic device 1000 may include a processor 1010, non-volatile memory (NVM) 1060, and dynamic random access memory (DRAM) 1070. Specifically, the processor 1010 may consist of a CPU core 1020, an internal memory controller 1030, a cache 1040, and a prefetcher 1050. To be understood, the configuration of the electronic device is not limited thereto and may further include other necessary components.

[0095] DRAM1070 and NVM1060 are both internal memories of electronic device 1000. The connectivity of the electronic device shown in Figure 5 is merely an example of an electronic device having a hybrid internal memory architecture, and DRAM1070 and NVM1060 shown in Figure 5 are merely examples of multi-level memory within the electronic device. In actual applications, the internal structure of the electronic device is not specifically limited, and the electronic device may also include other memories besides DRAM1070 and NVM1060.

[0096] The processor 1010 is a core component of the electronic device 1000 and can call different software programs within the electronic device 1000 to implement different functions. For example, the processor 1010 can provide access to the DRAM 1070 and NVM 1060. The processor 1010 may be a central processing unit (CPU). In addition to the CPU, the processor may be another application-specific integrated circuit (ASIC). The electronic device may further include multiple processors. Also, the processor may be a single-core processor or a multi-core processor. In a multi-core processor architecture, the processor may include multiple processor cores (CPU cores). For example, as shown in Figure 5, the processor 1010 may include one or more CPU cores 1020.

[0097] The internal memory controller 1030 is a crucial component of the electronic device 1000 that controls the internal memory and facilitates data exchange between the internal memory and the processor 1010 (e.g., CPU). In some cases, the internal memory controller 1030 may be located inside the northbridge chip. In other cases, the internal memory controller 1030 may be integrated into the processor 1010 (as shown in Figure 5), specifically, the internal memory controller 1030 may be integrated into the processor 1010's substrate. As can be understood, when the internal memory controller 1030 is located inside the northbridge chip, the internal memory controller needs to exchange data with the processor via the northbridge chip, resulting in significant data delay. When the internal memory controller 1030 is integrated into the processor 1010, the internal memory controller 1030 can exchange data directly with the processor.

[0098] As shown in Figure 5, the internal memory controller 1030 may be coupled to an NVM controller and a DRAM controller. The DRAM controller is used to control access to the DRAM 1070, and the NVM controller is used to control access to the NVM 1060. The NVM controller and the DRAM controller may also be called media controllers. In practical applications, in some cases the NVM controller and the DRAM controller may be independent of the internal memory controller 1030. In other cases the NVM controller and the DRAM controller may be integrated into the internal memory controller 1030 and function logically as part of the internal memory controller 1030 (as shown in Figure 5). In embodiments of this disclosure, the internal memory controller 1030 may be connected to the NVM 1060 and DRAM 1070 via an internal memory bus (e.g., a double-rate DDR bus). As can be understood, in practical applications the NVM controller may also communicate with the NVM 1060 via a PCI high-speed bus and other types of buses such as a Direct Media Interface (DMI) bus.

[0099] In addition to the method shown in Figure 5, the internal memory controller 1030 may be connected to the NVM 1060 and DRAM 1070. Other methods may also be used; for example, the internal memory controller 1030 may be directly connected to the DRAM 1070 via an internal memory bus. In other words, the internal memory controller 1030 may be connected to the DRAM 1070 via an internal memory bus, and the DRAM 1070 may be connected to the NVM 1060 via an internal memory bus.

[0100] As described above, in the electronic device shown in Figure 5, the DRAM 1070 may be connected to the processor 1010 via an internal memory bus. The DRAM 1070 has the advantage of fast access speed. The processor 1010 can access the DRAM 1070 at high speed and perform read or write operations on it. Typically, the DRAM 1070 is used to store various running software within the operating system, input / output data, and information exchanged with external memory. However, the DRAM 1070 is volatile, and when the power is turned off, the information in the DRAM 1070 is no longer stored.

[0101] The new NVM can be addressed in bytes and writes data bit by bit to non-volatile memory, thus it can be used as internal memory. In the embodiments of this disclosure, the NVM 1060 can function as internal memory of electronic device 1000 together with the DRAM 1070. Compared to the DRAM 1070, the NVM 1060 has non-volatile properties, allowing for better data storage. In the embodiments of this disclosure, the non-volatile memory that can be used as internal memory may be called Storage Class Memory (SCM).

[0102] As needs to be explained, DRAM is a type of volatile memory, and in practical applications, other types of random access memory (RAM) may be used as the internal memory of a computer system. For example, static random access memory (SRAM) may be used as the internal memory of a computer system. The NVM1060 shown in Figure 5 may include new types of non-volatile memory such as phase-change random access memory (PCM), resistive random access memory (RRAM), magnetic random access memory (MRAM), or ferroelectric random access memory (FRAM®), and the specific type of NVM in the embodiments of this disclosure is not limited here.

[0103] Compared to DRAM1070, NVM1060 has a slower access speed. Therefore, NVM1060 is typically used as the system's main memory, and DRAM1070 is used as a cache for NVM1060 to compensate for the slow access speed of the main memory NVM1060 and improve the access speed of the internal memory. As shown in Figure 5, in the electronic device shown in Figure 5, DRAM1070 functions as a cache for NVM1060. When the internal memory controller 1030 receives a memory access request sent by the processor 1010, it first determines whether the target address in the memory access request (i.e., the address of the internal memory block to be accessed) hits DRAM1070, and then determines whether the data to be accessed is stored in DRAM1070. If it determines that the target address in the memory access request hits DRAM1070, the internal memory controller 1030 can directly obtain the data to be accessed from DRAM1070, thereby reducing the access delay. If the internal memory controller 1030 determines that the target address in the memory access request does not hit the DRAM 1070, the internal memory controller 1030 retrieves the data to be accessed from the NVM 1060.

[0104] According to embodiments of the present disclosure, for the electronic device 1000 shown in Figure 5, the CPU core 1020 may be configured to acquire performance data displaying the usage rate of the internal memory (e.g., DRAM 1070) and to adjust the aggressiveness level of the prefetcher 1050 based on the performance data, the aggressiveness level being used to control the confidence conditions and the number of prefetches that the prefetcher 1050 generates. The prefetcher 1050 may be configured to perform prefetch operations based on the adjusted aggressiveness level.

[0105] According to some embodiments of this disclosure, performance data includes one or more of the following: average access delay of internal memory, occupancy of the electronic device's misstatus handling registers, bandwidth utilization of internal memory, and queue occupancy in the electronic device's on-chip internet.

[0106] According to some embodiments of this disclosure, the prefetcher 1050 is a stride prefetcher.

[0107] According to some embodiments of the present disclosure, if the performance data includes the average access delay of internal memory, the steps of the CPU core 1020 to obtain performance data that displays the usage of internal memory include the steps of obtaining the historical average access delay time of internal memory prior to a time window, obtaining the current average access delay time of internal memory within the time window, and calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of internal memory.

[0108] According to some embodiments of the present disclosure, the electronic device is equipped with a time window counter, a delay counter, a backfill counter, and a history delay counter, and the steps of the CPU core 1020 acquiring performance data that displays the usage of internal memory include: recording a time window using the time window counter, triggering the calculation of the average access delay when the time window counter reaches a set threshold, and adjusting the aggressiveness level based on the average access delay; recording the total delay time of missed caches within the time window using the delay counter; recording the number of access backfills of missed caches within the time window using the backfill counter; recording the historical average access delay time prior to the time window using the history delay counter; calculating the current average access delay time based on the total delay time and the number of access backfills of missed caches within the time window, and calculating the average access delay by weighting the current average access delay time and the historical average access delay time.

[0109] According to some embodiments of this disclosure, the steps by which the CPU core 1020 adjusts the prefetcher aggressiveness level based on performance data include: increasing the prefetcher aggressiveness level to increase the amount of prefetching to internal memory when the average access delay is less than a first threshold; and decreasing the prefetcher aggressiveness level to decrease the amount of prefetching to internal memory when the average access delay is greater than a second threshold.

[0110] According to some embodiments of the present disclosure, an electronic device processes multiple threads synchronously, and the step of obtaining performance data that displays the usage of internal memory includes the step of obtaining performance data that displays the usage of internal memory for each of the multiple threads. The step of adjusting the aggressiveness level of a prefetcher based on the performance data includes the step of adjusting the aggressiveness level of a prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an aggressiveness level for each thread.

[0111] According to some embodiments of the present disclosure, the steps by which the CPU core 1020 uses a time window counter to record a time window include one of the following: for processor clock cycles of electronic devices, incrementing the count by 1 for each clock cycle; for cache accesses of electronic devices, incrementing the count by 1 for each access; for cache access miss events of electronic devices, incrementing the count by 1 for each cache access miss; or for prefetch requests of prefetchers, incrementing the count by 1 for each prefetch request sent.

[0112] According to some embodiments of this disclosure, the cache 1040 may include a multilevel cache, and the missed cache is the last level cache of the missed multilevel cache, i.e., LLC Miss.

[0113] The electronic devices according to the embodiments of this disclosure can implement specific execution processes for functions such as data access and prefetching, and can specifically implement steps of several embodiments of the data prefetching method according to this disclosure beyond those described in conjunction with the drawings, which will not be repeated here. Similar data prefetching processes and similar technical effects can be achieved by using the electronic devices of the embodiments of this disclosure.

[0114] A further aspect of this disclosure provides a data prefetching device. Figure 6 shows a schematic block diagram of a data prefetching device according to some embodiments of this disclosure.

[0115] As shown in Figure 6, the data prefetching device 2000 includes an acquisition unit 2010, an adjustment unit 2020, and a prefetching unit 2030. According to some embodiments of this disclosure, the acquisition unit 2010 may be configured to acquire performance data that displays the usage of internal memory. The adjustment unit 2020 may be configured to adjust the aggressiveness level of the prefetcher based on the performance data, the aggressiveness level being used to control the confidence conditions and the number of prefetches that the prefetcher generates. The prefetching unit 2030 may be configured to perform prefetch operations based on the adjusted aggressiveness level.

[0116] According to some embodiments of this disclosure, performance data includes one or more of the following: average access delay of internal memory, occupancy of the electronic device's misstatus handling registers, bandwidth utilization of internal memory, and queue occupancy in the electronic device's on-chip internet.

[0117] According to some embodiments of the present disclosure, if the performance data includes the average access delay of internal memory, the steps by which the acquisition unit 2010 acquires performance data that displays the utilization of internal memory include: acquiring the historical average access delay time of internal memory prior to a time window; acquiring the current average access delay time of internal memory within the time window; and calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of internal memory.

[0118] According to some embodiments of the present disclosure, the steps of an acquisition unit 2010 acquiring performance data that displays the usage rate of internal memory include: recording a time window using a time window counter, triggering the calculation of the average access delay when the time window counter reaches a set threshold, and adjusting the aggressiveness level based on the average access delay; recording the total delay time of missed caches within the time window using a delay counter; recording the number of access backfills of missed caches within the time window using a backfill counter; recording the historical average access delay time prior to the time window using a history delay counter; calculating the current average access delay time based on the total delay time and the number of access backfills of missed caches within the time window, and calculating the average access delay time by taking a weighted average of the current average access delay time and the historical average access delay time.

[0119] According to some embodiments of the present disclosure, the steps by which the adjustment unit 2020 adjusts the prefetcher aggressiveness level based on performance data include: increasing the prefetcher aggressiveness level to increase the amount of prefetching to internal memory when the average access delay is less than a first threshold; and decreasing the prefetcher aggressiveness level to decrease the amount of prefetching to internal memory when the average access delay is greater than a second threshold.

[0120] According to some embodiments of the present disclosure, the step of an acquisition unit 2010 acquiring performance data that displays the usage rate of internal memory includes the step of acquiring performance data that displays the usage rate of internal memory for each of a plurality of threads, and the step of an adjustment unit 2020 adjusting the aggressiveness level of a prefetcher based on the performance data includes the step of adjusting the aggressiveness level of a prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an aggressiveness level for each thread.

[0121] According to some embodiments of the present disclosure, the step of recording a time window using a time window counter includes one of the following steps: for processor clock cycles of an electronic device, incrementing the count by 1 for each clock cycle; for cache accesses of an electronic device, incrementing the count by 1 for each access; for cache access miss events of an electronic device, incrementing the count by 1 for each cache access miss; or for prefetch requests of a prefetcher, incrementing the count by 1 for each prefetch request sent.

[0122] According to some embodiments of this disclosure, the cache includes a multilevel cache, and the missed cache is the last level cache of the missed multilevel cache.

[0123] According to some embodiments of this disclosure, the prefetcher is a stride prefetcher.

[0124] Specifically, the data prefetching device 2000 according to the embodiment of this disclosure can perform steps of the data prefetching method according to the embodiment of this disclosure that are described in conjunction with the drawings, and can achieve similar technical effects, and a detailed explanation is omitted here.

[0125] Another aspect of this disclosure provides an electronic device. Figure 7 shows a schematic block diagram of an electronic device according to some embodiments of this disclosure.

[0126] As shown in Figure 7, the electronic device 3000 may include a processor 3010 and a memory 3020. According to some embodiments of the present disclosure, the memory 3020 stores computer-readable code, and when the computer-readable code is executed by the processor 3010, the data prefetching method according to the embodiments of the present disclosure can be performed.

[0127] The processor 3010 can perform various operations and processes based on the program stored in the memory 3020. Specifically, the processor 3010 may be an integrated circuit having signal processing capabilities. The general-purpose processor may be a microprocessor, or it may be any ordinary processor, etc. For example, the processor may refer to a processor that can implement SMT.

[0128] Memory 3020 stores computer-readable code, and when this code is executed by processor 3010, the processor can be made to perform data prefetching methods according to some embodiments of the present disclosure. Memory 3020 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Note that the memory described herein may be any suitable type of memory. For example, by executing computer-readable code in memory 3020, the processor can perform steps of a data prefetching method beyond those described in conjunction with the drawings.

[0129] Further aspects of this disclosure provide non-temporary computer-readable storage media. Figure 8 shows schematic diagrams of computer-readable storage media according to some embodiments of this disclosure.

[0130] As shown in Figure 8, a computer-readable instruction 4010 is stored in the computer-readable storage medium 4000. When the computer-readable instruction 4010 is executed by a processor, the data prefetch method described above with reference to the drawings can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, and flash memory. For example, the computer-readable storage medium 4000 may be connected to an electronic device such as a computer, and subsequently, when the electronic device executes the computer-readable instruction 4010 stored in the computer-readable storage medium 4000, the data prefetch method according to the embodiments of the present disclosure described above can be performed.

[0131] Embodiments of this disclosure provide a data prefetching method, apparatus, electronic device, electronic apparatus, and medium used to adjust the level of prefetching activity of a prefetcher based on performance data displaying internal memory usage, thereby dynamically adjusting prefetch generation conditions based on the set activity level of the prefetcher, improving the accuracy and coverage of prefetching, and further improving the overall operational performance of the electronic device.

[0132] Furthermore, while this disclosure makes various references to several units within electronic devices and electronic apparatuses relating to embodiments of this disclosure, any number of different units may be used and run on the client and / or server. The units are merely illustrative, and different units may be used in different embodiments of the methods, electronic devices, and electronic apparatuses.

[0133] Those skilled in the art will understand that all or some of the steps of the above method are completed by instructing the relevant hardware by a computer program, which may be stored in a computer-readable storage medium such as read-only memory, magnetic disk, or optical disk. Optionally, all or some of the steps of the above embodiment may be implemented using one or more integrated circuits. Correspondingly, each module / unit in the above embodiment may be implemented in hardware form or in software function module form. This disclosure is not limited to any particular combination of hardware and software.

[0134] Unless otherwise specified, all terms used herein have the same meaning as those commonly understood by those skilled in the art in the field to which this disclosure belongs. Furthermore, as should be understood, these terms, as defined in ordinary dictionaries, should be interpreted as having the meaning consistent with their meaning in the context of their respective technologies, and not as idealized or highly formalized unless explicitly defined herein.

[0135] The foregoing is for illustrative purposes only and should not be considered as limiting the disclosure. While several exemplary embodiments of the disclosure have been described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and merits of the disclosure. Accordingly, all such modifications are intended to be included within the scope of the disclosure as defined by the claims. As should be understood, the foregoing is for illustrative purposes only and should not be considered as limiting to any specific embodiment disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The disclosure is limited by the claims and their equivalents.

Claims

1. A data prefetching method, applicable to an electronic device including internal memory, a cache, and a prefetcher, wherein the method is The steps include: acquiring performance data that displays the usage rate of the internal memory; A step of adjusting the assertiveness level of the prefetcher based on the performance data, wherein the assertiveness level is used to control the confidence conditions and the number of prefetches that the prefetcher generates. A data prefetching method comprising the step of performing a prefetch operation based on an adjusted aggressiveness level.

2. The aforementioned performance data is The average access delay of the internal memory, The occupancy rate of the misstatus processing register of the aforementioned electronic device, The bandwidth utilization rate of the aforementioned internal memory, The method according to claim 1, comprising one or more of the following: the queue occupancy rate in the on-chip internet of the electronic device.

3. If the performance data includes the average access delay of the internal memory, the step of obtaining performance data that displays the utilization rate of the internal memory is: The steps include obtaining the historical average access delay time of the internal memory prior to the time window, The steps include obtaining the current average access delay time of the internal memory within the aforementioned time window, The method according to claim 2, further comprising the step of calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of the internal memory.

4. The electronic device is equipped with a time window counter, a delay counter, a backfill counter, and a history delay counter, and the step of acquiring performance data that displays the usage rate of the internal memory is: A step of recording a time window using the time window counter, wherein when the time window counter reaches a set threshold, the calculation of the average access delay is triggered, and the aggressiveness level is adjusted based on the average access delay. The steps include recording the total delay time of missed caches within the time window using the aforementioned delay counter, The steps include recording the number of missed cache access backfills within the time window using the backfill counter, The steps include: recording the average historical access delay time prior to the time window using the aforementioned historical delay counter; The method according to claim 3, comprising the steps of: calculating the current average access delay time based on the total delay time of missed caches within the time window and the number of access backfills; and calculating the average access delay by taking a weighted average of the current average access delay time and the historical average access delay time.

5. The step of adjusting the aggressiveness level of the prefetcher based on the performance data is: If the average access delay is less than a first threshold, the process involves increasing the prefetching level of the prefetcher to increase the amount of prefetching to the internal memory. The method according to claim 4, further comprising the step of reducing the amount of prefetching to the internal memory by lowering the activity level of the prefetcher if the average access delay is greater than a second threshold.

6. The electronic device processes multiple threads synchronously and obtains performance data that displays the usage rate of the internal memory. The process includes the step of obtaining performance data for each of the aforementioned threads that displays the usage rate of the internal memory, and The step of adjusting the aggressiveness level of the prefetcher based on the performance data is: The method according to any one of claims 1 to 5, comprising the step of adjusting the assertiveness level of the prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an assertiveness level for each thread.

7. The step of recording a time window using the aforementioned time window counter is, A step of incrementing a count by 1 for each clock cycle with respect to the processor clock cycle of the electronic device, For cache access of the aforementioned electronic device, the step of incrementing the count by 1 for each access, For each cache access miss event of the aforementioned electronic device, the count is incremented by 1, or The method according to claim 4, comprising the step of incrementing a count by 1 for each prefetch request transmitted in response to a prefetch request of the prefetcher.

8. The method according to claim 4, wherein the cache includes a multilevel cache, and the missed cache is the last level cache of the missed multilevel cache.

9. An electronic device comprising a processor core, internal memory, cache, and prefetcher, The processor core is configured to acquire performance data displaying the usage of the internal memory, and to adjust the aggressiveness level of the prefetcher based on the performance data, the aggressiveness level being used to control the confidence conditions and the number of prefetches that the prefetcher generates, and The prefetcher is an electronic device configured to perform prefetch operations based on an adjusted activity level.

10. The aforementioned performance data is The average access delay of the internal memory, The occupancy rate of the misstatus processing register of the aforementioned electronic device, The bandwidth utilization rate of the aforementioned internal memory, The electronic device according to claim 9, which includes one or more of the following: the queue occupancy rate in the on-chip internet of the electronic device.

11. If the performance data includes the average access delay of the internal memory, the step of the processor core obtaining performance data that shows the utilization of the internal memory is: The steps include obtaining the historical average access delay time of the internal memory prior to the time window, The steps include obtaining the current average access delay time of the internal memory within the aforementioned time window, The electronic device according to claim 10, further comprising the step of calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of the internal memory.

12. The electronic device is equipped with a time window counter, a delay counter, a backfill counter, and a history delay counter, and the processor core acquires performance data that displays the usage rate of the internal memory, A step of recording a time window using the time window counter, wherein when the time window counter reaches a set threshold, the calculation of the average access delay is triggered, and the aggressiveness level is adjusted based on the average access delay. The steps include recording the total delay time of missed caches within the time window using the aforementioned delay counter, The steps include recording the number of missed cache access backfills within the time window using the backfill counter, The steps include: recording the average historical access delay time prior to the time window using the aforementioned historical delay counter; The electronic device according to claim 11, comprising the steps of: calculating the current average access delay time based on the total delay time of missed caches within the time window and the number of access backfills; and calculating the average access delay by taking a weighted average of the current average access delay time and the historical average access delay time.

13. The step of the processor core adjusting the aggressiveness level of the prefetcher based on the performance data is: If the average access delay is less than a first threshold, the process involves increasing the prefetching level of the prefetcher to increase the amount of prefetching to the internal memory. The electronic device according to claim 12, further comprising the step of reducing the prefetching amount to the internal memory by lowering the activity level of the prefetcher if the average access delay is greater than a second threshold.

14. The electronic device processes multiple threads synchronously and obtains performance data that displays the usage rate of the internal memory. The process includes the step of obtaining performance data for each of the aforementioned threads that displays the usage rate of the internal memory, and The step of adjusting the aggressiveness level of the prefetcher based on the performance data is: The electronic device according to any one of claims 9 to 13, comprising the step of adjusting the assertiveness level of the prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an assertiveness level for each thread.

15. The electronic device according to claim 12, wherein the cache includes a multilevel cache, and the missed cache is the last level cache of the missed multilevel cache.

16. A data prefetching device, An acquisition unit configured to acquire performance data that displays the usage rate of internal memory, An adjustment unit configured to adjust the assertiveness level of a prefetcher based on the performance data, wherein the assertiveness level is used to control the confidence conditions and the number of prefetches that the prefetcher generates. A data prefetching device including a prefetching unit configured to perform prefetching operations based on an adjusted aggressiveness level.

17. If the performance data includes the average access delay of the internal memory, the step of obtaining performance data that displays the utilization rate of the internal memory is: The steps include obtaining the historical average access delay time of the internal memory prior to the time window, The steps include obtaining the current average access delay time of the internal memory within the aforementioned time window, The data prefetching device according to claim 16, comprising the step of calculating the weighted average of the historical average access delay time and the current average access delay time as the average access delay of the internal memory.

18. The data prefetching device processes multiple threads synchronously and obtains performance data that displays the usage rate of the internal memory. The process includes the step of obtaining performance data for each of the aforementioned threads that displays the usage rate of the internal memory, and The step of adjusting the aggressiveness level of the prefetcher based on the performance data is: The data prefetching device according to claim 16, comprising the step of adjusting the assertiveness level of the prefetcher for each thread based on its respective performance data, wherein the prefetcher is set to have an assertiveness level for each thread.

19. An electronic device comprising at least one processor and at least one memory, wherein the at least one memory stores computer-readable code, and when the computer-readable code is executed by the at least one processor, the electronic device causes the at least one processor to perform the steps of the method according to any one of claims 1 to 8.

20. A non-temporary computer-readable storage medium that stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1 to 8.

Images