Data prefetching method and apparatus, and electronic device, electronic apparatus and medium

Abstract

The embodiments of the present disclosure provide a data prefetching method and apparatus, and an electronic device, an electronic apparatus and a medium. The data prefetching method in the embodiments of the present disclosure is applicable to an electronic device, wherein the electronic device comprises a memory, a cache and a prefetcher. The method comprises: acquiring performance data representing the utilization rate of a memory (S101); on the basis of the performance data, adjusting the aggressiveness level of a prefetcher (S102), wherein the aggressiveness level is used for controlling the prefetcher to generate a confidence condition for prefetching and a prefetch count; and on the basis of an adjusted aggressiveness level, performing a prefetching operation (S103).

Description

Data prefetching method, device, electronic device, electronic device and medium

[0001] This application claims priority to Chinese Patent Application No. 2024103914997 filed on April 1, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to a data pre-fetching method, apparatus, electronic device, electronic device, and medium. Background Art

[0003] A modern multi-issue high-performance CPU (Central Processing Unit) includes at least one processor core (CPU Core, or CPU core, processor core), and each CPU core can include multiple execution units to execute instructions. In existing CPU architectures, program instructions and data are generally stored in DRAM memory (Dynamic Random Access Memory). The CPU core operating frequency is much higher than the DRAM memory operating frequency. Therefore, it takes hundreds of CPU core clock cycles to obtain data and instructions from the memory, which will inevitably cause the CPU core to idle while waiting for relevant instructions and data, resulting in performance degradation. To address this, modern high-performance CPUs set up a multi-level cache architecture to save recently accessed data, and use a prefetcher to discover the CPU's data access patterns, thereby prefetching the data and instructions to be accessed into the cache in advance.

[0004] However, excessive and inaccurate prefetch requests can lead to increased memory access latency and cache contamination, which can significantly negatively impact CPU performance and ultimately lead to reduced performance. This problem is particularly pronounced on CPUs that support SMT (Simultaneous Multithreading).

[0005] Summary of the Invention

[0006] Embodiments of the present disclosure provide a data prefetching method, apparatus, electronic device, electronic device, and medium for adjusting the aggressiveness level of prefetching performed by a prefetcher based on performance data representing memory usage, thereby enabling the prefetcher to dynamically control the conditions for prefetching based on the set aggressiveness level, thereby improving the accuracy and coverage of prefetching, and further improving the overall operating performance of the electronic device.

[0007] According to one aspect of the present disclosure, a data prefetching method is provided. The method is applicable to an electronic device including a memory, a cache, and a prefetcher, and comprises: obtaining performance data representing memory usage; adjusting an aggressiveness level of the prefetcher based on the performance data, wherein the aggressiveness level is used to control a confidence condition for the prefetcher to generate prefetches and a prefetch quantity; and performing a prefetch operation based on the adjusted aggressiveness level.

[0008] According to some embodiments of the present disclosure, the performance data includes one or more of the following: average access latency of memory; occupancy rate of miss status processing registers of the electronic device; bandwidth usage rate of memory; and occupancy rate of queues in an on-chip interconnect network of the electronic device.

[0009] According to some embodiments of the present disclosure, when the performance data includes the average access delay of the memory, obtaining performance data characterizing the memory usage includes: obtaining the historical average access delay time of the memory before the time window; obtaining the current average access delay time of the memory within the time window; and calculating the weighted average of the historical average access delay time and the current average access delay time, wherein the weighted average is used as the average access delay of the memory.

[0010] According to some embodiments of the present disclosure, a time window counter, a delay counter, a backfill counter and a historical delay counter are configured in an electronic device, and obtaining performance data representing memory usage includes: using the time window counter to record the time window, wherein when the time window counter reaches a set threshold, it triggers the calculation of the average access delay and adjusts the aggressiveness level according to the average access delay; using the delay counter to record the total delay time of cache misses in the time window; using the backfill counter to record the number of access backfills of cache misses in the time window; using the historical delay counter to record the historical average access delay time before the time window; and calculating the current average access delay time based on the total delay time of cache misses and the number of access backfills in the time window, and taking a weighted average of the current average access delay time and the historical average access delay time to calculate the average access delay.

[0011] According to some embodiments of the present disclosure, adjusting the aggressiveness level of the prefetcher based on performance data includes: increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory when the average access delay is less than a first threshold; and reducing the aggressiveness level of the prefetcher so as to reduce the prefetch amount for the memory when the average access delay is greater than a second threshold.

[0012] According to some embodiments of the present disclosure, an electronic device synchronously processes multiple threads, and obtaining performance data representing memory usage includes: obtaining performance data representing memory usage for each of the multiple threads; and adjusting the aggressiveness level of the prefetcher according to the performance data includes: adjusting the aggressiveness level of the prefetcher for each thread according to the respective performance data, wherein the aggressiveness level is set for each thread in the prefetcher.

[0013] According to some embodiments of the present disclosure, recording the time window using a time window counter includes one of the following: for a processor clock cycle of an electronic device, the count is increased by one for each clock cycle; for a cache access of an electronic device, the count is increased by one for each access; for a cache access miss event of an electronic device, the count is increased by one for each cache access miss; or for a prefetch request of a prefetcher, the count is increased by one for each prefetch request sent.

[0014] According to some embodiments of the present disclosure, the cache includes a multi-level cache, and the miss cache is a last-level cache that misses the multi-level cache.

[0015] According to another aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor core, memory, a cache, and a prefetcher. The processor core is configured to: obtain performance data representing memory usage; and adjust an aggressiveness level of the prefetcher based on the performance data, wherein the aggressiveness level is used to control the confidence condition for the prefetcher to generate prefetches and the number of prefetches. The prefetcher is configured to perform prefetch operations based on the adjusted aggressiveness level.

[0016] According to another aspect of the present disclosure, a data prefetching device is provided. The data prefetching device includes: an acquisition unit configured to acquire performance data representing memory usage; an adjustment unit configured to adjust an aggressiveness level of a prefetcher based on the performance data, wherein the aggressiveness level is used to control a confidence condition for the prefetcher to generate prefetches and a prefetch quantity; and a prefetching unit configured to perform a prefetch operation based on the adjusted aggressiveness level.

[0017] According to another aspect of the present disclosure, an electronic device is provided, including a processor and a memory, wherein the memory stores computer-readable code, which, when executed by the processor, causes the processor to perform the steps of the data prefetching method according to an embodiment of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the processor executes the steps of the data prefetching method according to an embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present disclosure, but do not limit the present disclosure.

[0020] FIG1 shows a flow chart of a data prefetcher performing training and prefetching;

[0021] FIG2 shows a schematic flow chart of a data prefetching method according to some embodiments of the present disclosure;

[0022] FIG3 shows a schematic diagram of adjusting the prefetcher aggressiveness level according to some embodiments of the present disclosure;

[0023] FIG4 shows a schematic diagram of adjusting the prefetcher aggressiveness level for SMT according to some embodiments of the present disclosure;

[0024] FIG5 shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure;

[0025] FIG6 shows a schematic block diagram of a data pre-fetching device according to some embodiments of the present disclosure;

[0026] FIG7 shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure;

[0027] FIG8 shows a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0029] In addition, as shown in this disclosure and the claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not specifically refer to the singular, but also include the plural. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0030] Flowcharts are used in this disclosure to illustrate the steps of the methods according to the embodiments of the present disclosure. It should be understood that the preceding or following steps do not necessarily need to be performed in exact order. Instead, the various steps may be performed in reverse order or simultaneously. Furthermore, other operations may be added to these processes.

[0031] It is understood that the professional terms, nouns and other descriptions involved in this document have meanings well known to those skilled in the art.

[0032] With the development of broader general computing scenarios such as artificial intelligence, higher requirements are being placed on processor chips in terms of computing power, processing efficiency, and bandwidth. Providing high-performance electronic devices in terms of software and hardware has become an important research direction.

[0033] The following are explanations of terms that may be involved in at least some embodiments of the present disclosure.

[0034] Electronic devices usually include one or more processor cores (CPU Core) and multiple levels of cache. In the multi-level cache architecture, the first-level (L1) cache memory has the fastest access speed but the smallest capacity and is usually set within the processor core; the last-level (LLC, generally the third level) cache memory has the largest capacity and the slowest access speed and is usually shared by multiple processor cores; the second-level (L2) cache memory has an access speed and capacity between the L1 cache memory and the LLC cache memory and is usually also set within the processor core.

[0035] Data Prefetch: In CPU architectures, program instructions and data can be stored in dynamic random access memory (DRAM). The processor core's operating frequency is much higher than that of DRAM. Therefore, retrieving data and instructions from memory requires hundreds of processor core clock cycles, causing the processor core to idle while waiting for relevant instructions and data, resulting in performance loss. Therefore, modern high-performance electronic devices all include a multi-level cache architecture to store recently accessed data. Prefetchers are used to detect CPU data access patterns and pre-fetch upcoming data and instructions into the cache.

[0036] Prefetcher: A prefetcher used to prefetch instructions is called an instruction prefetcher; a prefetcher used to prefetch data is called a data prefetcher. For example, data prefetchers can be divided into first-level data prefetchers (i.e., data prefetchers that prefetch target data into the first-level cache memory (L1Cache)), second-level data prefetchers (i.e., data prefetchers that prefetch target data into the second-level cache memory (L2Cache)), and last-level data prefetchers (i.e., data prefetchers that prefetch target data into the last-level cache memory (LLCCache)).

[0037] As an example, Figure 1 shows a flowchart of a data prefetcher training and prefetching process. The prefetcher shown in Figure 1 is a first-level prefetcher, i.e., a prefetcher that prefetches target data or instructions into a first-level cache memory. Furthermore, the prefetcher shown in Figure 1 uses virtual addresses to train the prefetcher.

[0038] As shown in FIG1 , the prefetcher performs training and prefetching through the following steps S11 to S16 .

[0039] Step S11: The prefetcher receives virtual addresses and other attributes of at least part (e.g., all) of the access requests (e.g., historical access requests) from the CPU core, uses the virtual addresses and other attributes of the above access requests (e.g., historical access requests) for training to obtain the access rules of the CPU core, and forms prefetch requests based on the above access rules.

[0040] Step S12: The address translator 101 translates the prefetch virtual address of the prefetch request into a prefetch physical address. Based on the prefetch physical address, it is determined whether the target data or instruction of the prefetch request exists in the first-level cache memory. If the target data or instruction of the prefetch request exists in the first-level cache memory, the prefetch request is discarded. Accordingly, steps S13 to S16 below do not need to be executed for this prefetch request.

[0041] Step S13: If the target data or instruction of the prefetch request does not exist in the first-level cache memory, the prefetch object (e.g., target data or instruction) to be prefetched is obtained from the second-level cache memory based on the prefetch physical address. For example, a storage item is requested from the miss address buffer (MAB) 102, and the storage item is allocated to the prefetch request.

[0042] Step S14: the miss address cache memory 102 requests the target data or instruction from the next level of cache memory (eg, the second level cache memory) based on the storage entry.

[0043] Step S15 : The next level cache memory obtains the target data or instruction and returns the target data or instruction to the miss address cache memory 102 .

[0044] As shown in FIG1 , when the target data or instruction of the prefetch request is stored in the second-level cache memory, the first-level cache memory obtains the target data or instruction of the data prefetch request from the second-level cache memory. When the target data or instruction of the prefetch request is not stored in the second-level cache memory, the second-level cache memory can obtain the target data or instruction of the prefetch request from a memory located at the next level of the second-level cache memory (e.g., the last-level cache memory or the main memory). For example, the memory located at the next level of the second-level cache memory can be a third-level cache memory (e.g., the last-level cache memory) or a main memory (e.g., DRAM).

[0045] Step S16: After receiving the target data or instruction, the miss address cache memory 102 puts the target data or instruction into the first-level cache memory to complete the prefetch operation.

[0046] It should be noted that while the prefetcher is training and prefetching, the CPU core can also read data or instructions to execute related thread processing. Accordingly, for ease of description, Figure 1 also illustrates the process of the CPU core reading data or instructions. For example, the process of the CPU core reading data or instructions includes the following steps.

[0047] Step S10: The CPU core outputs the virtual address of the target data / instruction of the access request to the address translator 101 (for example, the address translation pipeline), and the address translator 101 translates the above virtual address into a physical address. Then, the CPU core determines whether the target data / instruction of the access request is in the first-level cache memory.

[0048] For example, if the target data or instruction of the access request exists in the first-level cache memory, the target data or instruction of the access request is retrieved from the first-level cache memory; if the target data or instruction of the access request does not exist in the first-level cache memory, a storage item is requested from the miss address cache memory (MAB) and the storage item is allocated to the above-mentioned access request. Based on the storage item, the miss address cache memory requests the target data or instruction of the access request from the cache memory of the next level (for example, the second-level cache memory, L2Cache). The cache memory of the next level obtains the target data or instruction of the access request and returns the target data or instruction of the access request to the miss address cache memory 102. Then, the miss address cache memory writes the target data or instruction of the access request to the first-level cache memory. Then, the first-level cache memory provides the target data or instruction of the access request to the CPU core so that the CPU core can complete the operation of reading data or instructions.

[0049] In high-performance CPUs, prefetching of virtual addresses of data / instructions often shares the address translator / address translation pipeline with normal data / instruction reading. In high-frequency, high-performance CPUs, the address translation pipeline may be multi-stage, and because the importance and priority of prefetch operations are lower than the importance and priority of CPU reading data or instructions, a prefetch operation may need to wait for multiple clock cycles before entering the address translation pipeline. At the same time, due to the multiple pipeline clock cycles of address translation, such a prefetch requires multiple clocks to be sent to the next level cache. In many cases, prefetch operations are time-sensitive, and the corresponding CPU core access requests often occur very quickly. The delay caused by the above-mentioned address translation often causes the prefetched data to arrive at the first-level cache memory too late, making it impossible to completely avoid cache misses.

[0050] In many high-performance CPUs, the amount of data stored in the miss address buffer (MAB) determines the number of requests that simultaneously miss the first-level cache. Therefore, the miss address buffer (MAB) is a relatively scarce resource. If the miss address buffer is full, a new request will be rejected and must then re-enter the address translation pipeline to determine which storage items can be allocated to the miss address buffer. This further increases cache prefetch latency, reducing system efficiency.

[0051] Prefetch Metrics: Prefetch performance metrics are used to evaluate the prefetch performance of a prefetcher. These metrics include accuracy, coverage, and lateness. Accuracy is the ratio of the number of correct prefetches to the total number of prefetches. Coverage is the ratio of demand requests (requests generated by memory access instructions in the program) that match the prefetched data to the total number of demand requests. Lateness is the ratio of the number of correct prefetches that were issued too late to the total number of prefetches. It's often difficult to achieve both accuracy and coverage simultaneously.

[0052] Prefetcher Aggressiveness: The higher the prefetcher's aggressiveness, the looser the prefetching conditions and the greater the number of prefetches generated. The lower the prefetcher's aggressiveness, the stricter the prefetching conditions and the smaller the amount of prefetched data. Generally speaking, for the same prefetcher, a higher aggressiveness level ensures relatively high prefetch coverage due to looser prefetching conditions and a larger number of prefetches that can be prefetched at a time. However, this consumes more processor resources and memory bandwidth, inevitably reducing prefetch accuracy. On the other hand, a lower aggressiveness level ensures relatively high prefetch accuracy and reduces memory bandwidth usage due to stricter prefetching conditions and a smaller number of prefetches that can be prefetched at a time. However, this also inevitably reduces prefetch coverage.

[0053] Simultaneous multithreading (SMT) technology (also known as simultaneous multithreading) is a key technology for improving overall CPU performance. It leverages the multi-issue and out-of-order execution mechanisms of high-performance CPU cores to simultaneously execute instructions from multiple threads. This allows a single physical CPU core to appear to software and the operating system as multiple virtual CPU cores. When executing a single thread, the multiple execution units and hardware resources within a modern multi-issue high-performance CPU core are often underutilized. When the thread stalls due to certain reasons (such as an L2 cache miss), the hardware execution units are forced to idle, wasting hardware resources and reducing performance-to-power ratio. In contrast, in SMT mode, when one thread stalls, other threads can continue to run, improving hardware resource utilization and boosting the CPU core's multithreaded throughput, overall performance, and performance-to-power ratio. However, due to sharing CPU core resources with other threads, the performance of a single thread running under SMT is often lower than its single-threaded performance. Depending on the maximum number of active threads supported, SMT can be called SMT 2 (up to two active threads), SMT 4 (up to four active threads), etc. For ease of description, electronic devices capable of implementing SMT technology can be described as SMT electronic devices or SMT processors, etc.

[0054] With SMT, the memory access subsystem, cache levels (L1, L2, LLC), and memory all need to process demand requests and prefetch requests from multiple threads simultaneously. This significantly reduces the cache capacity available to each thread, and the pressure on memory access increases exponentially. This situation is exacerbated under SMT4. During the prefetching process, excessive and inaccurate prefetch requests can lead to increased memory access latency and cache contamination. These prefetcher performance degradations can have a significant negative impact on CPU performance, ultimately leading to performance degradation. This problem is even more pronounced on CPUs that support SMT.

[0055] Therefore, it becomes particularly important to dynamically adjust the aggressiveness level of the prefetcher to balance the accuracy and coverage of the prefetcher, thereby improving the prefetching performance.

[0056] In response to the above problems, the embodiments of the present disclosure propose a data prefetching method, apparatus, electronic device, electronic device and medium, which can adjust the aggressiveness level of prefetching performed by the prefetcher based on performance data representing memory usage, so that the prefetcher can dynamically adjust the conditions for prefetching according to the set aggressiveness level, improve the accuracy and coverage of prefetching, and thereby improve the overall operating performance of the electronic device.

[0057] The data prefetching method provided according to an embodiment of the present disclosure is applicable to an electronic device, which may include a memory, a cache and a prefetcher. For example, the memory may refer to the above-mentioned DRAM memory. The cache may refer to a multi-level cache architecture, as an example, it may be a cache architecture including a three-level cache memory. The prefetcher may refer to any type of prefetcher, as an example, it may refer to a stride prefetcher. In addition, the prefetcher may be a first-level cache prefetcher or an arbitrary-level cache prefetcher, without limitation. Further, the number of prefetchers may be one or more, and the data prefetching method according to an embodiment of the present disclosure may set an aggressive level for different prefetchers or set the same aggressive level for two or more prefetchers. In addition, it is understood that, although described as a data prefetching method, the method mentioned herein is applicable to both data prefetchers and instruction prefetchers.

[0058] Specifically, the method provided according to an embodiment of the present disclosure includes: obtaining performance data characterizing memory usage; adjusting the aggressiveness level of the prefetcher based on the performance data, wherein the aggressiveness level is used to control the confidence conditions and prefetch quantity generated by the prefetcher; and performing prefetch operations based on the adjusted aggressiveness level.

[0059] As an implementation manner, the electronic device may be a simultaneous multi-threading (SMT) processor, that is, capable of executing instructions from multiple threads simultaneously within one operation cycle.

[0060] The data prefetching method of the present disclosure will be described below with reference to the accompanying drawings. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one accompanying drawing, the component is represented by the same or similar reference numerals in each of the accompanying drawings.

[0061] FIG2 shows a schematic flow chart of a data prefetching method according to some embodiments of the present disclosure, which includes steps S101 - S103 .

[0062] Step S101: Acquire performance data representing memory usage;

[0063] Step S102: adjusting the aggressiveness level of the prefetcher according to the performance data, wherein the aggressiveness level is used to control the confidence condition and the number of prefetches generated by the prefetcher; and

[0064] Step S103: performing a prefetch operation according to the adjusted aggressiveness level.

[0065] To ensure accuracy, prefetchers in electronic devices can be controlled to issue fewer prefetch requests, for example by setting stricter prefetch conditions. However, this will inevitably result in a loss of prefetch coverage. If more prefetch requests are generated to improve coverage, accuracy will decrease, and memory access pressure will inevitably increase, which in turn increases prefetch latency. In particular, under SMT, the memory access subsystem, cache levels (L1, L2, LLC), and memory all need to process demand requests and prefetch requests from multiple threads simultaneously. The available cache capacity per thread is significantly reduced, and the pressure on memory access increases exponentially. This situation is even more severe under SMT 4. Excessive and inaccurate prefetch requests can lead to problems such as longer memory access latency and increased cache pollution. These problems can have a significant negative impact on CPU performance, ultimately leading to performance degradation. Dynamically adjusting the aggressiveness of the prefetcher under SMT is particularly important.

[0066] In response to the above prefetching characteristics, the present disclosure proposes to dynamically adjust the aggressiveness level of the prefetcher based on performance data that characterizes memory usage. This is because the performance data of memory usage can reflect the state of prefetching. If the memory usage is high, that is, the access to the memory and the prefetch operation are at a relatively high level, the memory bandwidth occupancy increases, which will reduce the accuracy of prefetching and increase the prefetching delay. In this case, the prefetching conditions can be adjusted to be stricter to reduce the access to the memory by the prefetch operation; if the memory usage is low, that is, the access to the memory and the prefetch operation are at a relatively low level, the memory bandwidth occupancy is low, and the prefetching delay is small, but this will reduce the coverage of prefetching. In this case, the prefetching conditions can be adjusted to be looser to increase the access to the memory by the prefetch operation.

[0067] According to some embodiments of the present disclosure, the above-mentioned performance data includes one or more of the following: average access delay of memory; occupancy rate of miss status processing register of electronic device; bandwidth utilization rate of memory; and occupancy rate of queues in on-chip interconnection network of electronic device.

[0068] For example, the average memory access latency can be represented by the average access latency during a time window when prefetching data from the cache. A high access latency indicates increased memory bandwidth utilization, while a low access latency indicates a relatively high memory bandwidth utilization, allowing for more prefetching operations. The specific steps for obtaining the average memory access latency are described in detail below with reference to the accompanying figures.

[0069] As an example, the Miss-status Handling Registers (MSHR) of an electronic device is used to record each unfinished Cache Miss event, and the recorded information generally includes address, thread ID and other information. The occupancy rate of MSHR can represent the memory utilization rate. If the occupancy rate of MSHR is high, it can reflect that the prefetcher has more Cache Miss events, that is, the prefetch accuracy is reduced, and the memory utilization rate is at a relatively high level at this time. As an implementation method, new hardware can be added to the electronic device to count the average occupancy rate of MSHR over a period of time, or a counter can be added to the MSHR Entry to record the average occupancy rate of MSHR over a period of time.

[0070] As an example, a memory controller in an electronic device can be used to provide DRAM memory bandwidth usage as performance data for adjusting the prefetcher aggressiveness level. As an example, an on-chip interconnect network can be used to provide statistical occupancy as performance data for adjusting the prefetcher aggressiveness level.

[0071] According to the method of the embodiment of the present disclosure, the aggressiveness level of the prefetcher can be dynamically adjusted according to the obtained performance data, thereby controlling the confidence conditions and the number of prefetches generated by the prefetcher. The logic of the aggressiveness level adjustment is: if the performance data indicates that the memory usage rate is high, that is, the access to the memory and the prefetch operation are at a relatively high level, and the memory bandwidth occupancy rate increases, this will cause the accuracy of the prefetch to decrease and the prefetch delay to increase. In this case, the prefetch conditions can be adjusted to be stricter to reduce the access to the memory by the prefetch operation; if the performance data indicates that the memory usage rate is low, that is, the access to the memory and the prefetch operation are at a relatively low level, the memory bandwidth occupancy rate is low, and the prefetch delay is small, but this will cause the prefetch coverage to decrease. In this case, the prefetch conditions can be adjusted to be looser to increase the access to the memory by the prefetch operation.

[0072] As an implementation manner, the following describes a specific process of obtaining the average access delay of the memory when the performance data includes the average access delay of the memory.

[0073] According to some embodiments of the present disclosure, obtaining an average access latency of a memory includes: obtaining a historical average access latency of the memory before a time window; obtaining a current average access latency of the memory within the time window; and calculating a weighted average of the historical average access latency and the current average access latency. The calculated weighted average serves as the average access latency of the memory. Specifically, the prefetcher aggressiveness level is adjusted based on the weighted average.

[0074] According to some embodiments of the present disclosure, in order to obtain the average access delay, some improvements can be made to the CPU architecture, for example, some counters can be configured in the electronic device. According to some embodiments of the present disclosure, the electronic device is configured with a time window counter, a delay counter, a backfill counter, and a historical delay counter. Specifically, obtaining performance data representing the memory usage includes: using the time window counter to record the time window, wherein when the time window counter reaches a set threshold, triggering the calculation of the average access delay and adjusting the aggressiveness level according to the average access delay; using the delay counter to record the total delay time of cache misses in the time window; using the backfill counter to record the number of access backfills of cache misses in the time window; using the historical delay counter to record the historical average access delay time before the time window; and calculating the current average access delay time based on the total delay time of cache misses and the number of access backfills in the time window, and performing a weighted average of the current average access delay time and the historical average access delay time to calculate the average access delay.

[0075] As an example, a time window counter can be added to the CPU to count CPU clock cycles, for example, incrementing the count by one for each clock cycle. If the time interval threshold is reached, the prefetcher aggressiveness level is adjusted and the time window counter is cleared. It is understood that the time window counter is used as a trigger condition for adjusting the prefetcher aggressiveness level. Each time the set time window threshold is reached, it triggers the calculation of the average access latency, and the prefetcher aggressiveness level is adjusted based on the calculated average access latency.

[0076] It can be understood that the adjustment of the prefetcher aggressiveness level described herein may be to increase the aggressiveness level, to decrease the aggressiveness level, or to maintain the current aggressiveness level.

[0077] As other implementations, the time window counter can use not only cycles for counting, but also the number of cache accesses (cache access), that is, the count increases by one for each access; or the cache miss event (cache miss), that is, the count increases by one for each cache miss; or the prefetch request (Prefetch Sent) event of the prefetcher for counting, that is, the count increases by one for each prefetch request sent. It is understandable that the time window counter can also use other counting trigger conditions, which are not listed here one by one.

[0078] As an example, a new latency counter can be added to the CPU to measure the average latency of accesses within a time window. Furthermore, a timestamp field can be added to the MSHR entry. When a cache miss occurs, the timestamp at that time is written to the newly added timestamp field in the MSHR. When a cache miss request is backfilled, the current time is subtracted from the timestamp in the MSHR to determine the latency interval for that access. This time interval is then added to the latency counter, and the latency counter is reset to zero each time the time window counter reaches its threshold.

[0079] According to some embodiments of the present disclosure, the cache in the electronic computing device mentioned above may be a multi-level cache architecture, for example, including L1Cache, L2Cache, and LLC Cache. As an example, the cache miss (Cache Miss) recorded by the delay counter is the last level cache miss of the multi-level cache, that is, LLC Miss. As another example, it may be L1Miss or L2Miss. It is understandable that since LLC Cache is closest to the memory, recording LLC Miss can better reflect the memory usage compared to L1Miss or L2Miss.

[0080] As an example, a backfill counter can be added to the CPU to count the total number of cache miss access backfills within a time window. When a cache miss request data is backfilled, the backfill counter is incremented by one. Each time the time window counter reaches the threshold, the backfill counter data is cleared.

[0081] As an example, a historical delay counter may be added to the CPU to store the average delay calculated in each time window in the past.

[0082] Specifically, when the time window counter reaches a counting threshold (for example, when the record reaches N CPU clock cycles), the current average access delay time of the current window can be obtained by dividing the delay counter by the backfill counter. Then, the current average access delay time and the historical average access delay time recorded by the historical delay counter are weighted averaged to calculate the average access delay. As an example, the weight values ​​of the current average access delay time and the historical average access delay time can be set to the same. In addition, the new average access delay obtained by the current calculation can be used to update the historical delay counter as the historical average access delay time used in the next calculation process.

[0083] Based on the above process, the average access latency within a fixed time window can be obtained. Then, the prefetcher's prefetch aggressiveness level can be adjusted based on the average access latency. The prefetcher adjusts the prefetching process based on the aggressiveness level, improving the accuracy and coverage of prefetching, thereby improving the overall CPU performance.

[0084] According to some embodiments of the present disclosure, adjusting the aggressiveness level of the prefetcher based on performance data includes: increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory when the average access delay is less than a first threshold; and reducing the aggressiveness level of the prefetcher so as to reduce the prefetch amount for the memory when the average access delay is greater than a second threshold.

[0085] As an example, the first threshold may also be referred to as a rising threshold, for example, it may be set to be equal to 80 ns, and the second threshold may also be referred to as a falling threshold, for example, it may be set to be equal to 120 ns.

[0086] Figure 3 shows a schematic diagram of adjusting the prefetcher aggressiveness level according to some embodiments of the present disclosure. First, the time window counter is incremented by one every cycle. When the time window counter reaches the set threshold, the adjustment of the prefetcher aggressiveness begins. Then, the current average access delay of the Cache Miss in the current time window can be calculated based on the delay counter and the backfill counter, and the value of (current window average access delay + historical access delay counter) / 2 can be updated to the historical access delay counter, and compared with the set rising threshold and falling threshold to adjust the prefetcher aggressiveness level. If it is greater than the falling threshold, the prefetcher aggressiveness level is reduced; if it is less than the rising threshold, the prefetcher aggressiveness level is increased.

[0087] In general, the process of adjusting the prefetcher aggressiveness level may include: on the one hand, if the calculated average access delay of the cache miss is greater than the falling threshold (120ns), it means that the current prefetch access delay is high and the prefetch accuracy is reduced. In order to avoid problems such as excessive inaccurate prefetch requests leading to longer memory access delays and increased cache data pollution, the prefetcher aggressiveness level can be reduced, thereby making the prefetcher generation conditions more stringent, for example, requiring a higher confidence condition and appropriately reducing the number of one-time prefetches, thereby reducing memory access pressure and improving prefetch accuracy; on the other hand, if the calculated average access delay of the cache miss is less than the rising threshold (80ns), it means that the current prefetch access delay is low, that is, the memory utilization rate is low and more access operations can be performed. Therefore, the prefetcher aggressiveness level can be increased, thereby making the prefetcher generation conditions more relaxed, for example, requiring a lower confidence condition and appropriately increasing the number of one-time prefetches, thereby improving prefetch coverage. On the other hand, if the calculated average access delay of the cache miss is between the rising threshold and the falling threshold, the current aggressiveness level can be maintained.

[0088] Taking the Stride prefetcher as an example, Table 1 shows an example table of the aggressiveness level of the Stride prefetcher:

[0089] Table 1

[0090] As an example, for the Stride prefetcher, the initial aggressiveness level can be set to 2, that is, the confidence condition for generating a prefetch is that the confidence (Confidence) is greater than or equal to 4, and the maximum number of prefetches at a time is 2. Then, the aggressiveness level of the Stride prefetcher can be adjusted by referring to the process described in conjunction with Figure 3. The aggressiveness level, confidence condition, and prefetch number can refer to the data shown in Table 1.

[0091] As some implementation methods, adjusting the prefetcher parameters according to the aggressiveness level includes but is not limited to the confidence condition for generating prefetches and the number of prefetches generated each time. Parameters such as the prefetch address distance may also be adjusted, which are not limited here.

[0092] As described above, in the case of multi-threaded SMT, compared with single-threaded, memory access pressure will inevitably increase further. Excessive and inaccurate prefetch requests will lead to problems such as longer memory access latency and increased cache pollution. These problems will have a significant negative impact on CPU performance and ultimately lead to reduced operating performance.

[0093] The data prefetching method provided in accordance with an embodiment of the present disclosure is applicable to an SMT processor. Specifically, in the case of SMT, obtaining performance data representing memory usage includes: obtaining performance data representing memory usage for each of a plurality of threads; and adjusting the prefetcher's aggressiveness level based on the performance data includes: adjusting the prefetcher's aggressiveness level for each thread based on the respective performance data, wherein the prefetcher sets an aggressiveness level for each thread.

[0094] That is to say, in the case of multi-threading, the average memory access latency can be statistically calculated for different threads respectively, so as to set the aggressiveness level for each thread separately.

[0095] FIG4 shows a schematic diagram of adjusting the prefetcher aggressiveness level for SMT according to some embodiments of the present disclosure. As shown in FIG4 , for the process of dynamically adjusting the aggressiveness level described in conjunction with FIG3 , the aggressiveness level is set for each of the multiple threads. For example, the CPU can process instructions from four threads simultaneously, that is, n is equal to 3, and the prefetch aggressiveness level can be adjusted for thread 0, thread 1, thread 2, and thread 3, respectively. As an example, the information recorded in the MSHR includes information such as the thread number, thereby distinguishing prefetch requests from different threads, and calculating their average access delays respectively.

[0096] As some implementations, during the adjustment process, it is not limited to counting the average access delay of each thread separately. The access delay amount shared among threads can also be used to adjust the aggressiveness level shared among threads in the prefetcher, which is not limited here.

[0097] The disclosed embodiment provides a data prefetching method that can dynamically adjust the prefetcher aggressiveness level. It monitors the memory access latency through MSHR and dynamically adjusts the prefetcher aggressiveness level according to the latency. The above method is particularly suitable for the SMT scenario. When SMT is turned on and the access latency is high, the aggressiveness of prefetching is reduced, thereby reducing the number of prefetches to improve prefetching accuracy and alleviate the problems of memory bandwidth tension and cache pollution. When SMT is turned on and the access latency is not high, the aggressiveness of prefetching is increased, and the number of prefetches is increased to improve the coverage of prefetching.

[0098] The method provided by the disclosed embodiments monitors access latency and dynamically adjusts the prefetcher's aggressiveness. This goes beyond throttling the prefetcher and allows for increased aggressiveness, making the adjustment more timely and targeted. Furthermore, multiple prefetchers can be integrated simultaneously and their aggressiveness dynamically controlled. This allows for more refined adjustments based on the aggressiveness level of each prefetcher to maximize its performance.

[0099] In summary, the data prefetching method provided by the embodiment of the present disclosure can adjust the aggressiveness level of the prefetcher based on the performance data that characterizes the memory usage. As an implementation method, the average access delay of the memory can be compared with the set threshold to dynamically adjust the aggressiveness level of the prefetcher, so that the prefetcher can dynamically adjust the conditions for prefetching according to the set aggressiveness level, improve the accuracy and coverage of prefetching, and thus improve the overall operating performance of the electronic device.

[0100] According to another aspect of the present disclosure, an electronic device is also provided. The electronic device according to an embodiment of the present disclosure may include a processor core (CPU core, or referred to as a CPU core), a memory, a cache, and a prefetcher. The processor core can be configured to implement the following steps: obtaining performance data characterizing the usage rate of the memory; adjusting the aggressiveness level of the prefetcher according to the performance data, wherein the aggressiveness level is used to control the confidence condition and the number of prefetches generated by the prefetcher. The prefetcher can be configured to perform a prefetch operation according to the adjusted aggressiveness level.

[0101] As an implementation manner, the electronic device may be a simultaneous multi-threading (SMT) processor, that is, capable of executing instructions from multiple threads simultaneously within one operation cycle.

[0102] FIG5 shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure. As shown in FIG5 , the electronic device 1000 may include a processor 1010, a non-volatile memory (NVM) 1060, and a dynamic random access memory (DRAM) 1070. Specifically, the processor 1010 may be composed of a CPU core 1020, a memory controller 1030, a cache 1040, and a prefetcher 1050. It is understood that the composition of the electronic device is not limited to this and may also include other necessary components.

[0103] DRAM 1070 and NVM 1060 are both memories of electronic device 1000. The connection relationship of the electronic devices shown in FIG5 is merely an example of an electronic device with a hybrid memory architecture, and DRAM 1070 and NVM 1060 shown in FIG5 are merely an example of multi-level memory in an electronic device. In practical applications, the internal structure of an electronic device is not specifically limited, and the electronic device may also include other memories in addition to DRAM 1070 and NVM 1060.

[0104] The processor 1010 is a core component of the electronic device 1000. The processor 1010 can call different software programs in the electronic device 1000 to implement different functions. For example, the processor 1010 can access the DRAM 1070 and the NVM 1060. The processor 1010 can be a central processing unit (CPU). In addition to the CPU, the processor can also be other specific integrated circuits (application specific integrated circuit, ASIC). The electronic device can also include multiple processors. In addition, the processor can be a single-core processor or a multi-core processor. In a multi-core processor architecture, the processor can include multiple processor cores (CPU core). For example, as shown in Figure 5, the processor 1010 can include one or more CPU cores 1020.

[0105] The memory controller 1030 is an important component of the electronic device 1000 that controls the memory and exchanges data between the memory and the processor 1010 (e.g., CPU). In one case, the memory controller 1030 can be located inside the north bridge chip. In another case, the memory controller 1030 can be integrated into the processor 1010 (as shown in FIG5 ). Specifically, the memory controller 1030 can be integrated on the substrate of the processor 1010. It is understandable that when the memory controller 1030 is located inside the north bridge chip, the memory controller needs to exchange data with the processor through the north bridge chip, resulting in a large data delay. When the memory controller 1030 can be integrated into the processor 1010, the memory controller 1030 can directly exchange data with the processor.

[0106] As shown in FIG5 , the memory controller 1030 can be coupled to an NVM controller and a DRAM controller. The DRAM controller is used to control access to the DRAM 1070, and the NVM controller is used to control access to the NVM 1060. The NVM controller and the DRAM controller can also be referred to as media controllers. In actual applications, in one case, the NVM controller and the DRAM controller can be independent of the memory controller 1030. In another case, the NVM controller and the DRAM controller can also be integrated into the memory controller 1030 and logically serve as part of the memory controller 1030 (as shown in FIG5 ). In the embodiment of the present disclosure, the memory controller 1030 can connect the NVM 1060 and the DRAM 1070 via a memory bus (e.g., a double data rate DDR bus). It is understood that in actual applications, the NVM controller can also communicate with the NVM 1060 via other types of buses such as a PCI Express bus and a Direct Media Interface (DMI) bus.

[0107] In addition to connecting NVM 1060 and DRAM 1070 via the method shown in FIG5 , memory controller 1030 can also adopt other methods. For example, memory controller 1030 can be directly connected to DRAM 1070 via a memory bus. In other words, memory controller 1030 can be connected to DRAM 1070 via a memory bus, and DRAM 1070 can be connected to NVM 1060 via the memory bus.

[0108] As previously mentioned, in the electronic device shown in FIG5 , DRAM 1070 can be connected to processor 1010 via a memory bus. DRAM 1070 has the advantage of fast access speed. Processor 1010 can access DRAM 1070 at high speed and perform read or write operations on DRAM 1070. DRAM 1070 is typically used to store various running software in the operating system, input and output data, and information exchanged with external memory. However, DRAM 1070 is volatile; when the power is turned off, the information in DRAM 1070 is no longer retained.

[0109] Because the new NVM can be addressed by byte and data can be written to the non-volatile memory in bits, it can be used as memory. In the embodiment of the present disclosure, NVM 1060 can be used together with DRAM 1070 as the memory of the electronic device 1000. Compared with DRAM 1070, NVM 1060 has the characteristic of non-volatility, which can better save data. In the embodiment of the present disclosure, the non-volatile memory that can be used as memory can be referred to as storage class memory (SCM).

[0110] It should be noted that DRAM is a type of volatile memory. In practical applications, other random access memories (RAM) can also be used as the memory of the computer system. For example, static random access memory (SRAM) can also be used as the memory of the computer system. The NVM 1060 shown in Figure 5 may include: phase-change random access memory (PCM), resistive random access memory (RRAM), magnetic random access memory (MRAM) or ferroelectric random access memory (FRAM) and other new non-volatile memories. The specific type of NVM in the embodiment of the present disclosure is not limited here.

[0111] Because NVM 1060 has a relatively slow access speed compared to DRAM 1070, NVM 1060 is typically used as the system's main memory, while DRAM 1070 is used as a cache for NVM 1060 to compensate for the slow access speed of main memory NVM 1060 and improve memory access speed. As shown in FIG5 , in the electronic device shown in FIG5 , DRAM 1070 serves as a cache for NVM 1060. When memory controller 1030 receives a memory access request from processor 1010, it first determines whether the target address in the memory access request (i.e., the address of the memory block to be accessed) matches DRAM 1070 to determine whether the data to be accessed is stored in DRAM 1070. If it is determined that the target address in the memory access request matches DRAM 1070, memory controller 1030 can directly obtain the data to be accessed from DRAM 1070, thereby reducing access latency. When the memory controller 1030 determines that the target address in the memory access request does not hit the DRAM 1070 , the memory controller 1030 obtains the data to be accessed from the NVM 1060 .

[0112] According to an embodiment of the present disclosure, for the electronic device 1000 shown in FIG5 , the CPU core 1020 can be configured to obtain performance data representing the usage rate of memory (e.g., DRAM 1070); and adjust the aggressiveness level of the prefetcher 1050 based on the performance data, wherein the aggressiveness level is used to control the confidence condition for the prefetch generated by the prefetcher 1050 and the number of prefetches. The prefetcher 1050 can be configured to perform prefetch operations based on the adjusted aggressiveness level.

[0113] According to some embodiments of the present disclosure, the performance data includes one or more of the following: average access latency of memory; occupancy rate of miss status processing registers of the electronic device; bandwidth usage rate of memory; and occupancy rate of queues in an on-chip interconnect network of the electronic device.

[0114] According to some embodiments of the present disclosure, the prefetcher 1050 is a stride prefetcher.

[0115] According to some embodiments of the present disclosure, when the performance data includes the average access delay of the memory, the CPU core 1020 obtains the performance data characterizing the memory usage, including: obtaining the historical average access delay time of the memory before the time window; obtaining the current average access delay time of the memory within the time window; and calculating the weighted average of the historical average access delay time and the current average access delay time, wherein the weighted average is used as the average access delay of the memory.

[0116] According to some embodiments of the present disclosure, a time window counter, a delay counter, a backfill counter and a historical delay counter are configured in the electronic device, and the CPU core 1020 obtains performance data representing the memory usage, including: using the time window counter to record the time window, wherein when the time window counter reaches a set threshold, it triggers the calculation of the average access delay and adjusts the aggressiveness level according to the average access delay; using the delay counter to record the total delay time of cache misses in the time window; using the backfill counter to record the number of access backfills of cache misses in the time window; using the historical delay counter to record the historical average access delay time before the time window; and calculating the current average access delay time based on the total delay time of cache misses and the number of access backfills in the time window, and performing weighted averaging on the current average access delay time and the historical average access delay time to calculate the average access delay.

[0117] According to some embodiments of the present disclosure, the CPU core 1020 adjusts the aggressiveness level of the prefetcher according to the performance data, including: when the average access delay is less than a first threshold, increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory; and when the average access delay is greater than a second threshold, reducing the aggressiveness level of the prefetcher so as to reduce the prefetch amount for the memory.

[0118] According to some embodiments of the present disclosure, an electronic device synchronously processes multiple threads, and obtaining performance data representing memory usage includes: obtaining performance data representing memory usage for each of the multiple threads. Adjusting the aggressiveness level of a prefetcher based on the performance data includes: adjusting the aggressiveness level of the prefetcher for each thread based on the respective performance data, wherein the aggressiveness level of the prefetcher is set separately for each thread.

[0119] According to some embodiments of the present disclosure, the CPU core 1020 uses a time window counter to record a time window including one of the following: for a processor clock cycle of an electronic device, the count is increased by one for each clock cycle; for a cache access of an electronic device, the count is increased by one for each access; for a cache access miss event of an electronic device, the count is increased by one for each cache access miss; or for a prefetch request of a prefetcher, the count is increased by one for each prefetch request sent.

[0120] According to some embodiments of the present disclosure, the cache 1040 may include a multi-level cache, and the cache miss is a cache miss at the last level of the multi-level cache, ie, LLC Miss.

[0121] The electronic device according to the embodiments of the present disclosure can implement the specific execution process of functions such as data access and prefetching. In addition, the electronic device according to the embodiments of the present disclosure can specifically implement the steps of the data prefetching method according to some embodiments of the present disclosure described above in conjunction with the accompanying drawings, and the description is not repeated here. The electronic device using the embodiments of the present disclosure can perform similar data prefetching processes and achieve similar technical effects.

[0122] According to another aspect of the present disclosure, a data pre-fetching device is provided. Figure 6 shows a schematic block diagram of a data pre-fetching device according to some embodiments of the present disclosure.

[0123] As shown in FIG6 , the data prefetching device 2000 includes an acquisition unit 2010, an adjustment unit 2020, and a prefetching unit 2030. According to some embodiments of the present disclosure, the acquisition unit 2010 can be configured to acquire performance data representing memory usage. The adjustment unit 2020 can be configured to adjust the aggressiveness level of the prefetcher based on the performance data, wherein the aggressiveness level is used to control the confidence condition for the prefetcher to generate prefetches and the number of prefetches. The prefetching unit 2030 can be configured to perform prefetch operations based on the adjusted aggressiveness level.

[0124] According to some embodiments of the present disclosure, the performance data includes one or more of the following: average access latency of memory; occupancy rate of miss status processing registers of the electronic device; bandwidth usage rate of memory; and occupancy rate of queues in an on-chip interconnect network of the electronic device.

[0125] According to some embodiments of the present disclosure, when the performance data includes the average access delay of the memory, the acquisition unit 2010 obtains the performance data characterizing the memory usage rate, including: obtaining the historical average access delay time of the memory before the time window; obtaining the current average access delay time of the memory within the time window; and calculating the weighted average of the historical average access delay time and the current average access delay time, wherein the weighted average is used as the average access delay of the memory.

[0126] According to some embodiments of the present disclosure, the acquisition unit 2010 acquires performance data representing memory usage, including: using a time window counter to record a time window, wherein when the time window counter reaches a set threshold, it triggers the calculation of an average access delay and adjusts the aggressiveness level according to the average access delay; using a delay counter to record the total delay time of cache misses within the time window; using a backfill counter to record the number of access backfills of cache misses within the time window; using a historical delay counter to record the historical average access delay time before the time window; and calculating the current average access delay time based on the total delay time of cache misses and the number of access backfills within the time window, and performing a weighted average of the current average access delay time and the historical average access delay time to calculate the average access delay.

[0127] According to some embodiments of the present disclosure, the adjustment unit 2020 adjusts the aggressiveness level of the prefetcher according to the performance data, including: when the average access delay is less than a first threshold, increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory; and when the average access delay is greater than a second threshold, reducing the aggressiveness level of the prefetcher so as to reduce the prefetch amount for the memory.

[0128] According to some embodiments of the present disclosure, the acquisition unit 2010 acquires performance data characterizing memory usage, including: for each thread among multiple threads, separately acquiring performance data characterizing memory usage; and the adjustment unit 2020 adjusts the aggressiveness level of the prefetcher according to the performance data, including: for each thread, separately adjusting the aggressiveness level of the prefetcher according to the respective performance data, wherein the aggressiveness level is set for each thread in the prefetcher.

[0129] According to some embodiments of the present disclosure, recording the time window using a time window counter includes one of the following: for a processor clock cycle of an electronic device, the count is increased by one for each clock cycle; for a cache access of an electronic device, the count is increased by one for each access; for a cache access miss event of an electronic device, the count is increased by one for each cache access miss; or for a prefetch request of a prefetcher, the count is increased by one for each prefetch request sent.

[0130] According to some embodiments of the present disclosure, the cache includes a multi-level cache, and the miss cache is a last-level cache that misses the multi-level cache.

[0131] According to some embodiments of the present disclosure, the prefetcher is a stride prefetcher.

[0132] Specifically, the data prefetching device 2000 according to the embodiment of the present disclosure can implement the steps of the data prefetching method according to the embodiment of the present disclosure described above in conjunction with the drawings and achieve similar technical effects, which will not be repeated here.

[0133] According to another aspect of the present disclosure, an electronic device is provided. Figure 7 shows a schematic block diagram of an electronic device according to some embodiments of the present disclosure.

[0134] As shown in Figure 7, the electronic device 3000 may include a processor 3010 and a memory 3020. According to some embodiments of the present disclosure, the memory 3020 stores computer readable code, which, when executed by the processor 3010, may execute the data prefetching method provided according to embodiments of the present disclosure.

[0135] The processor 3010 can perform various actions and processes according to the program stored in the memory 3020. Specifically, the processor 3010 can be an integrated circuit with signal processing capabilities. A general-purpose processor can be a microprocessor or any conventional processor. For example, the processor can be a processor capable of implementing SMT.

[0136] The memory 3020 stores computer-readable code that, when executed by the processor 3010, can cause the processor to implement the data prefetching method according to some embodiments of the present disclosure. The memory 3020 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. It should be noted that the memory described herein can be any suitable type of memory. As an example, by executing the computer-readable code in the memory 3020, the processor can implement the steps of the data prefetching method described above in conjunction with the accompanying drawings.

[0137] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium is further provided. Figure 8 shows a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure.

[0138] As shown in Figure 8, computer-readable instructions 4010 are stored on the computer-readable storage medium 4000. When the computer-readable instructions 4010 are executed by the processor, the data prefetching method described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 4000 can be connected to an electronic device such as a computer, and then, when the electronic device runs the computer-readable instructions 4010 stored on the computer-readable storage medium 4000, the data prefetching method provided according to the embodiments of the present disclosure as described above can be performed.

[0139] Embodiments of the present disclosure provide a data prefetching method, apparatus, electronic device, electronic device, and medium for adjusting the aggressiveness level of prefetching performed by a prefetcher based on performance data representing memory usage, thereby enabling the prefetcher to dynamically adjust the conditions for prefetching based on the set aggressiveness level, thereby improving the accuracy and coverage of prefetching, and thereby improving the overall operating performance of the electronic device.

[0140] Furthermore, although the present disclosure makes various references to certain units in electronic devices and electronic apparatuses according to embodiments of the present disclosure, any number of different units may be used and run on the client and / or server. The units are merely illustrative, and different aspects of the methods, electronic devices, and electronic apparatuses may use different units.

[0141] Those skilled in the art will appreciate that all or part of the steps in the above method can be performed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software functional modules. The present disclosure is not limited to any particular combination of hardware and software.

[0142] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or highly formal sense unless expressly defined as such herein.

[0143] The above is an illustration of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the above is an illustration of the present disclosure and should not be considered as limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A data prefetching method, wherein: The method is applicable to an electronic device, wherein the electronic device includes a memory, a cache, and a prefetcher, and the method includes: Obtaining performance data representing the usage of the memory; Adjusting an aggressiveness level of the prefetcher according to the performance data, wherein the aggressiveness level is used to control a confidence condition and a number of prefetches generated by the prefetcher; and Prefetch operations are performed based on the adjusted aggressiveness level.

2. The method according to claim 1, wherein The performance data includes one or more of the following: an average access latency of the memory; an occupancy rate of a miss status processing register of the electronic device; Bandwidth usage of the memory; and The occupancy rate of the queue in the on-chip interconnection network of the electronic device.

3. The method according to claim 2, wherein: In a case where the performance data includes an average access delay of the memory, obtaining the performance data representing the usage rate of the memory includes: Obtaining a historical average access delay time of the memory before the time window; Obtaining a current average access delay time of the memory within the time window; and A weighted average of the historical average access delay time and the current average access delay time is calculated, wherein the weighted average is used as the average access delay of the memory.

4. The method according to claim 3, wherein: The electronic device is configured with a time window counter, a delay counter, a backfill counter, and a historical delay counter, and obtaining performance data representing the usage rate of the memory includes: Recording a time window using the time window counter, wherein when the time window counter reaches a set threshold, triggering calculation of the average access delay and adjusting the aggressiveness level according to the average access delay; Using the delay counter to record the total delay time of cache misses within the time window; Using the backfill counter to record the number of cache miss access backfills within the time window; Using the historical delay counter to record the historical average access delay time before the time window; and The current average access delay time is calculated based on the total delay time of cache misses in the time window and the number of access backfills, and the current average access delay time and the historical average access delay time are weighted averaged to calculate the average access delay.

5. The method according to claim 4, wherein Adjusting the aggressiveness level of the prefetcher according to the performance data includes: In a case where the average access delay is less than a first threshold, increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory; and When the average access delay is greater than a second threshold, the aggressiveness level of the prefetcher is reduced to reduce the prefetch amount for the memory.

6. The method according to any one of claims 1 to 5, wherein The electronic device synchronously processes multiple threads, and obtaining performance data representing the usage rate of the memory includes: For each of the plurality of threads, respectively obtain performance data representing a usage rate of the memory; and Adjusting the aggressiveness level of the prefetcher according to the performance data includes: For each thread, the aggressiveness level of the prefetcher is adjusted according to the respective performance data, wherein the aggressiveness level of the prefetcher is set for each thread separately.

7. The method according to claim 4, wherein: The recording of the time window by using the time window counter includes one of the following: For a processor clock cycle of the electronic device, a count is incremented by one for each clock cycle; For cache access of the electronic device, the access count is incremented by one each time; For a cache access miss event of the electronic device, a cache access miss count is incremented by one each time; or For the prefetch request of the prefetcher, a count of each sent prefetch request is increased by one.

8. The method according to claim 4, wherein The cache includes multiple levels of cache, and the miss cache is a cache at the last level of the multiple levels of cache that misses.

9. An electronic device, wherein: The electronic device includes a processor core, a memory, a cache and a prefetcher, The processor core is configured to: obtain performance data representing the usage of the memory, and adjust an aggressiveness level of the prefetcher according to the performance data, wherein the aggressiveness level is used to control a confidence condition and a number of prefetches generated by the prefetcher; and The prefetcher is configured to perform a prefetch operation according to the adjusted aggressiveness level.

10. The electronic device according to claim 9, wherein The performance data includes one or more of the following: an average access latency of the memory; an occupancy rate of a miss status processing register of the electronic device; Bandwidth usage of the memory; and The occupancy rate of the queue in the on-chip interconnection network of the electronic device.

11. The electronic device according to claim 10, wherein: In a case where the performance data includes an average access delay of the memory, the processor core acquiring the performance data representing the usage rate of the memory includes: Obtaining a historical average access delay time of the memory before the time window; Obtaining a current average access delay time of the memory within the time window; and A weighted average of the historical average access delay time and the current average access delay time is calculated, wherein the weighted average is used as the average access delay of the memory.

12. The electronic device according to claim 11, wherein The electronic device is configured with a time window counter, a delay counter, a backfill counter, and a historical delay counter, and the processor core obtains performance data representing the usage rate of the memory including: Recording a time window using the time window counter, wherein when the time window counter reaches a set threshold, triggering calculation of the average access delay and adjusting the aggressiveness level according to the average access delay; Using the delay counter to record the total delay time of cache misses within the time window; Using the backfill counter to record the number of cache miss access backfills within the time window; Using the historical delay counter to record the historical average access delay time before the time window; and The current average access delay time is calculated based on the total delay time of cache misses in the time window and the number of access backfills, and the current average access delay time and the historical average access delay time are weighted averaged to calculate the average access delay.

13. The electronic device according to claim 12, wherein: The processor core adjusting the aggressiveness level of the prefetcher according to the performance data includes: In a case where the average access delay is less than a first threshold, increasing the aggressiveness level of the prefetcher so as to increase the prefetch amount for the memory; and When the average access delay is greater than a second threshold, the aggressiveness level of the prefetcher is reduced to reduce the prefetch amount for the memory.

14. The electronic device according to any one of claims 9 to 13, wherein: The electronic device synchronously processes multiple threads, and obtaining performance data representing the usage rate of the memory includes: For each of the plurality of threads, respectively obtain performance data representing a usage rate of the memory; and Adjusting the aggressiveness level of the prefetcher according to the performance data includes: For each thread, the aggressiveness level of the prefetcher is adjusted according to the respective performance data, wherein the aggressiveness level of the prefetcher is set for each thread separately.

15. The electronic device according to claim 12, wherein The cache includes multiple levels of cache, and the miss cache is a cache at the last level of the multiple levels of cache that misses.

16. A data pre-fetching device, wherein: The device comprises: an acquisition unit configured to acquire performance data representing memory usage; an adjusting unit configured to adjust an aggressiveness level of the prefetcher according to the performance data, wherein the aggressiveness level is used to control a confidence condition for the prefetcher to generate prefetches and a number of prefetches; and The prefetch unit is configured to perform a prefetch operation according to the adjusted aggressiveness level.

17. The data pre-fetching device according to claim 16, wherein: In a case where the performance data includes an average access delay of a memory, obtaining the performance data representing the usage rate of the memory includes: Obtaining a historical average access delay time of the memory before the time window; Obtaining a current average access delay time of the memory within the time window; and A weighted average of the historical average access delay time and the current average access delay time is calculated, wherein the weighted average is used as the average access delay of the memory.

18. The data pre-fetching device according to claim 16, wherein: The data prefetching device synchronously processes multiple threads, and the acquiring of performance data representing the usage rate of the memory includes: For each of the plurality of threads, respectively obtain performance data representing a usage rate of the memory; and Adjusting the aggressiveness level of the prefetcher according to the performance data includes: For each thread, the aggressiveness level of the prefetcher is adjusted according to the respective performance data, wherein the aggressiveness level of the prefetcher is set for each thread separately.

19. An electronic device, wherein: The electronic device includes at least one processor and at least one memory, wherein the at least one memory stores computer readable code, which, when executed by the at least one processor, causes the at least one processor to perform the steps of the method according to any one of claims 1 to 8.

20. A non-transitory computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps of the method according to any one of claims 1 to 8.

Images