Data prefetching method and apparatus

Abstract

Disclosed are a data prefetching method and an apparatus, relating to the technical field of computers. The data prefetching method comprises: acquiring a control flow graph and events, recorded by a PMU, of basic blocks in the control flow graph; and, on the basis of the number of occurrences of the events of the basic blocks in the control flow graph, determining a target path among a plurality of paths in the control flow graph and, before a first basic block is executed, writing into a cache target data corresponding to the first basic block, the sum of the number of occurrences of all of the events of the basic blocks in the target path being greater than or equal to a first threshold value, and the target data being data among data required to be accessed during execution of the target path and of which the number of times of access or the frequency of access is greater than or equal to a second threshold value. Since the sum of the number of occurrences corresponding to the target path is greater than or equal to the first threshold value, the target path has a relatively high probability to be frequently executed. Thus, writing in advance into the cache the target data determined on the basis of the target path can improve data access efficiency when a processor performs execution on the basis of the control flow graph.

Description

A data prefetching method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202410504625.5, filed on April 24, 2024, entitled “A Data Prefetching Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a data prefetching method and apparatus. Background Technology

[0003] In processor architecture, a cache is set up between the processor and memory to temporarily store data between the processor and memory. The cache capacity is relatively limited. To balance cache capacity with processor data access efficiency, the cache typically stores data that the processor frequently accesses. Currently, compilers can perform static branch prediction on the source code. Before the source code is executed, the execution probability of each branch in the source code obtained from the static branch prediction is used to determine the branch path with the highest execution probability. Therefore, the data to be stored in the cache is the data that will be frequently accessed when executing according to that branch path.

[0004] However, during static branch prediction, when local branch probabilities are equal or zero, it is impossible to determine which branch to execute when executing the source code. This leads to inaccurate branch paths, and the data determined based on these branch paths may not be the data that the processor will frequently access. Storing the determined data in the cache will result in low overall efficiency for the processor to access data. Summary of the Invention

[0005] This application provides a data prefetching method and apparatus to solve the problem that the data written to the cache is not necessarily the data that the processor frequently accesses, resulting in low overall efficiency of data access by the processor.

[0006] The technical solution adopted in this application is as follows.

[0007] Firstly, this application provides a data prefetching method. This data prefetching method can be applied to a computer system or a computing device that implements the data prefetching method within that computer system. The computing device may be a server or a terminal. The data prefetching method includes: acquiring a control flow graph; and, based on the events of basic blocks in the control flow graph recorded by a performance monitoring unit (PMU), determining target paths for multiple paths in the control flow graph according to the occurrence frequency of the events of the basic blocks in the control flow graph. Furthermore, before executing the first basic block in the control flow graph, the target data corresponding to the first basic block is written to a buffer. Wherein, the sum of the occurrence frequencies of all events in the basic blocks of the target path is greater than or equal to a first threshold, and the target data is data whose access frequency or access frequency is greater than or equal to a second threshold among the data required to access when executing the target path.

[0008] In this application, the Processing Unit (PMU) records the occurrence counts of various events when the processor executes instructions in a basic block. The more times the PMU records the occurrence counts of events for a basic block, the more frequently the instructions in that basic block are executed. Furthermore, based on the occurrence counts of events for a basic block recorded by the PMU, a target path is determined from the multiple paths included in the control flow graph where the sum of the occurrence counts of events for multiple basic blocks is greater than or equal to a first threshold. This target path has a higher probability of being executed frequently, and the access frequency or access frequency of the target data determined based on the target path is also higher. Therefore, writing the data (target data) required for the execution of the first basic block in the control flow graph into the cache can improve the overall efficiency of data access when the processor executes according to the control flow graph.

[0009] In one possible example, the first basic block includes instructions for accessing the target data.

[0010] In one possible example, the target data can be written to a cache and persisted when the hint instruction is executed, by deploying a hint instruction in the first basic block.

[0011] For example, a hint instruction can be deployed on a line adjacent to the instruction for accessing target data included in the first basic block.

[0012] In one possible scenario, the basic blocks in the control flow graph include basic blocks that are inside nested loops.

[0013] In this application, since hot functions or code snippets are usually nested loop structures, the processor determines the control flow graph including nested loops (structures) from all control flow graphs, avoiding the need to perform subsequent target data identification and data prefetching on all control flow graphs, thus improving the efficiency of writing the data required by hot functions or code snippets into the cache.

[0014] In one possible scenario, the sum of the occurrence counts of all events of the basic blocks included in each of the multiple paths is a candidate value, and one of the candidate values ​​among the multiple paths is a first threshold.

[0015] In one possible example, the first threshold is the maximum value among multiple candidate values. That is, the target path is the path with the largest sum of the occurrences of all events among multiple paths.

[0016] In this application, the path with the highest sum of the occurrence counts of all events is selected as the target path from multiple paths. This target path has the highest probability of being executed frequently among the multiple paths. Therefore, the access frequency or access frequency of the target data determined based on this target path is also the highest. Thus, writing the target data required for the execution of the first basic block in the control flow graph into the cache can improve the overall efficiency of data access when the processor executes according to the control flow graph.

[0017] In another possible example, the first threshold is any one of the top N candidate values ​​from a pool of candidate values. N is a positive integer greater than or equal to 1.

[0018] In one possible implementation, determining the target path of multiple paths in the control flow graph based on the occurrence count of events of basic blocks in the control flow graph includes: searching multiple paths in the control flow graph starting from the initial block, and then determining the sum of the occurrence counts of events of basic blocks included in each of the multiple paths. Therefore, based on the sum of the occurrence counts of events of basic blocks included in each of the multiple paths, a target path is determined from the multiple paths whose sum of the occurrence counts of events is greater than or equal to a first threshold. Notably, the exiting basic block of each of the multiple paths does not belong to a basic block in the control flow graph that is within a nested loop.

[0019] In this application, the control flow graph is searched starting from the initial block in the control flow graph to obtain multiple paths. The exit basic block of each of these multiple paths does not belong to the basic block in the control flow graph that is in a nested loop. This ensures that the multiple paths obtained are relatively long paths (including paths with a large number of basic blocks), reduces the additional performance loss caused by judging paths with a small number of basic blocks, and improves the efficiency of determining the target path.

[0020] In one possible example, a depth-first search can be used to search the control flow graph.

[0021] In one possible scenario, each of the aforementioned multiple paths includes all the basic blocks in the control flow graph that are within nested loops.

[0022] In this application, since each path includes all basic blocks within a nested loop, when searching for paths in the control flow graph, the path will not exit from the last basic block within the nested loop to the exiting basic block in the control flow graph. That is, all basic blocks within the nested loop are fully included in each path. This ensures that each path is a relatively long path (including paths with a large number of basic blocks), avoiding the inclusion of shorter paths in the sum of event occurrences, which would cause additional performance overhead and improve the efficiency of determining the target path.

[0023] In one possible implementation, before executing the first basic block in the control flow graph, the target data corresponding to the first basic block is written to a cache. This includes: parsing the source code corresponding to the target path to obtain the target data, and then determining the highest-priority first basic block from among the multiple basic blocks included in the control flow graph. Thus, the target data is written to the cache before executing the first instruction included in the first basic block. Here, the reuse score of the target data is the top M reuse scores among the multiple data reuse scores indicated by the source code, where M is an integer greater than or equal to 1. The reuse score of the target data is used to indicate the degree of reuse of the target data, and each of the multiple basic blocks includes a first instruction for accessing the target data.

[0024] In this application, since the first basic block is the highest priority among multiple basic blocks, and this priority indicates the priority of writing the target data into the cache, writing the target data into the cache before executing the first instruction included in the first basic block can improve the overall efficiency of data access.

[0025] In one possible scenario, data reuse scores can be determined based on one or more of the following: data reuse method, memory access patterns, read / write types, and space usage limits.

[0026] In one possible scenario, the priority of a basic block is determined based on one or more of the following indicators: the loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0027] In one possible example, the priority of a basic block is determined in sequence based on one or more of the following indicators: the loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0028] Let's take the first instruction in basic block a (referred to as instruction a) and the first instruction in basic block b (referred to as instruction b) as examples. First, compare the loop depth of instruction a with the loop depth of instruction b. When the loop depths of instructions a and b are different, the basic block corresponding to the instruction with the higher loop depth has a higher priority. If the priority cannot be determined based on the loop depth, then the priority is determined sequentially based on access patterns, occurrence counts, line numbers, read / write target data, and vectorized / scalarized memory access, until the priority of instructions a and b is determined.

[0029] For more details on this example, please refer to the following specific implementation method, which will not be repeated here.

[0030] In one possible implementation, before executing the first basic block in the control flow graph and before writing the target data corresponding to the first basic block into the cache, the data prefetching method further includes: deploying a second instruction in the first basic block to indicate that the target data should be written into the cache; and deploying a computational basic block including a third instruction to calculate the space occupied by the target data before the initial basic block included in the control flow graph. The next basic block after the computational basic block is the initial basic block of the control flow graph and the initial basic block of the control flow graph including the second instruction. The space occupied by the target data is used to indicate the execution of the initial basic block of the control flow graph, or to execute the initial basic block of the control flow graph including the second instruction.

[0031] In this application, when the size of the target data is unknown, the size of the target data is calculated in advance to determine whether the target data should be written into the cache in advance, so as to balance the efficiency of data access and the capacity of the cache, thereby improving the overall efficiency of data access while meeting the capacity of the cache.

[0032] Secondly, this application provides a data prefetching apparatus. This data prefetching apparatus is applied to a computer system or to a computing device supporting the computer system in implementing a data prefetching method. The data prefetching apparatus includes modules for executing the data prefetching method in the first aspect or any optional implementation of the first aspect. For example, the data prefetching apparatus includes: an acquisition module, a determination module, and a writing module.

[0033] The acquisition module is used to acquire the control flow graph, as well as the events of the basic blocks in the control flow graph recorded by the performance monitoring unit (PMU).

[0034] The determination module is used to determine the target path of multiple paths in the control flow graph based on the occurrence count of events of basic blocks in the control flow graph. The sum of the occurrence counts of all events of basic blocks in the target path is greater than or equal to a first threshold.

[0035] The write module is used to write the target data corresponding to the first basic block into the cache before executing the first basic block in the control flow graph. The target data is the data that needs to be accessed when executing the first basic block. The target data is the data whose access frequency or access frequency is greater than or equal to the second threshold among the data that needs to be accessed when executing the target path.

[0036] In one possible implementation, the sum of the occurrence counts of all events of the basic blocks included in each of the multiple paths is a candidate value, and one of the candidate values ​​of the multiple paths is a first threshold.

[0037] In one possible implementation, the first threshold is the maximum value among multiple candidate values.

[0038] In one possible implementation, a determining module is specifically used to search multiple paths in the control flow graph starting from the initial basic block; the exit basic block of each path in the multiple paths does not belong to the basic blocks in the control flow graph that are in nested loops; the sum of the occurrence counts of events of the basic blocks included in each path in the multiple paths is determined; and based on the sum of the occurrence counts of events of the basic blocks included in each path in the multiple paths, a target path is determined from the multiple paths whose sum of the occurrence counts of events is greater than or equal to a first threshold.

[0039] In one possible implementation, a writing module is specifically used to parse the source code corresponding to the target path to obtain the target data. Then, it determines the highest-priority first basic block from among the multiple basic blocks included in the control flow graph. Before executing the first instruction included in the first basic block, the target data is written to a cache. Here, the reuse score of the target data is the top M reuse scores among the multiple data reuse scores indicated by the source code, where M is an integer greater than or equal to 1. The reuse score of the target data is used to indicate the degree of reuse of the target data, and each of the multiple basic blocks includes a first instruction for accessing the target data.

[0040] In one possible implementation, the priority of a basic block is determined based on one or more of the following indicators: the loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0041] In one possible implementation, the priority of a basic block is determined sequentially based on one or more of the following indicators: the loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0042] In one possible implementation, the basic blocks in the control flow graph include basic blocks that are nested within loops.

[0043] In one possible implementation, the apparatus further includes: a dynamic prediction module; the dynamic prediction module is configured to deploy a second instruction in the first basic block to instruct the target data to be written to a cache; and to deploy a computational basic block including a third instruction to calculate the space occupied by the target data before the initial basic block included in the control flow graph. The next basic block after the computational basic block is the initial basic block of the control flow graph and the initial basic block of the control flow graph including the second instruction; the space occupied by the target data is used to instruct the execution of the initial basic block of the control flow graph, or to execute the initial basic block of the control flow graph including the second instruction.

[0044] In one possible implementation, each of the multiple paths includes all the basic blocks in the control flow graph that are within nested loops.

[0045] Thirdly, this application provides a chip, including: a processor and a power supply circuit; the power supply circuit is used to supply power to the processor, and the processor is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0046] Fourthly, this application provides a computing device, including a memory and a processor, wherein the memory is used to store computer instructions; when the processor executes the computer instructions, it implements the method in the first aspect or any possible implementation of the first aspect.

[0047] Fifthly, this application provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processing device, implement the method described in the first aspect or any possible implementation of the first aspect.

[0048] In a sixth aspect, this application provides a computer program product comprising a computer program or instructions that, when executed by a processing device, implement the method in the first aspect or any possible implementation thereof.

[0049] The beneficial effects of aspects two through six above can be referred to in the first aspect or any possible implementation of the first aspect, and will not be elaborated here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. Attached Figure Description

[0050] Figure 1 is a schematic diagram of a cache structure;

[0051] Figure 2 is a schematic diagram of the architecture of an HBM allocation system;

[0052] Figure 3 is a schematic diagram of a computer system provided in this application;

[0053] Figure 4 is a flowchart illustrating a compilation method provided in this application;

[0054] Figure 5 is a flowchart illustrating the data processing method provided in this application;

[0055] Figure 6 is a flowchart illustrating a target path determination method provided in this application;

[0056] Figure 7 is a flowchart illustrating a method for writing target data into a cache according to this application;

[0057] Figure 8 is a flowchart illustrating the priority determination method provided in this application;

[0058] Figure 9 is a complete flowchart of a data prefetching method provided in this application;

[0059] Figure 10 is a schematic diagram of the transformation of the directed acyclic graph provided in this application;

[0060] Figure 11 is a control flow graph of dynamic multi-branch insertion provided in this application;

[0061] Figure 12 is a schematic diagram of the structure of a data prefetching device provided in this application;

[0062] Figure 13 is a schematic diagram of the structure of a data prefetching device provided in this application;

[0063] Figure 14 is a schematic diagram of the structure of a computing device provided in this application. Detailed Implementation

[0064] To facilitate understanding, the technical terms used in this application will be introduced first.

[0065] Source code is a text file written by programmers. Programmers write code in human-readable languages, such as Java, C++, and C#. Source code follows the conventions and rules of a particular language. While source code is human-readable, it is not readable by computing devices (such as processors).

[0066] Executable code is code that a computing device can recognize and execute; it contains binary instructions that the computing device can understand. A compiler can convert source code into executable code.

[0067] The Processing Unit (PMU) is a hardware device typically located inside the processor. The PMU is primarily used to track and count low-level hardware events of the processor, such as processor-related events (e.g., the number of instruction executions, the number of exceptions caught, and the number of processor clock cycles) and cache-related events (e.g., the number of times each level of the cache is accessed, and the number of cache misses). These events characterize the processor's behavior during the execution of executable code.

[0068] Source code must adhere to certain coding standards during its development, such as object-oriented programming (OOP) and procedural programming (POP). Languages ​​for object-oriented programming include C++, Java, and C#, while languages ​​for procedural programming include Fortran and C.

[0069] Source code written according to certain coding standards can include code elements such as functions / methods, fields / variables.

[0070] A function / method refers to a subroutine within a class. A method typically consists of a series of statements and performs a specific function. In object-oriented programming, it is called a method; in procedural programming, it is called a function. In this embodiment, it is referred to as a "function".

[0071] A loop is a type of program that may be contained within a function. A loop is a piece of code that needs to be executed multiple times if a certain condition is met, and can exit the loop when the condition is no longer met. Alternatively, a loop is a piece of code that needs to be executed multiple times if a certain condition is not met, and can exit the loop when the condition is met.

[0072] Fields / variables store data such as integers and characters, strings, hash tables, pointers, etc. Fields / variables can be used as the data mentioned in the embodiments of this application, such as target data.

[0073] Memory access refers to data access operations performed in the source code. Memory access can be categorized into vectorized memory access and scalarized memory access. Vectorized memory access involves operating on multiple memory addresses at once, allowing the processor to acquire multiple data points simultaneously for parallel computation. Scalarized memory access involves accessing memory addresses one by one, where the processor acquires individual data points sequentially and performs computations on each one.

[0074] A memory access node consists of a base address and an index. A memory access node can be represented as [A, index], where A represents the base address, which is the address used to access the data, and index indicates the location of the data to be accessed at that base address.

[0075] Memory access instructions are instructions that include access nodes. There are many types of memory access instructions. One example is an assignment instruction, which assigns data from the location pointed to by index I at base address A to variable P. Executing this assignment instruction requires reading data from the location pointed to by index I at base address A to determine the value to be assigned to variable P. Memory access instructions with different semantics include different memory access nodes, and the number of memory access nodes also varies.

[0076] A control flow graph (CFG) is an abstract representation of a process or program, and is an abstract data structure used in compilers. Maintained internally by the compiler, it represents all paths traversed during program execution. It uses a graph to represent the possible flow of execution for all basic blocks within a process, and can also reflect the real-time execution process of a process.

[0077] A basic block (BB) is also called a foundational block. For any given function, the compiler breaks it down into multiple independent sets of instructions according to a preset standard. Each independent set of instructions is called a BB. The first BB executed in a function is the function's entry BB or initial BB, and the last BB executed is the function's exit BB or exit block. The first BB executed in a loop is the loop's header basic block (header BB). Each BB represents a continuous segment of code in the program without branches or jumps.

[0078] A function exists at a point where it can jump to multiple execution paths. This point is typically the location of statements with conditional statements, such as if, while, or else statements. When execution reaches this point, only one of these execution paths can be executed at a time. Each execution path can be called a branch. The probability that any branch will be executed is called the branch probability.

[0079] The branch probability is related to the statement with a conditional decision at that position. In this application, the compiler can analyze the specific semantics of the statement with a conditional decision at that position to estimate the branch probability of each branch. For example, if the value of x is an integer from 1 to 10, and the statement with a conditional decision at that position is if x>8, then it can be said that this position will be divided into two branches: one branch for x>8 and one branch for x<8. Based on the range of x values, the branch probability for x>8 is 20%, and the branch probability for x<8 is 80%.

[0080] High-performance computing (HPC) refers to the use of supercomputers, parallel computing clusters, or other high-performance computing systems to perform large-scale and complex computing tasks.

[0081] A cache is used to implement high-speed data buffering between the processor and main memory. The cache within a processor can include Level 1 cache (L1 cache), Level 2 cache (L2 cache), Level 3 cache (L3 cache), and last level cache (LLC). Typically, the read / write speeds of L1 cache, L2 cache, L3 cache, and LLC decrease sequentially, while their capacity usually increases gradually. The LLC is the last level cache located between the processor core and main memory. Among the caches shown above, the LLC is the furthest from the processor core, has the largest capacity, and the highest access latency. For example, this LLC may include high bandwidth memory (HBM).

[0082] HBM, as an on-chip resource, often has a smaller capacity due to limitations in hardware manufacturing processes. Compared to static random-access memory (SRAM) used as the last-level cache, HBM supports large-capacity data storage, effectively expanding the cache capacity to 200GB (gigabyte). HBM also boasts higher bandwidth, effectively improving data read and write speeds and making it more suitable for concurrent access scenarios. HBM is commonly used in high-performance computing, graphics processing units (GPUs), and artificial intelligence (AI), such as AI servers, to meet the ever-increasing demands for memory bandwidth and energy efficiency.

[0083] A memory footprint, also known as a memory access footprint, represents the maximum cache size used by a program during its execution.

[0084] A kernel function represents a hotspot function or code snippet (usually a code snippet with multiple nested loops).

[0085] In HPC scenarios, the HPC services executed are typically large-scale. Although horizontal scaling is used to distribute HPC tasks as parallel as possible across different node processes, the amount of tasks executed within a single node remains relatively large in order to avoid excessive cross-node communication and synchronization overhead. Therefore, maximizing the computing power of a single node is an important consideration in the architectural design of HPC hardware platforms.

[0086] The common data access patterns in HPC services are usually manifested as: (1) performing multiple iterations on data over a large scope.

[0087] (2) Multiple threads or processes on a single node concurrently access hot data.

[0088] For data iteration operations, hardware caching technology is often used to ensure that frequently accessed data resides in a cache closer to the chip, effectively mitigating latency increases caused by bursty memory accesses without user intervention. For concurrent data access, HBM (Hardware-Based Memory Management) is often used to mitigate the memory bandwidth bottleneck caused by high concurrency accesses, thereby delaying latency decay caused by memory accesses. HBM cache, combining both technologies, has emerged to effectively optimize latency decay caused by high-concurrency access to frequently accessed data without user intervention.

[0089] In traditional central processing unit (CPU) architectures, memory access from dynamic random-access memory (DRAM) to the processor core typically involves multiple levels of cache (such as L1, L2, and L3). In high-performance computing or CPU architectures with high memory access, HBM is commonly added to accelerate memory access. HBM can typically be configured in three usage modes: Flat mode, where the entire HBM is treated as DDR; cache mode; and hybrid mode, where HBM is divided into DDR and cache as needed. In cache mode, HBM is configured as LLC.

[0090] Figure 1 shows a schematic diagram of a cache structure. L1, L2, and the unified cache (UC) are arranged from closest to furthest from the processor core, with DRAM connected to the UC. When the HBM is in cache mode, the UC is connected to the HBM, acting as the LLC of the processor core. Registers are closest to the processor core.

[0091] Currently, to fully utilize HBM's storage resources, hot data—that is, frequently accessed data—is often identified in the source code. This frequently accessed data is then stored in HBM, and the processor core retrieves this data from the nearest HBM, thus improving data access efficiency.

[0092] The following provides a solution for identifying hot data in source code and storing it in HBM. Specifically, the compiler performs static analysis on the source code to identify target scenarios (i.e., memory-intensive access within a function scope and a footprint larger than the HBM capacity). It then models and analyzes data reuse within these target scenarios to determine the hot / cold rating or ranking of the data. Subsequently, it generates HBM prefetch instructions (such as hint instructions) for hot data with high hot / cold ratings (frequent access) or high rankings (in descending order), informing HBM to increase the residency level of hot data within HBM, thereby improving the processor core's data access efficiency.

[0093] As shown in Figure 2, Figure 2 is a schematic diagram of the architecture of an HBM allocation system.

[0094] ①: The compiler can identify memory-intensive kernels in the source code through static analysis.

[0095] ②: Probabilistic branch path identification, which involves combining and sorting kernels, determining the execution probability of each kernel through static analysis, and concatenating the kernels with high execution probability within the function. It also involves identifying data (variables) with high reusability within the concatenated kernels, such as tracing back to the original variable groups through memory access nodes within the kernels, and calculating the reusability of each variable group.

[0096] ③: Generate a hint instruction to instruct that highly reused data be resident in HBM. Insert the hint instruction at the first memory access node (i.e., the first occurrence location) of the highly reused data. When the processor core executes the corresponding hint instruction, it prompts HBM to resident the highly reused data. This hint instruction can be vectorized or non-vectorized.

[0097] The compiler can also obtain events recorded by the PMU in the CPU and update the predicted probabilities based on the feedback from the PMU events. The feedback data (profile) in Figure 2 includes the events recorded by the PMU.

[0098] However, in the static branch prediction process, the determined execution path is inaccurate when the local branch probabilities are equal or zero. Consequently, the data determined based on the aforementioned execution path may not be the data that the processor will frequently access. Storing the determined data in the cache will result in low overall efficiency of the processor's data access.

[0099] Based on this, this application provides a data prefetching method. The data prefetching method includes: acquiring a control flow graph, and, based on the events of basic blocks in the control flow graph recorded by the PMU, determining target paths for multiple paths in the control flow graph according to the occurrence count of events of the basic blocks in the control flow graph, and then, before executing the first basic block in the control flow graph, writing the target data corresponding to the first basic block into a buffer. Wherein, the sum of the occurrence counts of all events of the basic blocks in the target path is greater than or equal to a first threshold, and the target data is data whose access frequency or access frequency is greater than or equal to a second threshold among the data required to access when executing the target path.

[0100] In this application, the Processing Unit (PMU) records the occurrence counts of various events when the processor executes instructions in a basic block. The more times the PMU records the occurrence counts of events for a basic block, the more frequently the instructions in that basic block are executed. Furthermore, based on the occurrence counts of events for a basic block recorded by the PMU, a target path is determined from the multiple paths included in the control flow graph where the sum of the occurrence counts of events for multiple basic blocks is greater than or equal to a first threshold. This target path has a higher probability of being executed frequently, and the access frequency or access frequency of the target data determined based on the target path is also higher. Therefore, writing the data (target data) required for the execution of the first basic block in the control flow graph into the cache can improve the overall efficiency of data access when the processor executes according to the control flow graph.

[0101] The above method can be applied to the computer system (or computing device) shown in Figure 3, which is a schematic diagram of a computer system provided in this application. The computer system includes a processor 100 and a cache 200. Optionally, it may also include a memory 300.

[0102] The memory 300 can be understood as the memory of the processor 100, which may include the compiler 110 and the PMU 120.

[0103] It is worth noting that this application does not limit the specific form of the system; the system can be a computer chip, such as a system on chip (SoC).

[0104] The processor 100 shown in Figure 3 includes a compiler 110 with compilation capabilities. This compiler can be a software program running on the processor or a hardware module on the processor. Here, we are only illustrating the example of compiler 110 being a part of processor 100.

[0105] In some possible embodiments, compiler 110 may also be a software program or hardware module independent of processor 100, such as a software program or hardware module deployed on another processor. In this scenario, compiler 110 can compile executable code and then send the executable code to processor 100 so that processor 100 can execute the executable code.

[0106] The following description uses the computer system architecture shown in Figure 3 as an example. In scenarios where the compiler is independent of the processor, the functions of the compiler, processor, and PMU are the same as those of the compiler 110, processor 100, and PMU 120 shown in the computer system architecture of Figure 3.

[0107] This application does not limit the type of processor 100; any processor 100 capable of executing executable code is applicable to the embodiments of this application. The processor 100 can be a CPU, GPU, etc. The processor 100 can also be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0108] In one possible embodiment, the cache 200 in the computer system includes a multi-level cache. The last level cache (LLC) 210 in this multi-level cache can be HBM. The "multi-level cache" divides the cache 200 into multiple levels; the closer the cache level is to the core of the processor 100, the smaller its size, resulting in faster read / write speeds and relatively smaller capacity. In other words, LLC 210 is the cache furthest from the core of the processor 100 and has the largest capacity. LLC 210 can be used to cache data exchanged between the processor 100 and memory. From the processor 100's perspective, LLC 210 can be understood as the processor 100's cache. From the memory's perspective, LLC can also be used as a memory cache.

[0109] Compared to SRAM as an LLC 210, HBM supports large-capacity data storage and can effectively expand the capacity of cache 200. HBM also has higher bandwidth, which can effectively improve the data read and write speed of cache 200, making it more suitable for concurrent access scenarios.

[0110] In this application, the compiler 110 may also have analysis capabilities, enabling it to analyze source code to determine target data that the source code needs to access frequently or regularly, generate hint instructions for the target data in the source code, and insert them into the corresponding positions in the source code. These hint instructions instruct the target data to be persistently stored in an LLC. The compiler 110 then compiles the source code with the inserted hint instructions to generate executable code that the processor 100 needs to execute.

[0111] It is worth noting that this embodiment uses a hint instruction to indicate that the target data needs to be persistently stored in LLC 210 as an example. In other embodiments, the hint instruction can also indicate that the target data is persistently stored in other levels of cache included in cache 200, such as L3.

[0112] Processor 100 can obtain executable code compiled by compiler 110 and execute it. During the execution of the executable code, processor 100 can move data that needs to be processed from memory 300 to cache 200, and can also store the processed data in cache 200. When processor 100 executes the code corresponding to the hint instruction in the executable code, it can issue a resident instruction to LLC 210, which instructs LLC 210 to persistently store the target data.

[0113] During the execution of executable code by processor 100, PMU120 can monitor the process and record events related to processor 100 (especially events during the execution of executable code by processor 100) and events related to cache 200.

[0114] The compiler 110 can retrieve the events recorded by the PMU 120, update the determined target data, add hint instructions to the source code based on the updated target data, and compile to generate new executable code. The processor 100 can obtain and execute the new executable code.

[0115] The memory 300 is typically used to store computer program code that the processor 100 needs to execute. In this embodiment, the memory 300 can store source code that the compiler 110 needs to compile. The compiler 110 can call the source code from the memory 300 to analyze and compile the source code. The memory 300 can also store executable code generated after compiling the source code. After compiling and obtaining executable code, the compiler 110 can store the executable code in the memory 300, and the processor 100 can call and execute the executable code.

[0116] The memory 300 typically uses DRAM. Besides DRAM, the memory 300 can also be other random access memory, such as SRAM. Alternatively, the memory 300 can also be read-only memory (ROM). For example, a read-only memory can be a programmable read-only memory (PROM) or an erasable programmable read-only memory (EPROM). The memory 300 can also be a dual in-line memory module (DIMM), i.e., a module composed of dynamic random access memory (DRAM), or a solid-state drive (SSD). The memory 300 can also be a combination of the above-mentioned memory types. This application embodiment does not limit the number or type of memory 300.

[0117] In the above description, the target data is obtained by the compiler 110 through analyzing the loops included in the functions of the source code. This target data can be all or part of the data that the functions in the source code need to call during execution. In practical applications, programmers can annotate data that needs to be called frequently during the source code writing process. The compiler 110 can recognize the annotation and use the annotated data as the target data.

[0118] Regarding the compiler 110 shown in Figure 3, the following provides a method for the compiler 110 to compile source code.

[0119] As shown in Figure 4, Figure 4 is a flowchart illustrating a compilation method provided in this application. The compilation method shown in Figure 4 includes the following steps ①-⑦.

[0120] ①: Compiler 110 receives profiles sent by the core in processor 100, such as events collected by PMU.

[0121] ②: Compiler 110 performs lexical / syntactic analysis on the source code according to the compilation configuration file to obtain a syntax tree or other intermediate representation, such as an abstract syntax tree or intermediate representation (IR).

[0122] ③: Compiler 110 can convert the above intermediate representation to obtain IR in the form of control flow graph (CFG) or static single assignment (SSA).

[0123] ④: Compiler 110 performs static single-assignment optimization on the control flow graph or IR in SSA form to obtain the optimized control flow graph or IR in SSA form.

[0124] ⑤: Compiler 110 performs inter-process optimization on the optimized control flow graph or IR in SSA form to obtain a further optimized control flow graph or IR in SSA form. The aforementioned SSA optimization may include dead code elimination or constant propagation, etc.

[0125] ⑥: Compiler 110 performs register transfer language (RTL) optimization based on the optimized control flow graph or IR in SSA form to obtain the executable code corresponding to the source code.

[0126] ⑦: The processor core in processor 100 executes executable code. This executable code includes code corresponding to the hint instruction. When the processor core executes the code corresponding to the hint instruction, it sends a resident instruction to LLC 210. This resident instruction is used to instruct LLC 210 to store the target data, that is, to persistently store the target data in LLC 210.

[0127] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0128] As shown in Figure 5, Figure 5 is a flowchart illustrating the data processing method provided in this application. The method shown in Figure 5 can be executed by the processor 100 in Figure 3, specifically by the compiler 110, and the content shown in Figure 5 can be applied to step ④ shown in Figure 4. Taking the execution of this method by the processor 100 as an example, Figure 5 includes the following steps S510 to S530.

[0129] S510, processor 100 acquires the control flow graph.

[0130] In one possible implementation, processor 100 acquires intermediate data, such as control flow graphs, generated by compiler 110 during the compilation phase.

[0131] In one possible scenario, intermediate data generated by compiler 110 during the compilation phase can be stored in memory 300 or cache 200. Therefore, processor 100 can obtain the control flow graph from memory 300 or cache 200.

[0132] Control flow diagrams present the control flow of a program in a graphical form, making the control logic of the program more intuitive and easier to understand and analyze.

[0133] As shown in Figure 5, the control flow graph obtained by the processor 100 includes: BB-6, BB-3, BB-4, BB-7, and BB-5. Among them, BB-3, BB-4, and BB-7 are in loop 1, and BB-3 can exit to BB-5, and BB-4 can exit to BB-5.

[0134] In one possible scenario, the basic blocks in the control flow graph include basic blocks that are inside nested loops.

[0135] In this application, since hot functions or code snippets are usually nested loop structures, the processor 100 determines the control flow graph including nested loops (structures) from all control flow graphs, avoiding the need to perform subsequent target data identification and data prefetching on all control flow graphs, thus improving the efficiency of writing the data required by hot functions or code snippets into the cache.

[0136] Furthermore, the control flow graph obtained in this application is a control flow graph that includes nested loops. Therefore, the basic blocks in the control flow graph obtained in this application include basic blocks that are inside nested loops.

[0137] In one possible implementation, the processor 100 acquires the control flow graph by: the processor 100 identifying nested loops (structures) in the source code, thereby determining the basic blocks included in the nested loops, and determining a control flow graph including the basic blocks included in the nested loops from all control flow graphs.

[0138] Regarding the nested loop content in the source code that the processor 100 identifies, please refer to the description under step ① in Figure 9 below, which will not be repeated here.

[0139] S520, processor 100 acquires events of basic blocks in the control flow graph recorded by PMU.

[0140] During the execution of executable code corresponding to a program or code segment, the PMU will continuously record various events related to processor performance, such as instruction execution events, cache access events, branch prediction events, memory access events, pipeline-related events, cache events, instruction cycle counts, and timer events.

[0141] In one possible scenario, since a basic block is a continuous code segment in a program without branches or jumps, the PMU will record the type and number of events triggered by the basic block when the executable code corresponding to the basic block is executed (also known as the number of acquisitions).

[0142] For example, when processor 100 executes the executable code corresponding to basic block a, the types of events triggered and the number of times they are sent are: instruction execution event - 3 times, cache access event - 5 times, memory access event - 2 times, etc.

[0143] It is worth noting that the type of events for which the processor 100 acquires basic blocks in the control flow graph recorded by the PMU can be one or more of the various events described above, and this application does not limit this. For example, in one possible embodiment, the events for basic blocks in the control flow graph recorded by the PMU are cache access events and memory access events. In another possible embodiment, the events for basic blocks in the control flow graph recorded by the PMU are only cache access events.

[0144] In one possible implementation, the processor 100 acquires events of basic blocks in the control flow graph recorded by the PMU, including: the processor 100 acquires events of basic blocks in the control flow graph recorded by the PMU from the memory 300.

[0145] For example, the memory 300 is memory connected to the processor 100.

[0146] For example, since the PMU typically sends recorded events to memory connected to the processor 100 for storage, the processor 100 can obtain events of basic blocks stored in memory via a bus or other interface.

[0147] It is worth noting that, to distinguish events in basic blocks of the control flow graph recorded by the PMU, the processor 100 can determine the boundaries of basic blocks by detecting changes in instruction addresses during program execution. When the program executes to the entry point of a basic block, the address of that instruction can be recorded, and changes in the address can be checked in subsequent event records. If an address change is detected, it indicates that execution has entered another basic block. Therefore, the events recorded before the address change can be used as the type and frequency of events for the corresponding basic block.

[0148] As shown in Figure 5, the processor 100 obtained the following event occurrence counts: BB-6 occurred 20 times, BB-3 occurred 34 times, BB-4 occurred 37 times, BB-7 occurred 23 times, and BB-5 occurred 9 times.

[0149] S530 and processor 100 determine the target paths of multiple paths in the control flow graph based on the number of occurrences of events in the basic blocks of the control flow graph.

[0150] Among them, the sum of the occurrence counts of all events in the basic block of the target path is greater than or equal to the first threshold.

[0151] In one possible implementation, the processor 100 searches multiple paths in the control flow graph, and then determines the sum of the number of events of all basic blocks included in each of the multiple paths. Based on the sum of the number of events of all basic blocks included in each of the multiple paths, a target path whose sum of the number of events is greater than or equal to a first threshold is determined.

[0152] In one possible example, processor 100 begins its search from the initial basic block of the control flow graph.

[0153] For a detailed description of the possible implementation methods mentioned above, please refer to the content shown in Figure 6 below, which will not be repeated here.

[0154] In one possible scenario, each of the aforementioned multiple paths includes all the basic blocks in the control flow graph that are within nested loops.

[0155] Since each path includes all basic blocks within a nested loop, when searching for paths in the control flow graph, the processor 100 will not exit from the last basic block within the nested loop to the exit basic block in the control flow graph. In other words, all basic blocks within the nested loop are fully included in each path. This ensures that each path is a relatively long path (including paths with a large number of basic blocks), avoiding the need to include shorter paths in the sum of event occurrences and improving the efficiency of determining the target path.

[0156] As shown in Figure 5, by searching the control flow graph, processor 100 obtains a path: BB-6→BB-3→BB-4→BB-7→BB-5. During the search, processor 100 avoids the paths BB-6→BB-3→BB-5 and BB-6→BB-3→BB-4→BB-5, thereby improving the efficiency of subsequently determining the target path based on the sum of the occurrence counts of all basic blocks included in the path.

[0157] It is important to note that each of the above paths includes all basic blocks within a nested loop only once, meaning there will be no loops. For example, BB-6→BB-3→BB-4→BB-7→BB-5 will not result in a path like BB-6→BB-3→BB-4→BB-7→BB-3→BB-5.

[0158] In one possible scenario, the sum of the occurrence counts of all events of the basic blocks included in each of the multiple paths is a candidate value, and one of the candidate values ​​among the multiple paths is a first threshold.

[0159] For example, processor 100 searches for two paths (path a and path b) in the control flow graph and determines the sum of the occurrence counts of all events included in path a and path b, respectively, as candidate values ​​a and b. Then processor 100 can use candidate value a or candidate value b as a first threshold.

[0160] If candidate value a is greater than candidate value b, and candidate value b is used as the first threshold, processor 100 can choose either path a or path b as the target path, or select path a with the larger sum of occurrences as the target path.

[0161] If candidate value a is greater than candidate value b, and candidate value a is used as the first threshold, processor 100 can select path a with the larger sum of occurrences as the target path.

[0162] In one possible example, the first threshold is the maximum value among multiple candidate values. The processor 100 determines the sum of occurrences of the target path by using the maximum value among multiple candidate values ​​as the first threshold. This sum is the maximum of the sum of occurrences of all paths. The target data determined using this target path is the most frequently accessed data. Therefore, writing the target data into the cache improves the overall efficiency of data access when the processor 100 executes according to the control flow graph.

[0163] For example, the processor 100 takes the path corresponding to the larger of the candidate values ​​a and b as the target path.

[0164] Regarding the processor 100's determination of the target path content of multiple paths in the control flow graph based on the occurrence count of events in the basic block of the control flow graph, a possible embodiment is provided below, as shown in Figure 6. Figure 6 is a flowchart illustrating a target path determination method provided in this application. S530 in Figure 5 may include the following steps S610 to S630.

[0165] The S610 processor 100 searches the control flow graph for multiple paths starting from the initial basic block.

[0166] In this context, the exit basic block of each of the multiple paths does not belong to the basic blocks in the control flow graph that are nested loops.

[0167] For example, processor 100 uses depth-first search (DFS) to search from the initial basic block in the control flow graph to obtain multiple paths.

[0168] In the above search process, a series of search conditions (also known as tracing conditions) are set. For example, during the DFS process performed by processor 100, if the searched basic block does not return to the first basic block (header BB) of the nested loop, the search does not proceed from the exit edge of the nested loop. When the searched basic block returns to the header BB, processor 100 traverses the exit edge of the nested loop to perform tracing.

[0169] Normally, an exit block is not inside a nested loop, but rather outside of it. This is because an exit block typically includes instructions to break out of the loop.

[0170] For example, a loop structure typically consists of an entry block, one or more loop body blocks, and a loop exit condition block. Correspondingly, there is also an exit block located outside the loop structure, which is either the first block after exiting the loop or the last block of the entire control flow graph.

[0171] It's worth noting that if the above loops are nested loops, then there is an exit block outside each loop structure. In this case, the exit block of each path not belonging to the blocks within nested loops in the control flow graph means that the last block of each path does not belong to the blocks within nested loops. Typically, the last block is used to represent the return of a function or the end of the program.

[0172] For a detailed description of the above search process, please refer to step ③ in Figure 9 below, which will not be repeated here.

[0173] S620, processor 100 determines the sum of the number of occurrences of events for the basic blocks included in each of the multiple paths.

[0174] For example, processor 100 determines the sum of the occurrences of all basic blocks based on all basic blocks included in each of the multiple paths.

[0175] For example, if path a is BB-6→BB-3→BB-4→BB-7→BB-5, processor 100 adds up the occurrence counts of the events corresponding to BB-6, BB-3, BB-4, BB-7, and BB-5 respectively to obtain the sum of the occurrence counts.

[0176] S630, processor 100 determines the target path from the multiple paths whose sum of the number of occurrences of events of the basic blocks included in each path is greater than or equal to a first threshold.

[0177] For example, the processor 100 determines a target path whose sum of the number of occurrences of events is greater than or equal to a first threshold by comparing the sum of the number of occurrences of events of the basic blocks included in each of the multiple paths.

[0178] For example, as shown in Figure 6, the control flow graph searched by processor 100 includes: BB-6, BB-3, BB-4, BB-7, and BB-5. Among them, BB-3, BB-4, and BB-7 are in loop 1, and BB-3 can exit to BB-5, and BB-4 can exit to BB-5.

[0179] Processor 100 searches the control flow graph to obtain paths a: BB-6→BB-3→BB-4→BB-7→BB-5, path b: BB-6→BB-3→BB-5, and path c: BB-6→BB-3→BB-4→BB-5. Then, it determines that the sum of the occurrence counts of events included in path a is 123, path b is 63, and path c is 100. Therefore, path a, with the largest sum of occurrence counts, is selected as the target path.

[0180] Please continue to refer to Figure 5, which shows a method that also includes the following step S540.

[0181] Before executing the first basic block in the control flow graph, the S540 and processor 100 write the target data corresponding to the first basic block into the cache.

[0182] Among them, the target data is the data that needs to be accessed when the first basic block is executed, and the target data is the data whose access frequency or access frequency is greater than or equal to the second threshold among the data that needs to be accessed when executing the target path.

[0183] In one possible scenario, when the processor 100 executes according to the control flow graph, the target data corresponding to the first basic block is written to the cache before the processor 100 has executed all the instructions included in the first basic block.

[0184] For example, the processor 100 can insert a hint instruction into the first basic block, and when the processor executes the hint instruction, write the target data into the cache and persist it, such as for 1 minute or 10 minutes. The specific storage time is not limited in this application. The hint instruction is used to instruct the target data to be written into the cache and persisted.

[0185] In one possible example, processor 100 may insert a hint instruction on the line adjacent to the first instruction containing the access target data included in the first basic block.

[0186] For example, processor 100 may insert a hint instruction on the line before the first instruction.

[0187] Regarding the process by which the processor 100 writes the target data corresponding to the first basic block into the cache before executing the first basic block in the target path, the following is a possible embodiment. As shown in FIG7, FIG7 is a flowchart illustrating a method for writing target data into the cache provided in this application. S540 in FIG5 above may include the following steps S710 to S730.

[0188] The S710 processor 100 parses the source code corresponding to the target path to obtain the target data.

[0189] The reuse score of the target data is the top M scores among the reuse scores of multiple data indicated by the source code, where M is an integer greater than or equal to 1. The reuse score of the target data is used to indicate the degree of reuse of the target data.

[0190] In one possible implementation, the data reuse score can be determined based on one or more of the following: data reuse method (e.g., parallel or serial reuse), memory access pattern (e.g., regular or random), read / write type (e.g., read or write), and footprint constraints.

[0191] For the content of determining the data reuse score based on the data reuse method, memory access pattern, read / write type, and space occupation limit, please refer to the content of Formulas 2 to 11 below, which will not be elaborated here.

[0192] In one possible implementation, the processor 100 parses the source code corresponding to the target path to obtain target data, including: the processor 100 determines the reuse score of multiple data indicated by the source code corresponding to the target path, sorts the reuse scores of the multiple data, and takes the top M data as target data.

[0193] For example, the source code corresponding to the target path shows that data a, data b, and data c need to be accessed. The processor 100 determines that the reuse scores of data a, data b, and data c are 100, 150, and 120, respectively. Then, the processor 100 sorts the reuse scores of data a, data b, and data c, and obtains the sorting result as: data b-150, data c-120, data a-100. Thus, the processor 100 uses the first two sorted data (data b and data c) as the target data.

[0194] For example, in the example above, if M is 1, then the data that is sorted first (data b) will be used as the target data.

[0195] In another possible implementation, the processor 100 parses the source code corresponding to the target path to obtain target data, including: the processor 100 determines the reuse score of multiple data indicated by the source code corresponding to the target path, and determines the M data with the higher reuse scores among the multiple data as target data.

[0196] The process by which the processor 100 obtains the target data in the above implementation is merely an optional example provided in the embodiments of this application and should not be construed as a limitation of this application. In some other possible implementations, the processor 100 selects the data corresponding to the reuse score of the multiple data at the top preset ratio (such as 1% or 3%) as the target data based on the reuse scores of multiple data. The aforementioned preset ratio can be set according to user needs, and this application does not limit it.

[0197] S720, processor 100 determines the first basic block with the highest priority from the multiple basic blocks included in the control flow graph.

[0198] Each of the aforementioned basic blocks includes a first instruction for accessing the target data.

[0199] In one possible implementation, the processor 100 determines the first basic block with the highest priority from a plurality of basic blocks included in the control flow graph, including: the processor 100 determines the basic block including the first instruction for accessing target data from the control flow graph, and then determines the priority among the basic blocks, and determines the first basic block with the highest priority based on the priority among the basic blocks.

[0200] It is worth noting that the first instruction to access the target data mentioned above can be a first instruction to access part of the target data, or a first instruction to access all the target data. For example, if the target data is a 3x4 matrix, the first instruction can be to access the data of the 2x2 matrix within the 3x4 matrix, or it can be to access all the data in the 3x4 matrix.

[0201] In one possible scenario, the priority between basic blocks can be determined based on one or more of the following indicators: the loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0202] The loop depth mentioned above refers to the number of nested loops within a program or piece of code. For example, if a program has one loop nested within another, then the loop depth of this program is 2. A larger loop depth means that there are more levels of nested loops in the program, which usually leads to an increase in program execution time. The loop depth indicated by the first instruction above also represents the loop depth at which the first instruction is located. For example, if the first instruction is in the outer loop of the 2-level loop in the example above, then the loop depth indicated by the first instruction is 1. If the first instruction is in the inner loop of the 2-level loop in the example above, then the loop depth indicated by the first instruction is 2.

[0203] Regarding the determination of priorities among basic blocks, two possible examples are provided below.

[0204] Example 1: Processor 100 may determine the priority of multiple basic blocks based solely on the loop depth indicated by the first instruction. For example, the loop depth indicated by the first instruction in basic block a is 1, and the loop depth indicated by the first instruction in basic block b is 2. Since a higher loop depth indicates a higher priority for the basic block, processor 100 determines basic block b as the first basic block with the highest priority.

[0205] Example 2: The processor 100 can determine the priority of multiple base blocks based on the loop depth and access pattern of the first instruction. For example, the loop depth indicated by the first instruction in base block a is 2, and the access pattern is regular access; the loop depth indicated by the first instruction in base block b is 2, and the access pattern is random access. The processor 100 first compares the loop depths and selects the base block with the larger loop depth of the included first instructions as the highest priority base block. Since the loop depths of the first instructions in base blocks a and b are the same, the processor 100 further compares the access patterns of the first instructions in base blocks a and b. Since the access pattern of the first instructions is regular access, the corresponding base block has a higher priority. Therefore, the processor 100 determines base block a as the highest priority first base block.

[0206] It is worth noting that other embodiments of this application may include more examples to determine the priority of multiple basic blocks by utilizing the loop depth indicated by the first instruction, the access pattern of the first instruction, the occurrence frequency of events in the basic block where the first instruction is located, the line number of the first instruction in the source code, the first instruction being used to indicate reading or writing target data, and the first instruction being a combination of one or more indicators such as vectorized memory access or scalarized memory access. Furthermore, the above examples only use two basic blocks as examples; other embodiments of this application may include more basic blocks in the control flow graph, such as 5 or 10, etc., which is not limited in this application.

[0207] In another possible scenario, the priority among the aforementioned basic blocks can be determined in sequence based on one or more of the following indicators: the loop depth indicated by the first instruction (also known as the loop depth in which the first instruction is located), the access pattern of the first instruction, the number of events occurring in the basic block in which the first instruction is located, the line number of the first instruction in the source code, whether the first instruction is used to indicate reading or writing target data, and whether the first instruction is a vectorized memory access or a scalarized memory access.

[0208] For an example of this situation, please refer to the content shown in Figure 8 below, which will not be repeated here.

[0209] Before executing the first instruction that accesses the target data included in the first basic block, the S730 processor 100 writes the target data into the cache.

[0210] In one possible implementation, before executing the first instruction in the first basic block that accesses the target data, the processor 100 writes the target data into a cache, including: inserting a hint instruction before the first instruction in the first basic block. The processor 100 compiles the basic block after the hint instruction to obtain the corresponding executable code. When the processor 100 executes the executable code corresponding to the hint instruction, it writes the target data into the cache in advance and persists the target data in the cache.

[0211] In one possible scenario, the processor 100 may insert a hint instruction on the line preceding the first instruction.

[0212] In another possible scenario, the processor 100 may insert a hint instruction at any position after the first instruction in the multi-line instruction included in the first basic block and before the first instruction.

[0213] In another possible embodiment, the processor 100 may insert a hint instruction at any position after the first instruction in the multi-line instructions included in the first basic block and before the last instruction in the multi-line instructions included in the first basic block.

[0214] Figure 8 is a flowchart illustrating the priority determination method provided in this application. Figure 8 shows the process in S720 of Figure 7, where the priorities between basic blocks are determined sequentially based on the aforementioned indicators. Figure 8 uses the example of the first instruction accessing the target data residing in two different basic blocks (basic block a and basic block b), where the first instruction in basic block a can be referred to as instruction a, and the first instruction in basic block b can be referred to as instruction b. Figure 8 includes the following steps ① to ⑥.

[0215] Step ①: Processor 100 compares the loop depth of instruction a with the loop depth of instruction b.

[0216] When instructions a and b are at different loop depths, the basic block corresponding to the instruction at the higher loop depth has higher priority. For priority ranking of the same target data, since the target data dimensions are the same, without lightweight instrumentation of the actual upper and lower bounds of the loop, the instruction at the higher loop depth has a higher probability of obtaining a higher footprint.

[0217] For example, if the loop depth of instruction a is greater than the loop depth of instruction b, then basic block a has a higher priority, and processor 100 will use basic block a as the first basic block.

[0218] If the loop depth of instruction a is less than the loop depth of instruction b, then basic block b has a higher priority, and processor 100 will use basic block b as the first basic block.

[0219] If the loop depth of instruction a is equal to the loop depth of instruction b, then processor 100 executes step ②.

[0220] Step 2: Processor 100 compares the access patterns of instruction a and instruction b.

[0221] Access patterns include regular access (also known as regular memory access or direct access / memory access) and random access (also known as random memory access). Regular access has a higher probability of consecutive access / memory access and can be prefetched using vectorized instructions, that is, multiple data are written to the cache using vectorized instructions, resulting in relatively high data access efficiency.

[0222] If the access patterns of instructions a and b are inconsistent, the processor 100 will use the basic block corresponding to the instruction with the regular access pattern in instructions a and b as the first basic block.

[0223] For example, when instruction a has a regular access pattern and instruction b has a random access pattern, processor 100 will use basic block a as the first basic block.

[0224] When the access pattern of instruction b is regular and the access pattern of instruction a is random, processor 100 will use basic block b as the first basic block.

[0225] If the access patterns of instructions a and b are consistent, the processor will execute step ③.

[0226] Step 3: Processor 100 compares the occurrence count 'a' of the event corresponding to basic block a with the occurrence count 'b' of the event corresponding to basic block b.

[0227] If the difference between the occurrence count a and the occurrence count b is greater than the threshold A, then the basic block corresponding to the larger of the occurrence count a and the occurrence count b is taken as the first basic block.

[0228] For example, if the difference between the occurrence count a and the occurrence count b is greater than the threshold A, and the occurrence count a is relatively large, then basic block a is taken as the first basic block.

[0229] If the difference between the occurrence count a and the occurrence count b is greater than the threshold A, and the occurrence count b is larger, then basic block b is selected as the first basic block.

[0230] If the difference between the number of occurrences a and the number of occurrences b is less than the threshold A, then proceed with step ④.

[0231] If the difference between the occurrence count a and the occurrence count b is equal to the threshold A, the processor 100 may, according to the user's needs, use the basic block corresponding to the larger of the occurrence count a and the occurrence count b as the first basic block, or execute the content of step ④.

[0232] In one possible scenario, when the occurrence count a and occurrence count b satisfy the following formula 1, the basic block corresponding to the larger value of occurrence count a and occurrence count b is the first basic block.

[0233] Here, bb_count(a) represents the number of occurrences a, bb_count(b) represents the number of occurrences b, and gap represents the threshold A, which is set by the user.

[0234] Step 4: Processor 100 compares the line numbers of instruction a and instruction b in the source code.

[0235] In one possible scenario, the processor 100 uses the basic block corresponding to the instruction with the smaller line number between instructions a and b as the first basic block.

[0236] For example, if the line number of instruction a is less than the line number of instruction b, then processor 100 will use basic block a as the first basic block.

[0237] If the line number of instruction a is greater than the line number of instruction b, then processor 100 will use basic block b as the first basic block.

[0238] If the line number of instruction a is the same as the line number of instruction b, then processor 100 executes step ⑤.

[0239] In another possible scenario, if the difference between the line number of instruction a and the line number of instruction b is greater than the threshold B, then the processor 100 will use the basic block corresponding to the smaller of the occurrence count a and the occurrence count b as the first basic block.

[0240] If the difference between the line number of instruction a and the line number of instruction b is less than the threshold B, then processor 100 executes step ⑤.

[0241] If the difference between the line number of instruction a and the line number of instruction b is equal to the threshold B, then processor 100 will use the basic block corresponding to the smaller of the occurrence count a and the occurrence count b as the first basic block, or execute the content of step ⑤.

[0242] Step 5: Processor 100 compares the read / write types of instruction a and instruction b.

[0243] Because write operations are expensive, the target data accessed by write operation (write type) instructions a and b is written to the cache.

[0244] If the read / write types of instructions a and b are different, the processor 100 will use the basic block corresponding to the instruction with the read / write type of write in instructions a and b as the first basic block.

[0245] For example, if instruction a and instruction b have different read / write types, and instruction a has a write type, then processor 100 will use basic block a as the first basic block.

[0246] If the read / write types of instructions a and b are the same, then processor 100 executes step ⑥.

[0247] Step 6: Processor 100 compares instruction a and instruction b to determine whether memory access is vectorized or scalarized.

[0248] If the memory access types of instructions a and b are different, the processor 100 will use the basic block corresponding to the vectorized memory access instruction in instructions a and b as the first basic block.

[0249] If instruction a and instruction b have the same memory access type, then processor 100 will use either basic block a or basic block b as the first basic block.

[0250] In one possible embodiment, if the size of the target data cannot be determined, the target data cannot be directly persisted in the cache to avoid allocating too much space in the cache to store the target data, which would cause other data to be frequently replaced in the cache. This would not only affect the lifespan of the cache, but also lead to low overall data access efficiency of the processor core.

[0251] Based on this, before S540 in Figure 5, the above data prefetching method in this embodiment further includes:

[0252] Processor 100 copies the control flow graph obtained in S510 above, and then deploys a computational basic block, including a third instruction for calculating the space occupied by the target data, before the initial basic blocks of the two identical control flow graphs (control flow graph a and control flow graph b). The next basic block after the computational basic block is the initial basic block of control flow graph a and the basic block in control flow graph b. A second instruction for instructing the writing of the target data is deployed in the first basic block of either control flow graph a or control flow graph b.

[0253] In one possible scenario, processor 100 compares the occupied space of the target data determined by the computational basic block with the size of a threshold A. Then, processor 100 determines whether to execute according to control flow graph a or control flow graph b based on the size of the occupied space and the threshold A.

[0254] For example, a second instruction is deployed in the first basic block of control flow graph b. If processor 100 determines that the occupied space of the target data is greater than threshold A, then processor 100 executes according to control flow graph a. If processor 100 determines that the occupied space of the target data is less than threshold A, then processor 100 executes according to control flow graph b.

[0255] If the processor 100 determines that the space occupied by the target data is equal to the threshold A, then the processor 100 executes according to the user's requirements, either according to control flow graph a or control flow graph b.

[0256] Regarding the aforementioned data prefetching method, a possible embodiment is provided below. As shown in Figure 9, Figure 9 is a complete flowchart of a data prefetching method provided in this application. The content shown in Figure 9 may include steps ① to ⑥.

[0257] Step 1: Processor 100 identifies and analyzes nested loops in the source code.

[0258] Regarding the processor 100's ability to identify nested loops in source code, the following are two possible examples.

[0259] Example 1: Nested loops are often represented by indentation, with each nesting level increasing the indentation level. Nested loops typically have different indentation levels, and processor 100 identifies nested loops by determining the indentation of the source code.

[0260] Example 2: Processor 100 can identify nested loops by looking for for, while, or similar loop structures in the source code.

[0261] In one possible implementation, the processor 100 analyzes nested loops in the source code, including: after identifying the memory access node of the innermost loop in the nested loop, the processor 100 determines the data dimension (e.g., array or matrix) and the number of boundary iterations of the loop based on the current memory access tracing information.

[0262] In one possible scenario, processor 100 determines the data dimensions involved in the memory access, including: processor 100 traces the base address and index of the direct memory access, and derives the independent variable (iv) of the current loop from the index.

[0263] For example, during the loop, each iteration accesses an element in the data. The index can be used to determine the current iteration position and then obtain the IV value corresponding to that position.

[0264] For example, a loop iterates through elements in the data one by one according to certain conditions (such as the number of iterations in a for loop or the condition in a while loop). During each iteration, the current iteration position is determined by the index. The index typically starts from 0 and increments by one with each iteration until the data length is reached or the loop termination condition is met. The processor 100 can access the value of the inductive variable at the corresponding position in the data using the index.

[0265] Furthermore, the processor 100 performs boundary iteration count analysis for each loop dimension based on the data dimension.

[0266] For example, for one-dimensional data (such as an array), the number of iterations of a loop is usually equal to the length of the data (the number of elements).

[0267] For two-dimensional data (such as two-dimensional arrays or matrices), the number of iterations of a loop is usually equal to the number of rows in the data structure multiplied by the number of columns.

[0268] For multidimensional data, the number of loop iterations depends on the size of each dimension of the data; that is, the number of loop iterations equals the product of the sizes of each dimension. For example, for a three-dimensional array, the number of loop iterations equals the product of the size of the first dimension, the size of the second dimension, and the size of the third dimension.

[0269] The boundary iteration count analysis includes the analysis of the boundary iteration count of non-vectorized loops and the analysis of the boundary iteration count of vectorized loops.

[0270] For the analysis of the boundary iteration count of non-vectorized loops, please refer to the above example, which will not be repeated here.

[0271] Regarding the analysis of the boundary iteration count of vectorized loops, since processor 100 can process multiple elements, multiple rows or multiple columns, and multiple dimensions of data at one time in the scenario of vectorized loops, the boundary iteration count of vectorized loops will be much smaller than the boundary iteration count of non-vectorized loops. The specific smaller value is determined by the parallel computing capability of the processor.

[0272] Furthermore, processor 100 determines, based on the number of boundary iterations, which loops are the outermost loops of the multidimensional data. These loops are typically those that directly process the multidimensional data. It also checks the memory access nodes used in the boundary expressions to ensure they are the dominating nodes of the outermost loops of the multidimensional data. For each memory access node, it ensures it is accessible during boundary computation. The boundary of this memory access node can represent the dimensionality of the multidimensional data, which can be a specific numerical value or an inductive variable.

[0273] If the calculation of the boundary iteration count involves dynamic multi-branching (e.g., selecting different boundary calculation methods based on conditions), ensure that the memory access node corresponding to the selected boundary calculation method can be correctly obtained.

[0274] In this context, the dominant node refers to the node that controls the number of times the loop is executed in a nested loop.

[0275] In one possible scenario, for a scenario where the number of boundary iterations of the innermost loop is unknown, the processor 100 can perform scene recognition and boundary judgment by expanding the outer loop, thereby obtaining more memory access nodes and the number of boundary iterations.

[0276] In one possible embodiment, the processor 100 can determine the size of the data in the loop based on the boundary iteration number determined above. Then, the processor 100 can determine whether to pre-write the data to a cache (such as HBM) and persist the cache, and determine how much storage space in the HBM to allocate to the data, based on the size.

[0277] Step 2: The processor 100 obtains the control flow graph including nested loops, and the number of occurrences of events of basic blocks in the control flow graph recorded by the PMU.

[0278] For details of this step, please refer to the descriptions of S510 and S520 in Figure 5 above, which will not be repeated here.

[0279] For example, the control flow graph acquired by processor 100 is BB-A, BB-B, BB-C, BB-D, BB-E, BB-F, BB-G. BB-C and BB-D are in loop a, and BB-B, BB-C, BB-D, BB-E, and BB-F are in loop b. The number of occurrences of events BB-A is 10, BB-B is 5, BB-C is 5, BB-D is 6, BB-E is 5, BB-F is 2, and BB-G is 4.

[0280] Step 3: The processor 100 determines the target path of multiple paths in the control flow graph based on the number of occurrences of events in the basic blocks of the control flow graph.

[0281] Since the more times a basic block receives data on the occurrence of events during sampling, the longer the program stays in that basic block (code segment), the greater the bottleneck and potential for improvement. Processor 100 can concatenate the basic blocks within a function that have a high number of event occurrences and the basic blocks on their necessary paths. It can then use a combination of Depth-First Search (DFS) and dynamic programming to find the path (target path) with the highest sum of event occurrences in the basic blocks. Finally, it can filter and combine the kernels along the target path, transforming the problem of combining function paths and kernels into a longest path problem.

[0282] For example, processor 100 uses Depth-First Search (DFS) combined with dynamic programming to solve the longest path problem, with a time complexity of O(|V|+|E|). Meanwhile, since the number of exiting edges of a single basic block is generally no more than 2 (in a few cases, due to switch statements, the number of exiting edges may be between (2, n-1)), by the standard of amortized analysis, the time complexity of this method is essentially O(|V|). Once the longest path problem is solved, all basic blocks reached by the path can be traced.

[0283] The following illustrates the DFS search process:

[0284] Step 1: Processor 100 begins its search from the initial basic block in the control flow graph.

[0285] Step 2: When a basic block is not a header BB appearing in a loop of the traversal list (recorded), is not marked as visited, and does not appear in recorded, add the BB to recorded and traverse its child nodes BB using DFS. After all child nodes BB have been traversed, mark the BB as visited and add it to recorded.

[0286] Step 3: When BB is the header BB that appears in a certain loop in recorded, traverse all exit nodes BB of that loop using DFS.

[0287] Step 4: Stop the search if the basic block BB is otherwise.

[0288] Step 5: Repeat steps 2-4 until you reach the exit block in the control flow graph.

[0289] Since the traversal record of the basic blocks is in post-order, the reverse order of the recorded basic blocks is a topological sort.

[0290] After obtaining the topological sorting (i.e., multiple paths) of the basic blocks during program execution, the processor 100 determines the sum of the occurrence counts of all basic blocks included in each of the multiple paths.

[0291] For the content regarding determining the sum of the occurrence counts of events for all basic blocks in each of multiple paths, please refer to the descriptions shown in S530 of Figure 5 and Figure 6 above, which will not be repeated here.

[0292] As shown in Figure 9, the determined target path is BB-A→BB-B→BB-C→BB-D→BB-E→BB-F→BB-G.

[0293] In one possible implementation, in the case of a directed acyclic graph (DAG), the longest path problem has a fixed linear solution with a time complexity of O(|V|+|E|). Since the CFG contains cycles, processor 100 converts the CFG into a DAG. The transformed DAG maps the following relationships one-to-one with the basic blocks of the original CFG:

[0294] G=CFG graph, V={v_1,v_2,...,v_n}, E={e_1,e_2,...,e_m}, weight(e_j)=bb_count(v_i).

[0295] Where v_i is a basic block in the CFG with index i, e_j is an exit edge in the CFG, and if e_j is an exit edge of v_i, bb_count represents the number of times the event occurs.

[0296] As shown in Figure 10, Figure 10 is a schematic diagram of the conversion of a directed acyclic graph (DAG) provided in this application. In Figure 10, a is the original control flow graph, and b is the control flow graph in the form of a DAG obtained from the control flow graph conversion. The following shows the process by which the processor 100 converts the original control flow graph into a DAG form, including steps A to C.

[0297] Step A: During the DFS process, if the basic block being searched has not returned to the header BB in the loop (BB-3 in Figure 10), do not search outwards from the exit edge (i.e., do not search the paths from BB-3 to BB-5 and from BB-4 to BB-5).

[0298] Step B: When the basic block searched by processor 100 returns to header BB in the loop, the search for the exit edge of the loop begins (BB-3 to BB-5 and BB-4 to BB-5 as shown in Figure 10a).

[0299] Step C: The processor 100 connects the start of the exit edge to the back edge of the loop (i.e., the path from BB-7 to BB-3 shown in Figure 10a), thereby converting the original control flow graph into a control flow graph in the form of a directed acyclic graph.

[0300] Furthermore, the processor 100 can use topological sorting to determine the longest path (target path) in the control flow graph in the form of a directed acyclic graph.

[0301] Step 4: The processor 100 determines the target data based on the data required to be accessed by the target path.

[0302] The target data is X in Figure 9.

[0303] Processor 100 calculates a data reuse score for the data (memory access group) required for the target path. The data reuse score can be determined by factors such as data reuse method, memory access pattern, read / write type, and space usage limitations. For example, processor 100 can calculate the reuse score of a single memory access (an element in the data) within the memory access group according to the following formula 2: Level(mem ref) = parallel(x) * reuse(x) * regular(x) Formula 2

[0304] Where parallel(x) represents the parallel reuse score, reuse(x) represents the serial reuse score, and regular(x) represents the memory access regularity score.

[0305] For parallel(x), processor 100 can obtain the score by detecting OpenMP parallel loops and identifying shared data, as shown in Formula 3 below:

[0306] For reuse(x), processor 100 determines whether the memory accessed is for the same object and calculates the serial reuse score. Specifically, firstly, based on the data's mem refs, it is divided into 8 groups of mem ref types according to three dimensions (read / write, regular memory access, parallelism). Then, based on whether other memory accesses of the same memory access group occur within the basic block or loop containing the mem ref, the basic reuse count base_reuse(x) is calculated, as shown in Formula 4 below:

[0307] The processor 100 calculates the score of the memory access based on the granularity of the memory access belonging to the same circular structure. For example, when memory accesses of the same type are located in the same BB, because the BB granularity is small, the reuse score assigned to subsequent memory accesses of the same type located in the same BB is not high, and the overall score does not exceed 0.1.

[0308] Subsequently, based on the dimensions identified in the data itself and the loop depth, the data reuse weight is calculated, and the serial reuse score of the data is calculated, as shown in Formulas 5 to 7 below: used_dim(x)=min(loop_depth,var_dim) Formula 5 power(x)=log2(max(0,loop_depth-used_dim(x))+2) Formula 6

[0309] Where loop_depth represents the loop depth, var_dim represents the dimension of the data, used_dim(x) represents the dimension of the footprint used by the data in memory access, and power(x) represents the reuse of the data footprint within the loop.

[0310] Without additional information from lightweight instrumentation, on one hand, processor 100 estimates the footprint dimension (used_dim) of the data used in that memory access based on the data's dimension and the loop depth. A larger footprint dimension indicates a larger footprint of the data used in that memory access, resulting in a higher memory access score. On the other hand, loop_depth–used_dim reflects the reuse of the data's footprint within the loop; more reuse leads to a higher memory access score. The growth rates of these two factors in the memory access score are adjusted: O(n^2) for the former and log(n) for the latter. Finally, the serial reuse score for a single memory access is obtained by multiplying the score weights of the two factors by the base score.

[0311] For regular(x), when HBM and DDR latency are comparable, the following patterns exist: Regular access: One cache line in the cache can be used for s memory access instructions; Random access: One cache line is generally used for only one memory access instruction, consuming more bandwidth. Based on this, the memory access regularity score for a single memory access can be determined according to the following formula 8:

[0312] Furthermore, the processor 100 calculates the memory access group reuse score based on the accumulated single memory access reuse score, according to the following formula 9: Level(refgroup)=write_cost(x)*footprint_cost(x)*∑Level(mem ref) Formula 9

[0313] Here, write_cost(x) represents the score for the way data is read and written, and footprint_cost(x) represents the score for the footprint restriction.

[0314] For write_cost(x), processor 100 can determine it using the following formula 10:

[0315] For footprint_cost(x), processor 100 prioritizes data sizes within... The memory access groups are within a certain size range. In the experiment, the prefetching effect of data within a specific size range is better than that of data outside that range. (If the data is smaller than the L2 cache, the data may be accessed within the kernel, and the prefetch instruction cannot leave the kernel. At the same time, compared with the program footprint, the impact is small; if the data is too large, the allocation of prefetch instructions will affect the bandwidth of DDR and HBM. At the same time, a large part of the capacity in HBM will be occupied by the data, resulting in an increase in the number of passive replacements of the remaining data that needs to be stored for a long time). The processor 100 can determine the footprint_cost(x) using the following formula 11:

[0316] Step 5: The processor 100 determines the first basic block with the highest priority from the multiple basic blocks included in the control flow graph.

[0317] Each of these multiple basic blocks includes a first instruction for accessing the target data.

[0318] For details on step ⑤, please refer to the description shown in Figure 8 above, which will not be repeated here.

[0319] As shown in Figure 9, the first basic block is BB-C.

[0320] Step 6: The processor 100 deploys a second instruction in the first basic block to instruct the target data to be written to the cache.

[0321] For example, processor 100 deploys the aforementioned second instruction in a first basic block on a line adjacent to the first instruction. This second instruction is a hint instruction.

[0322] When the processor 100 executes the second instruction, it can write the target data into the cache (such as HBM) in advance and persist the target data in the HBM, thereby improving the efficiency of the core in the processor 100 in accessing the target data.

[0323] In one possible implementation, during static analysis, if the object accessed is an input parameter or input data (i.e., the data accessed is an input parameter or input data), and the boundary iteration count is unknown during source code and compilation, then the size of its boundary iteration count will be determined by runtime data. To accurately determine the memory access size, perform reasonable data analysis and allocation optimization, and avoid allocating too much data in the HBM cache leading to frequent data replacements, a dynamic multi-branch design is added.

[0324] Building upon static analysis, runtime checks on the loop footprint are added. This is achieved by analyzing the outermost loop that the attribution variables in memory access nodes can point to, and using this as the boundary for footprint checks. By adding basic blocks to the outside of the control flow graph to calculate the footprint size and generating corresponding hint insertion branches, runtime decisions for static optimization are implemented.

[0325] As shown in Figure 11, Figure 11 is the control flow graph for dynamic multi-branch insertion provided in this application. BB-82 in Figure 11 is the computational basic block deployed outside the control flow graph, used to calculate the footprint of the data. This computational basic block includes instructions for calculating the footprint of the data (such as the target data mentioned above).

[0326] In one possible example, the computational base block includes instructions that can obtain the number of boundary iterations during the runtime of the memory access node by reading the inductive variable of the loop, and then calculate the size of the footprint based on the number of boundary iterations.

[0327] In one possible scenario, processor 100 generates the aforementioned instruction, which is an intermediate representation instruction.

[0328] In Figure 11, the computational basic blocks are deployed before control flow graphs a and b. Specifically, the next basic block after the computational basic block is the initial basic block of control flow graph a (e.g., BB-83) and the initial basic block of control flow graph b (e.g., BB-84).

[0329] Control flow graph a includes BB-83, BB-13, BB-16, BB-19, BB-37, BB-44, BB-36, BB-43, BB-35, BB-73, and BB-28. Control flow graph b includes BB-84, BB-74, BB-75, BB-76, BB-77, BB-78, BB-79, BB-80, BB-81, BB-73, and BB-28.

[0330] Among them, BB-73 and BB-28 are basic blocks shared by control flow graph a and control flow graph b. BB-19 and BB-37 are in loop 13, BB-16, BB-19, BB-37, BB-44, and BB-36 are in loop 12, BB-13, BB-16, BB-19, BB-37, BB-44, BB-36, BB-43, and BB-35 are in loop 11, BB-76 and BB-77 are in loop 19, BB-75, BB-76, BB-77, BB-78, and BB-79 are in loop 18, and BB-74, BB-75, BB-76, BB-77, BB-78, BB-79, BB-80, and BB-81 are in loop 20.

[0331] In one possible example, control flow graph b is obtained by copying control flow graph a from processor 100, and the first basic block in control flow graph b also deploys a second instruction. The content of the second instruction deployment in the first basic block of control flow graph b can be referred to the description of step ⑥ in Figure 9 above, and will not be repeated here.

[0332] Processor 100 determines the occupied space A of the target data. If the occupied space A is less than the threshold A, it executes according to control flow graph b, pre-writing the target data to the cache and persisting it. If the occupied space A is greater than the threshold A, it executes according to control flow graph a, without pre-writing the target data to the cache. If the occupied space A is equal to the threshold A, it executes according to either control flow graph a or control flow graph b.

[0333] The data prefetching method provided by this application has been described in detail above with reference to Figures 3 to 11. The data prefetching device provided by this application will now be described with reference to Figure 12, which is a schematic diagram of the structure of a data prefetching device provided by this application. The data prefetching device 1200 can be used to implement the functions of the processor 100 in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments.

[0334] As shown in Figure 12, the data prefetching device 1200 includes an acquisition module 1210, a determination module 1220, and a writing module 1230. The data prefetching device 1200 is used to implement the functions of the processor 100 in the method embodiments corresponding to Figures 3 to 11. In one possible example, the specific process by which the data prefetching device 1200 implements the above-described data prefetching method includes the following steps:

[0335] The acquisition module 1210 is used to acquire the control flow graph, as well as the events of the basic blocks in the control flow graph recorded by the PMU.

[0336] The determination module 1220 is used to determine the target path of multiple paths in the control flow graph based on the occurrence count of events of basic blocks in the control flow graph, wherein the sum of the occurrence counts of all events of basic blocks in the target path is greater than or equal to a first threshold.

[0337] The writing module 1230 is used to write the target data corresponding to the first basic block into the cache before executing the first basic block in the control flow graph. The target data is the data that needs to be accessed when executing the first basic block. The target data is the data whose access frequency or access frequency is greater than or equal to the second threshold among the data that needs to be accessed when executing the target path.

[0338] To further realize the functions of the method embodiments shown in Figures 3 to 11 above, this application also provides a data prefetching device, as shown in Figure 13. Figure 13 is a second structural schematic diagram of a data prefetching device 1200 provided by this application. The data prefetching device 1200 further includes a dynamic prediction module 1240.

[0339] The dynamic prediction module 1240 is used to deploy a second instruction in the first basic block to instruct the target data to be written to the cache. Before the initial basic block included in the control flow graph, a computational basic block including a third instruction to calculate the space occupied by the target data is deployed; the next basic block after the computational basic block is the initial basic block of the control flow graph and the initial basic block of the control flow graph including the second instruction. The space occupied by the target data is used to instruct the execution of the initial basic block of the control flow graph, or to execute the initial basic block of the control flow graph including the second instruction.

[0340] As an example of a hardware functional unit, the acquisition module 1210 may include at least one computing device, such as a server. Alternatively, the acquisition module 1210 may also be a device implemented using an ASIC or a PLD. The aforementioned PLD may be implemented using a CPLD, FPGA, GAL, or any combination thereof.

[0341] It should be noted that, in other embodiments, the acquisition module 1210 can be used to execute any step in the data prefetching method, the determination module 1220 can be used to execute any step in the data prefetching method, and the writing module 1230 can be used to execute any step in the data prefetching method. The steps implemented by the acquisition module 1210, the determination module 1220, and the writing module 1230 can be specified as needed. By implementing different steps in the data prefetching method through the acquisition module 1210, the determination module 1220, and the writing module 1230, all functions of the data prefetching device can be realized.

[0342] It is worth noting that the processor 100 in the foregoing embodiment may correspond to the data prefetching device 1200 and may correspond to the corresponding subject that executes the method according to the embodiments of this application in Figures 3 to 11. The operation and / or function of each module in the data prefetching device 1200 are respectively to implement the corresponding flow of each method in the corresponding embodiments in Figures 3 to 11. For the sake of brevity, they will not be described in detail here.

[0343] In addition, the data prefetching device shown in Figures 12 and 13 can also be implemented by a communication device. The communication device here can refer to the computer system (including processor 100) in the foregoing embodiments, or when the communication device is a chip or chip system applied to a computing device, the data prefetching device can also be implemented by a chip or chip system.

[0344] This application also provides a chip system, which includes a control circuit and an interface circuit. The interface circuit is used to acquire a control flow graph, and the control circuit is used to implement the functions of the computing device in the above method according to the control flow graph.

[0345] In one possible design, the chip system also includes a memory for storing program instructions and / or data. This chip system can be composed of chips or may include chips and other discrete components.

[0346] This application also provides a computing device. As shown in Figure 14, which is a schematic diagram of the structure of a computing device 1400 provided in this application, the computing device 1400 includes: a bus 1402, a processor 1404, a memory 1406, and a communication interface 1408. The processor 1404, the memory 1406, and the communication interface 1408 communicate with each other via the bus 1402. The computing device 1400 can be a server or a terminal device, and it can be the computer system shown in Figure 3 above. It is worth noting that this application does not limit the number of processors and memories in the computing device 1400.

[0347] Bus 1402 can be, but is not limited to, PCIe bus, universal serial bus (USB), or inter-integrated circuit (I2C) bus, EISA bus, UB, CXL, CCIX, etc. Bus 1402 can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one line is used in Figure 14, but this does not mean that there is only one bus or one type of bus. Bus 1402 can include a path for transmitting information between various components of computing device 1400 (e.g., memory 1406, processor 1404, communication interface 1408).

[0348] Processor 1404 may include any one or more processors such as CPU, GPU, microprocessor (MP), or digital signal processor (DSP).

[0349] Memory 1406 may include volatile memory, such as random access memory (RAM). Memory 1406 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0350] The memory 1406 stores executable program code, and the processor 1404 executes this executable program code to implement the functions of the aforementioned acquisition module, determination module, and writing module, thereby realizing the aforementioned data prefetching method. That is, the memory 1406 stores instructions for executing the data prefetching method.

[0351] The communication interface 1408 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 1400 and other devices or communication networks.

[0352] This application also provides a computing device cluster. The computing device cluster includes at least one computing device 1400. The memory 1406 of one or more computing devices 1400 in the computing device cluster may store the same instructions for executing a data prefetching method.

[0353] The computing device 1400 can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device 1400 can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0354] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN).

[0355] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a data prefetching method.

[0356] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform a data prefetching method.

[0357] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0358] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data prefetching method, characterized in that, The method includes: Acquire the control flow graph, and the events of the basic blocks in the control flow graph recorded by the performance monitoring unit (PMU); Based on the occurrence count of events in the basic blocks of the control flow graph, a target path is determined for multiple paths in the control flow graph, wherein the sum of the occurrence counts of all events in the basic blocks of the target path is greater than or equal to a first threshold. Before executing the first basic block in the control flow graph, the target data corresponding to the first basic block is written into the cache. The target data is the data that needs to be accessed when executing the first basic block. The target data is the data whose access frequency or access frequency is greater than or equal to a second threshold among the data that needs to be accessed when executing the target path.

2. The method according to claim 1, characterized in that, The sum of the occurrence counts of all events of the basic blocks included in each of the multiple paths is a candidate value, and one of the candidate values ​​of the multiple paths is a first threshold.

3. The method according to claim 2, characterized in that, The first threshold is the maximum value among multiple candidate values.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the target path of multiple paths in the control flow graph based on the occurrence frequency of events of basic blocks in the control flow graph includes: Search the control flow graph for multiple paths starting from the initial basic block; the exit basic block of each of the multiple paths does not belong to the basic block in the control flow graph that is inside a nested loop; Determine the sum of the number of occurrences of events for each basic block included in the multiple paths; Based on the sum of the occurrence counts of events of the basic blocks included in each of the multiple paths, a target path is determined from the multiple paths in which the sum of the occurrence counts of the events is greater than or equal to a first threshold.

5. The method according to any one of claims 1 to 4, characterized in that, The step of writing the target data corresponding to the first basic block into the cache before executing the first basic block in the control flow graph includes: Parse the source code corresponding to the target path to obtain the target data; the reuse score of the target data is the top M reuse scores of multiple data indicated by the source code, where M is an integer greater than or equal to 1, and the reuse score of the target data is used to indicate the degree of reuse of the target data; The first basic block with the highest priority is determined from the plurality of basic blocks included in the control flow graph, each of the plurality of basic blocks including a first instruction for accessing the target data; Before executing the first instruction included in the first basic block, the target data is written to a cache.

6. The method according to claim 5, characterized in that, The priority of basic blocks is determined based on one or more of the following criteria; The loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block where the first instruction is located, the line number of the first instruction in the source code, the first instruction being used to indicate reading or writing the target data, and the first instruction being either vectorized memory access or scalarized memory access.

7. The method according to claim 6, characterized in that, The priority of basic blocks is determined according to one or more of the following indicators in turn; The loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block where the first instruction is located, the line number of the first instruction in the source code, the first instruction being used to indicate reading or writing the target data, and the first instruction being either vectorized memory access or scalarized memory access.

8. The method according to any one of claims 1 to 7, characterized in that, The basic blocks in the control flow graph include basic blocks that are nested loops.

9. The method according to any one of claims 1 to 8, characterized in that, Before executing the first basic block in the control flow graph and before writing the target data corresponding to the first basic block into the cache, the method further includes: A second instruction for instructing the target data to be written into the cache is deployed in the first basic block; Before the initial basic block included in the control flow graph, a computational basic block including a third instruction for calculating the space occupied by the target data is deployed; the next basic block of the computational basic block is the initial basic block of the control flow graph and the initial basic block of the control flow graph including the second instruction; the space occupied by the target data is used to indicate the execution of the initial basic block of the control flow graph, or the execution of the initial basic block of the control flow graph including the second instruction.

10. The method according to any one of claims 1 to 9, characterized in that, Each of the multiple paths includes all basic blocks in the control flow graph that are within nested loops.

11. A data prefetching device, characterized in that, The device includes: An acquisition module is used to acquire the control flow graph, and events of basic blocks in the control flow graph recorded by the performance monitoring unit (PMU). The determination module is used to determine the target path of multiple paths in the control flow graph based on the occurrence count of events of basic blocks in the control flow graph, wherein the sum of the occurrence counts of all events of basic blocks in the target path is greater than or equal to a first threshold. The writing module is used to write the target data corresponding to the first basic block into a cache before executing the first basic block in the control flow graph. The target data is the data that needs to be accessed when executing the first basic block. The target data is the data whose access frequency or access frequency is greater than or equal to a second threshold among the data that needs to be accessed when executing the target path.

12. The apparatus according to claim 11, characterized in that, The sum of the occurrence counts of all events of the basic blocks included in each of the multiple paths is a candidate value, and one of the candidate values ​​of the multiple paths is a first threshold.

13. The apparatus according to claim 12, characterized in that, The first threshold is the maximum value among multiple candidate values.

14. The apparatus according to any one of claims 11 to 13, characterized in that, The determining module is specifically used to search multiple paths in the control flow graph starting from the initial basic block; the exit basic block of each of the multiple paths does not belong to the basic block in the control flow graph that is in a nested loop; determine the sum of the occurrence counts of events of the basic blocks included in each of the multiple paths; and determine the target path from the multiple paths whose sum of the occurrence counts of events is greater than or equal to a first threshold based on the sum of the occurrence counts of events of the basic blocks included in each of the multiple paths.

15. The apparatus according to any one of claims 11 to 14, characterized in that, The writing module is specifically used to parse the source code corresponding to the target path to obtain target data; the reuse score of the target data is the top M reuse scores of multiple data indicated by the source code, where M is an integer greater than or equal to 1, and the reuse score of the target data is used to indicate the degree of reuse of the target data; the first basic block with the highest priority is determined from the multiple basic blocks included in the control flow graph, each of the multiple basic blocks including a first instruction to access the target data; and the target data is written to a cache before executing the first instruction included in the first basic block.

16. The apparatus according to claim 15, characterized in that, The priority of basic blocks is determined based on one or more of the following criteria; The loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block where the first instruction is located, the line number of the first instruction in the source code, the first instruction being used to indicate reading or writing the target data, and the first instruction being either vectorized memory access or scalarized memory access.

17. The apparatus according to claim 16, characterized in that, The priority of basic blocks is determined according to one or more of the following indicators in turn; The loop depth indicated by the first instruction, the access pattern of the first instruction, the number of events occurring in the basic block where the first instruction is located, the line number of the first instruction in the source code, the first instruction being used to indicate reading or writing the target data, and the first instruction being either vectorized memory access or scalarized memory access.

18. The apparatus according to any one of claims 11 to 17, characterized in that, The basic blocks in the control flow graph include basic blocks that are nested loops.

19. The apparatus according to any one of claims 11 to 18, characterized in that, The device further includes: a dynamic prediction module; The dynamic prediction module is configured to deploy a second instruction in the first basic block to instruct the target data to be written to the cache; and to deploy a computational basic block including a third instruction to calculate the space occupied by the target data before the initial basic block included in the control flow graph; the next basic block after the computational basic block is the initial basic block of the control flow graph and the initial basic block of the control flow graph including the second instruction; the space occupied by the target data is used to instruct the execution of the initial basic block of the control flow graph, or to execute the initial basic block of the control flow graph including the second instruction.

20. The apparatus according to any one of claims 11 to 19, characterized in that, Each of the multiple paths includes all basic blocks in the control flow graph that are within nested loops.

21. A chip, characterized in that, include: Processor and power supply circuitry; The power supply circuit is used to supply power to the processor; The processor is used to execute the method according to any one of claims 1 to 10.

22. A computing device, characterized in that, It includes a memory and a processor, the memory being used to store computer instructions; when the processor executes the computer instructions, it implements the method of any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by a processing device, implement the method of any one of claims 1 to 10.

24. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed on the processing device, the method of any one of claims 1 to 10 is implemented.

Images