A cache hit rate improvement method based on data prefetching

Abstract

The present application relates to the technical field of computer system cache, and particularly relates to a cache hit rate improving method based on data prefetching, which comprises the following steps: proposing a data prefetching method, defining control parameters and analyzing the range, obtaining the optimal solution of the parameters by using a machine learning algorithm, and helping thread synchronization and construction. The helping thread in the present application can prefetch the data required by the main thread into the shared cache in advance, reduces the influence caused by the missing cost due to the data cache missing of the main thread, and improves the hit rate of the data cache.

Description

Technical Field

[0001] This invention relates to the field of computer system caching technology, and in particular to a method for improving cache hit rate based on data prefetching. Background Technology

[0002] In recent years, processor speeds have advanced rapidly. However, memory access speeds, limited by integrated circuit manufacturing processes, have widened the gap with processor speeds, thus restricting processor performance. Data prefetching technology utilizes the idea of ​​simultaneous computation and memory access, fully realizing the parallelism of computation and memory access. By pre-loading the data required for computation into a high-speed cache closer to the processor, it hides the cost of missing data due to cache gaps.

[0003] Prefetching is essentially a predictive technique that anticipates what instructions or data the processor will generate in subsequent processing and pre-loads them into the cache. When the processor accesses the cache, it can directly hit the cache, reducing the cost of missing data due to memory access errors.

[0004] Traditional data prefetching techniques include hardware prefetching, software prefetching, and a combination of hardware and software prefetching. Hardware prefetching suffers from blindness, resulting in a low hit rate and significant hardware overhead. While software prefetching improves the hit rate compared to hardware prefetching, it introduces additional instruction overhead by inserting instructions during compilation, reducing cache efficiency. The combination of hardware and software prefetching is guided by the compiler software and reduces hardware overhead, but it has poor versatility.

[0005] With the rapid development of multi-core processor technology, data prefetching technology using helper threads has become widely adopted. The core idea is to utilize the ample hardware resources of multi-core processors to execute helper threads, thereby partially hiding cache access latency. Helper threads assist the main thread in pre-executing data that the main thread will need, without altering the source code or its final execution result. After obtaining the data required by the main thread, the helper thread loads it into a shared cache space. When the main thread needs to access the data, it can directly hit this cache, reducing the impact of cache misses caused by the main thread accessing the cache and improving cache hit rate and utilization efficiency.

[0006] However, in practical applications, traditional helper thread prefetching methods may encounter problems where the helper thread cannot keep up with the main thread's speed due to differences in computational load and memory access volume between the main thread and the helper thread, resulting in data memory access errors for the main thread. Therefore, it is necessary to optimize parameters to enable coordination between the main thread and the helper thread, thereby improving the efficiency of data prefetching. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method for improving cache hit rate based on data prefetching, thereby further optimizing the performance of data cache in the processor.

[0008] This invention is achieved through the following technical solution:

[0009] A method for improving cache hit rate based on data prefetching includes: defining data prefetching control parameters, using machine learning algorithms to find the optimal solution for the parameters, and assisting in thread synchronization and construction.

[0010] Preferably, the data prefetch control parameters are defined as follows: the data prefetch method adopts a helper thread data prefetch structure that is controllable by control parameters, and there are two threads in the processor. The main thread only works on the computational operations in the application, and the helper thread only works on the memory access operations in the application.

[0011] Preferably, the step of using machine learning algorithms to find the optimal solution for the parameters includes: in order to fully consider the difference in workload between the main thread and the helper thread during actual program execution, as well as the problem of low hit rate of the main thread due to different execution speeds, it is necessary to define the adjustment range of the control parameters, and finally construct a cost function by combining program computation and memory access workload, and use the gradient descent algorithm in machine learning to find the optimal solution for the control parameters.

[0012] Preferably, the helper thread synchronization and construction includes: extracting the memory access operation code from the program source code after compilation by the compiler, marking the memory access instruction part that requires data prefetching, and using the marked code as the code that the helper thread needs to execute.

[0013] Preferably, in order to keep the helper thread's work progressing faster than the main thread so that it can retrieve the required data and put it into the shared cache before the main thread, the workload of the main thread and the helper thread is controlled by parameter control, and synchronization is performed periodically.

[0014] Preferably, the synchronization between the main thread and the helper thread only requires controlling the execution speed of the helper thread through parameter control, to prevent the helper thread from being too fast and prefetching too much data, which would pollute the cache, or too slow, which would reduce the hit rate of the main thread.

[0015] Compared with the prior art, the significant advantages of this invention are:

[0016] First, by using helper threads, the impact of missing data caused by cache errors when the main thread accesses data is inefficient is reduced. Second, by controlling the parameters of the helper threads, the workload of the main thread and the helper threads is balanced, ensuring that the helper threads always run ahead of the main thread, thus overcoming the low data prefetching efficiency of the traditional helper thread model. Finally, the gradient descent algorithm from machine learning is used to efficiently find the optimal parameter solution, which is more efficient than manual enumeration. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a classification diagram of the help thread prefetching situation in this invention.

[0019] Figure 2 This invention provides a flowchart of the parameter control prefetching model. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1:

[0022] Please see Figures 1-2 As shown, embodiments of the present invention provide a method for improving cache hit rate based on data prefetching, including: proposing a data prefetching method and defining its parameters, using a machine learning algorithm to obtain the optimal solution for the parameters, and facilitating the synchronization and construction of helper threads. The data prefetching method employs a parameter-controllable helper thread data prefetching structure. The main thread and helper threads run the same logic code program, where the main thread only performs computational operations, while the helper threads only perform cache access operations. The helper threads are faster than the main thread and place the data needed by the main thread into a shared cache. When the main thread needs data, it can directly read it from the shared cache, improving the main thread's hit rate.

[0023] Furthermore, to improve prefetching efficiency and fully consider the workload of the main thread and the memory access workload of the helper threads, this invention defines the following parameterized adjustments to the helper thread memory access advance, the helper thread prefetch amount, and the synchronization rate between the main thread and the helper threads:

[0024] Define the workload of the main thread executing the memory access task as K;

[0025] Define the prefetch work performed by the helper thread as P;

[0026] Define the synchronization interval between the main thread and the helper thread as B.

[0027] Furthermore, it is necessary to analyze the range of values ​​for the three parameters defined above. In actual program execution, the workload of the main thread may differ significantly from that of the helper threads. This can lead to the helper threads not always finishing before the main thread, requiring multiple synchronization operations between the main thread and the helper threads to catch up. This directly impacts overall execution efficiency. Figure 1 To classify the prefetching scenarios for the helper thread in this invention, the invention analyzes the fetching ranges of the three predefined parameters to obtain the optimal solution and maximize system efficiency. This invention uses the gradient descent algorithm from machine learning to find the optimal solution for the parameters. Compared to the time-consuming and laborious traditional enumeration method, the gradient descent algorithm can obtain the optimal solution for the parameters faster and more accurately.

[0028] Furthermore, after obtaining the optimal parameter solution, this invention constructs a helper thread based on the parameter values. In this invention, the program is pre-compiled by the compiler, and the memory access instructions that need to be pre-fetched are marked. The marked instructions are extracted to generate the code for the helper thread. The working status of the helper thread is determined by whether the helper thread has pre-stored the data needed by the main thread into the shared cache when the main thread requests data.

[0029] During processor execution, the main thread executes the same program as the helper threads. The main thread's task is to perform computational operations, while the helper threads' task is to perform memory access operations. When the helper thread's execution progresses faster than the main thread, it can pre-write the data that the main thread will need next into the shared data cache. Each operation performed by the helper thread represents the generation of one cache line of data, while each operation performed by the main thread represents the reading of data pre-fetched by the helper thread. The execution speeds of the main thread and the helper threads must be coordinated. When they reach the agreed synchronization interval, if the helper thread is slower than the main thread, the main thread needs to be blocked to allow the helper thread to catch up; conversely, if the main thread is slower than the helper thread, the helper thread also needs to be blocked to prevent the helper thread from pre-fetching too much data due to its rapid execution, thus "polluting" the data cache. Therefore, it is necessary to fully consider the computational workload of the main thread and the memory access workload of the helper thread. This invention provides parameterized adjustments to the helper thread's memory access advance, helper thread pre-fetch amount, and the synchronization rate between the main thread and the helper thread.

[0030] The workload of memory access tasks assigned to the main thread is defined as K. At the very beginning of the program execution, the main thread performs memory access operations on the data cache first, and the helper threads then perform their memory cache access tasks. The workload of memory access assigned to the helper threads is defined as P. The helper threads will assist the main thread in prefetching data P times. The interval between synchronization between the main thread and the helper threads is B. After performing P memory access tasks, the helper threads synchronize with the main thread once to determine whether there is a decrease in efficiency due to the main thread or the helper threads progressing too fast or too slow.

[0031] The K value mentioned above represents the memory access workload of the main thread, and the P value represents the memory access workload of the helper thread. The magnitudes of K and P are determined by the memory access and computation workload of different applications. In this invention, the ratio of memory access workload to computation workload in the application is used to determine the maximum value of K and P. The B value mentioned above represents the synchronization interval between the main thread and the helper thread. The smaller the value of B, the more frequently the main thread and the helper thread synchronize. Under normal circumstances, the maximum value of K+P is selected as the maximum value of B.

[0032] This invention analyzes the memory access and computational workload of the source program, as well as processor performance parameters, to obtain the single memory access time. single operation time Let 'a' be the workload of the main thread's memory access task and 'b' be the workload of the helper thread's memory access task. The calculated cost function formula is:

[0033]

[0034] This invention utilizes the gradient descent algorithm to select different values ​​of a and b, and then continuously adjusts them to minimize the values, ultimately obtaining the x and y values. Since the parameters a and b represent the memory access workload of the main thread and the helper thread, their relationship with K and P satisfies... And K+P=B, thus we can obtain the optimal solutions for K and P.

[0035] In this invention, the program is pre-compiled by the compiler, which extracts the code for memory access and marks the memory access instructions that need to be prefetched. If some of the data prefetched by the memory access tasks is used as branch instructions in subsequent program operations, these instructions will not be considered critical instructions. The marked instructions are then extracted to generate the code for the helper thread. This invention starts the helper thread by inserting a trigger instruction in the main thread. The functionality of the helper thread is determined by whether it has pre-stored the data needed by the main thread into the shared cache when the main thread requests data.

[0036] In this invention, the synchronization between the main thread and the helper thread does not require any hardware changes. It only requires controlling the execution speed of the helper thread through parameter control to prevent the helper thread from causing "pollution" of the cache if it is too fast, and from causing the main thread's hit rate to decrease if it is too slow.

[0037] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for improving cache hit rate based on data prefetching, the method comprising: The application relates to a data prefetching control parameter definition, parameter optimal solution obtained by using a machine learning algorithm, and thread synchronization and construction. The data prefetching control parameter definition comprises a data prefetching method using a control parameter to control a thread data prefetching structure, two threads existing in a processor, a main thread only working on operation operation in an application program, and a help thread only working on memory operation in the application program. The parameter optimal solution obtained by using the machine learning algorithm comprises adjusting a range of the control parameter, finally combining program operation and memory workload to construct a cost function, and using a gradient descent algorithm in machine learning to obtain the control parameter optimal solution. The parameterization adjustment of the help thread memory advance, the help thread prefetching amount, and the main thread and the help thread synchronization rate is defined as follows: the workload of the main thread executing the memory task is defined as K; the prefetching workload of the help thread is defined as P; the synchronization interval of the main thread and the help thread is defined as B; the maximum value of K+P is selected as the maximum value of B; and K+P=B, so that the optimal solution of K and P is obtained. After the parameter optimal solution is obtained, the help thread is built according to the parameter value; the thread synchronization and construction comprises the following steps: program source code is extracted after being compiled by a compiler, memory operation code is marked, and the marked code is used as the code to be executed by the help thread; a trigger instruction is inserted in the main thread to start the help thread; whether the help thread works normally is judged by whether the help thread has stored the data required by the main thread in the shared cache in advance when the main thread requests the data. Through analyzing the memory workload and operation workload of the source program and the processor performance parameters, a single memory time T a , a single operation time T b , a main thread memory task workload a, and a helper thread memory task workload b are obtained; and a cost function formula is obtained through calculation. f ( x ) = [x • a(T a + T b ) + y • b(T a - T b )] 2 By using gradient descent algorithm, select different a, b value, and then constantly revised to make f (x) reach the minimum, the final x, y value; because a, b parameters represent the main thread of the memory workload and help thread memory workload, and the relationship between K and P satisfy The workload of the main thread and the help thread is controlled by the parameter control method, and the workload is regularly synchronized. The synchronization of the main thread and the help thread only needs to control the execution speed of the help thread by the parameter control method. In the processor execution process, the program executed by the main thread is consistent with that executed by the help thread, the task of the main thread is operation operation, and the task of the help thread is memory operation; when the help thread executes an operation, the help thread represents that a cache line of data is generated; when the main thread executes an operation, the main thread represents that the data prefetched by the help thread is read; the execution speed of the main thread and the help thread needs to be coordinated, when the two reach the agreed synchronization interval, if the help thread is slower than the main thread, the main thread needs to be blocked to make the help thread catch up with the main thread; if the main thread is slower than the help thread, the help thread also needs to be blocked to prevent the help thread from prefetching too much data due to too fast execution, and to prevent the pollution of the data cache; the help thread memory advance, the help thread prefetching amount, and the main thread and the help thread synchronization rate are parameterized and adjusted by fully considering the operation task amount of the main thread and the memory task amount of the help thread. ​ ​

Images