Cache partitioning method and system for ensuring service quality of multiple delay-critical programs

Abstract

The present invention discloses a cache partitioning method and system for ensuring the service quality of multiple delay-critical programs, including: for multiple delay-critical programs, obtaining performance parameters when each delay-critical program is running alone; calculating the performance index of each delay-critical program according to the obtained performance parameters; classifying each delay-critical program according to the performance index of each delay-critical program; when multiple delay-critical programs are mixed and executed, cache allocation is performed according to the type of the delay-critical program; calculating the relaxation index when multiple delay-critical programs are mixed and executed after cache allocation, and dynamically adjusting the cache space according to the relaxation index. This method improves the utilization rate of the last-level cache LLC resources as much as possible while ensuring the performance of multiple delay-critical programs.

Description

Technical Field

[0001] The present invention relates to the technical field of parallel and distributed computing, and in particular to a cache partitioning method and system for ensuring the service quality of multiple delay-critical programs. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] Data centers have evolved from concepts to maturity. In data centers, in order to improve resource utilization, the number of programs running in collaboration will increase. The benefit of executing multiple programs together is that it can increase server utilization, but it will lead to a decrease in program performance. At present, there are more and more user-oriented programs that have strict requirements on response delay. In many cases, it is necessary to execute multiple latency-critical programs and ensure program performance. Therefore, how to ensure the performance of multiple latency-critical programs when they are executed together has become a challenge. Summary of the invention

[0004] In order to solve the deficiencies of the prior art, the present invention provides a cache partitioning method and system for ensuring the service quality of multiple delay-critical programs; the present invention uses the perf tool to perform performance statistics on performance counters, calculates the performance indicators of delay-critical programs, classifies delay-critical programs according to the performance indicators, and uses the cache partitioning technology (CAT) provided by Intel to preliminarily allocate last-level cache (LLC) resources for different types of delay-critical programs; monitors the performance of the program in real time according to the slack indicator slack, and further adjusts the cache resources to improve the fairness of the program. This method ensures the performance of multiple delay-critical programs while improving the utilization rate of the last-level cache LLC resources as much as possible.

[0005] In a first aspect, the present invention provides a cache partitioning method for ensuring the quality of service of multiple delay-critical programs;

[0006] A cache partitioning approach that guarantees quality of service for multiple latency-critical programs, including:

[0007] For multiple delay-critical programs, obtain performance parameters of each delay-critical program when it runs separately; calculate the performance index of each delay-critical program based on the obtained performance parameters;

[0008] classifying each delay-critical program according to its performance metric;

[0009] When multiple delay-critical programs are mixed and executed, cache allocation is performed based on the type of delay-critical programs;

[0010] The relaxation index of multiple delay-critical programs when they are mixedly executed after cache allocation is calculated, and the cache space is dynamically adjusted according to the relaxation index.

[0011] In a second aspect, the present invention provides a cache partitioning system that ensures the quality of service of multiple delay-critical programs;

[0012] A cache partitioning system that guarantees quality of service for multiple latency-critical applications, including:

[0013] An acquisition module is configured to: for a plurality of delay-critical programs, acquire performance parameters of each delay-critical program when it is run separately; and calculate a performance index of each delay-critical program according to the acquired performance parameters;

[0014] a classification module configured to: classify each delay-critical program according to a performance indicator of each delay-critical program;

[0015] an allocation module, configured to: perform cache allocation according to the type of the delay-critical program when multiple delay-critical programs are mixedly executed;

[0016] The adjustment module is configured to calculate the relaxation index when multiple delay-critical programs are mixedly executed after cache allocation, and dynamically adjust the cache space according to the relaxation index.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] (1) The present invention classifies programs according to performance indicators when the programs are run individually, thus avoiding blind classification.

[0019] (2) The present invention uses the slack indicator to adjust cache resources, thereby improving the fairness of the program and the utilization rate of cache resources while ensuring the quality of program service.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0026] In this embodiment, all data is obtained in compliance with laws and regulations and based on the user's consent, and is used legally.

[0027] The contention for shared resources is the main reason for the degradation of program performance. How to solve the contention problem for shared resources becomes the key to coordinating the coordinated operation of multiple latency-critical programs. Studies have shown that the last-level cache is one of the main competing resources that affect program performance. In recent years, Intel technology that supports last-level cache (LLC) allocation can better manage cache space using cache monitoring technology (CMT) and cache allocation technology (CAT). We can use these two technologies to ensure the performance of latency-critical programs by preventing latency-critical programs from polluting each other's cache space. In addition, by monitoring the performance of programs in real time, timely adjusting the cache space occupied by programs with severe performance violations and programs with less severe performance violations can improve program fairness and thus improve the utilization of cache resources.

[0028] Terminology explanation:

[0029] Latency-critical applications are applications that have strict requirements on tail latency. Tail latency is an important performance indicator for latency-critical applications.

[0030] LLC refers to: Last Level Cache, which is the highest level cache that is usually shared by all functional units on a chip (such as CPU cores, IGPs, and DSPs).

[0031] CAT stands for Cache Allocation Technology, whose basic goal is to enable resource allocation based on application priority or class of service (CLOS). The Intel Xeon processor E5 v4 family (and the communications-focused subset of the Intel Xeon processor E5 v3 family) introduces the ability to configure and utilize cache allocation technology on the last-level cache.

[0032] Embodiment 1

[0033] This embodiment provides a cache partitioning method that ensures the quality of service of multiple delay-critical programs;

[0034] A cache partitioning approach that guarantees quality of service for multiple latency-critical programs, including:

[0035] S101: for a plurality of delay-critical programs, obtaining performance parameters of each delay-critical program when it is run separately; and calculating a performance index of each delay-critical program according to the obtained performance parameters;

[0036] S102: classifying each delay critical program according to a performance indicator of each delay critical program;

[0037] S103: when multiple delay-critical programs are mixedly executed, cache allocation is performed according to the type of the delay-critical program;

[0038] S104: Calculate the relaxation index when multiple delay-critical programs are mixedly executed after the cache is allocated, and dynamically adjust the cache space according to the relaxation index.

[0039] Furthermore, the S101: delay critical program refers to a user-oriented program with strict requirements on response time.

[0040] Furthermore, the S101: obtaining the performance parameters of each delay-critical program when it runs independently; specifically, the performance parameters are obtained by using the perf tool.

[0041] Furthermore, the S101: obtaining performance parameters of each delay-critical program when it runs separately; the specific performance parameters include:

[0042] instructions (number of instructions), mem_load_uops_retired.l3_hit (number of L3 cache hits), mem_load_uops_retired.l3_miss (number of L3 cache misses), or intel_cqm / llc_occupancy / (cache usage size) events.

[0043] Furthermore, the S101: calculating the performance index of each delay-critical program according to the acquired performance parameters; specifically includes:

[0044]

[0045]

[0046] Among them, HPKI_LLC represents the first performance indicator; MPKI_LLC represents the second performance indicator;

[0047] Performance indicators MPKI_LLC (misses per thousand instructions on LLC), HPKI_LLC (hit per thousand instructions on LLC). To ensure that each latency-critical program has no CPU contention, the initial state of the system places each latency-critical program in a different core.

[0048] Furthermore, the S102: classifying each delay critical program according to the performance indicator of each delay critical program specifically includes:

[0049] If HPKI_LLC < 0.3 and MPKI_LLC > 6, the latency-critical program is classified as cache-aggressive;

[0050] If HPKI_LLC ≥ 0.6 and MPKI_LLC ≥ 0.2, the latency-critical program is classified as cache-sensitive;

[0051] If HPKI_LLC < 0.6 and MPKI_LLC < 0.2, the latency-critical program is classified as cache-insensitive;

[0052] For latency-critical programs that do not meet any of the above conditions, they are classified as cache-medium.

[0053] The classification method is shown in Table 1 below:

[0054] Table 1: Program classification

[0055] HPKI_LLC MPKI_LLC Aggressive <0.3 >6 Sensitive ≥0.6 ≥0.2 Insensitive <0.6 <0.2 Medium other other

[0056] Furthermore, the S103: when multiple delay-critical programs are mixedly executed, cache allocation is performed according to the type of the delay-critical program; specifically includes:

[0057] For cache-aggressive latency-critical programs, allocate 10% of the cache space;

[0058] For cache-sensitive latency-critical programs, allocate 40% cache space;

[0059] For cache-insensitive latency-critical programs, allocate 20% of the cache space;

[0060] For latency-critical programs with medium cache size, allocate 30% of the cache space.

[0061] It should be understood that the characteristics of aggressive programs are: HPKI_LLC is very small, MPKI_LLC is very large, which means that the cache reuse is very poor, that is, it occupies more cache space but cannot be fully utilized. According to this characteristic, 10% of the cache space, that is, 2-way cache, can be allocated (for the case where the cache space is 15M and CLOS is equal to 20).

[0062] It should be understood that the characteristics of sensitive programs are: HPKI_LLC is large, MPKI_LLC is large, which means that the cache space utilization is very high. According to this characteristic, 40% of the cache space, that is, 8-way cache, is allocated.

[0063] It should be understood that the characteristics of insensitive programs are: HPKI_LLC is very small, MPKI_LLC is very small, and based on this characteristic, 20% of the cache space, that is, 4-way cache, is allocated.

[0064] It should be understood that the characteristic of medium-sized programs is that the cache space utilization rate is relatively high. Based on this characteristic, 30% of the cache space, namely, 6-way cache, is allocated.

[0065] Furthermore, after the step S103 and before the step S104, the following steps are further included:

[0066] Cache Allocation Technology is used to isolate each delay-critical program by dividing it into different CLOS (Classes of Service).

[0067] It should be understood that the purpose of isolation is to reduce the impact between delay-critical programs.

[0068] Furthermore, the S104: calculating the relaxation index when multiple delay-critical programs are mixedly executed after the cache is allocated; specifically includes:

[0069]

[0070] Among them, T co-run is the running time of multiple programs when they are mixed and executed, T solo It is the running time when the program is executed alone, and slack represents the slack index.

[0071] It should be understood that the slack index during mixed execution of the calculation program is used to dynamically adjust the cache space to ensure fairness of the program.

[0072] Furthermore, the S104: dynamically adjusting the cache space according to the relaxation index specifically includes:

[0073] S1041: If the slack index of a certain delay-critical program is less than 0.1, that is, the performance violation is relatively minor, then reduce the cache space occupied by the current delay-critical program in one way, and allocate the cache space to the delay-critical program with slack>0.3, that is, the performance violation is relatively serious;

[0074] If the slack index of the latency-critical program is less than 0.3 after reducing the cache space, the cache space occupied by the program can be further reduced, and the cache space can be allocated to programs with slack>0.3. If the slack of the program is greater than 0.3 after modifying the cache space, the cache modification operation in the previous step is restored;

[0075] S1042: If the slack index of a certain delay-critical program is greater than 0.3, it means that the program has a serious performance violation due to lack of cache space, and the cache space of the current delay-critical program is increased;

[0076] First, monitor whether there are any unallocated cache resources:

[0077] If yes, then increase the cache space occupied by the current delay-critical program by 1 channel, and re-execute S1042 operation;

[0078] If not, determine whether there is a program with slack < 0.1. If there is such a program, perform S1041. If there is no program with slack < 0.1, perform program migration.

[0079] S1043: If the slack index of all current delay-critical programs is greater than 0.3, the number of delay-critical programs running in collaboration is reduced, and 1 or 2 is selected according to the actual situation.

[0080] It should be understood that if the slack index slack of a delay-critical program is <0.1, it means that the performance of the program when it is executed in a mixed manner is almost equal to the performance when it is executed alone. The cache space (1-way cache) occupied by the program can be reduced and the cache space can be given to the program with slack>0.3. If the program slack>0.3 after reducing the cache space, the cache adjustment operation of the program is restored.

[0081] If the slack index of a latency-critical program is > 0.3, it means that the program has serious performance violations, and the cache space of the program needs to be increased. Monitor whether there are unallocated cache resources; if so, increase the cache space occupied by the program (1-way cache) until the slack index is < 0.1; if there is no space, reduce the cache space occupied by the program with 1-way slack < 0.1 and give the space to the program with slack > 0.3.

[0082] If the slack index of the delay-critical programs is greater than 0.3, reduce the number of programs running in collaboration. The number can be 1 or 2 based on the actual situation.

[0083] Figure 1 The cache resource partitioning method is described. For each latency-critical program that needs to be executed, the perf tool is used to obtain performance events and calculate its performance indicators. The cache resources are dynamically adjusted according to the program type and slack indicators to meet the service quality of the program.

[0084] Embodiment 2

[0085] This embodiment provides a cache partitioning system that ensures the quality of service of multiple delay-critical programs;

[0086] A cache partitioning system that guarantees quality of service for multiple latency-critical applications, including:

[0087] An acquisition module is configured to: for a plurality of delay-critical programs, acquire performance parameters of each delay-critical program when it is run separately; and calculate a performance index of each delay-critical program according to the acquired performance parameters;

[0088] a classification module configured to: classify each delay critical program according to a performance indicator of each delay critical program;

[0089] an allocation module, configured to: perform cache allocation according to the type of the delay-critical program when multiple delay-critical programs are mixedly executed;

[0090] The adjustment module is configured to calculate the relaxation index when multiple delay-critical programs are mixedly executed after cache allocation, and dynamically adjust the cache space according to the relaxation index.

[0091] It should be noted that the acquisition module, classification module, allocation module and adjustment module described above correspond to steps S101 to S104 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules described above as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0092] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0094] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cache partitioning method to ensure the quality of service of multiple latency-critical programs, Its characteristics are: include: For multiple delay-critical programs, obtain performance parameters of each delay-critical program when it runs separately; Calculate the performance index of each delay-critical program based on the obtained performance parameters; classifying each delay-critical program according to its performance metric; When multiple delay-critical programs are mixed and executed, cache allocation is performed based on the type of delay-critical programs; Calculate the slack index when multiple delay-critical programs are mixedly executed after cache allocation, and dynamically adjust the cache space according to the slack index; According to the relaxation index, the cache space is dynamically adjusted; specifically, the following are included: (1): If the slack index of a delay-critical program is less than 0.1, that is, the performance violation is relatively minor, then reduce the cache space occupied by the current delay-critical program and allocate the cache space to the delay-critical program with slack>0.3, that is, the performance violation is relatively serious; If the slack index of the latency-critical program is less than 0.3 after reducing the cache space, continue to reduce the cache space occupied by the program and allocate the cache space to the program with slack>0.3; if the slack of the program is greater than 0.3 after modifying the cache space, restore the cache modification operation in the previous step; (2): If the slack index of a delay-critical program is greater than 0.3, it means that the program has serious performance violations due to lack of cache space, so the cache space of the current delay-critical program is increased; First, monitor whether there are any unallocated cache resources. If so, increase the cache space occupied by the current delay-critical program by 1 and re-execute (2). If not, determine whether there is a program with slack < 0.

1. If there is such a program, execute (1). If there is no program with slack < 0.1, perform program migration. (3) If the slack index of all current delay-critical programs is greater than 0.3, reduce the number of delay-critical programs that run in coordination, and select 1 or 2 based on the actual situation.

2. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 1, Its characteristics are: Obtain the performance parameters of each latency-critical program when it runs separately; specifically, use the perf tool to obtain the parameters.

3. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 1, Its characteristics are: Get the performance parameters of each latency-critical program when it is run separately; Specific performance parameters, including: number of instructions, L3 cache hits, L3 cache misses, or cache usage size.

4. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 1, Its characteristics are: Based on the obtained performance parameters, calculate the performance indicators of each delay-critical program; specifically including: Among them, HPKI_LLC represents the number of hits per thousand instructions on LLC; MPKI_LLC represents the number of misses per thousand instructions on LLC.

5. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 4, Its characteristics are: Each latency critical program is classified according to its performance indicators; specifically, it includes: If HPKI_LLC < 0.3 and MPKI_LLC > 6, the latency-critical program is classified as cache-aggressive; If HPKI_LLC ≥ 0.6 and MPKI_LLC ≥ 0.2, the latency-critical program is classified as cache-sensitive; If HPKI_LLC < 0.6 and MPKI_LLC < 0.2, the latency-critical program is classified as cache-insensitive; Programs that do not meet the above delay-critical requirements are classified as cache-medium.

6. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 5, Its characteristics are: When multiple latency-critical programs are mixed and executed, cache allocation is performed based on the type of latency-critical program; specifically, the following are included: For cache-aggressive latency-critical programs, allocate 10% of the cache space; For cache-sensitive latency-critical programs, allocate 40% cache space; For cache-insensitive latency-critical programs, allocate 20% of the cache space; For latency-critical programs with medium cache size, allocate 30% of the cache space.

7. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 1, Its characteristics are: After the step of performing cache allocation according to the type of the delay-critical program when multiple delay-critical programs are mixedly executed, the step of calculating the relaxation index when multiple delay-critical programs are mixedly executed after the cache allocation, and before the step of dynamically adjusting the cache space according to the relaxation index, further comprises: Cache partitioning technology is used to isolate each delay-critical program by partitioning it into different CLOS.

8. The cache partitioning method for ensuring the quality of service of multiple delay-critical programs as claimed in claim 1, Its characteristics are: Calculate the slackness metrics when multiple latency-critical programs are mixed and executed after cache allocation; specifically: Among them, T co-run is the running time of multiple programs when they are mixed and executed, T solo It is the running time when the program is executed alone, and slack represents the slack index.

9. A cache partitioning system that ensures the quality of service for multiple latency-critical programs. Its characteristics are: include: An acquisition module is configured to: for a plurality of delay-critical programs, acquire performance parameters of each delay-critical program when it is run separately; and calculate a performance index of each delay-critical program according to the acquired performance parameters; a classification module configured to: classify each delay critical program according to a performance indicator of each delay critical program; an allocation module, configured to: perform cache allocation according to the type of the delay-critical program when multiple delay-critical programs are mixedly executed; An adjustment module is configured to: calculate a relaxation index when multiple delay-critical programs are mixedly executed after cache allocation, and dynamically adjust the cache space according to the relaxation index; According to the relaxation index, the cache space is dynamically adjusted; specifically, the following are included: (1): If the slack index of a delay-critical program is less than 0.1, that is, the performance violation is relatively minor, then reduce the cache space occupied by the current delay-critical program and allocate the cache space to the delay-critical program with slack>0.3, that is, the performance violation is relatively serious; If the slack index of the latency-critical program is less than 0.3 after reducing the cache space, continue to reduce the cache space occupied by the program and allocate the cache space to the program with slack>0.3; if the slack of the program is greater than 0.3 after modifying the cache space, restore the cache modification operation in the previous step; (2): If the slack index of a delay-critical program is greater than 0.3, it means that the program has serious performance violations due to lack of cache space, so the cache space of the current delay-critical program is increased; First, monitor whether there are any unallocated cache resources. If so, increase the cache space occupied by the current delay-critical program by 1 and re-execute (2). If not, determine whether there is a program with slack < 0.

1. If there is such a program, execute (1). If there is no program with slack < 0.1, perform program migration. (3) If the slack index of all current delay-critical programs is greater than 0.3, reduce the number of delay-critical programs that run in coordination, and select 1 or 2 based on the actual situation.

Images