Interference quantification method and system for mixed load

Abstract

The present invention discloses a method and system for quantifying interference in a mixed load scenario. The mixed load scenario includes both online and offline loads. The interference quantification method includes obtaining first values ​​of a characteristic indicator of a competing resource at multiple time points when both the online and offline loads are operating. The competing resource includes hardware resources required for both the online and offline loads. The method also calculates the degree of discreteness between the multiple first values ​​to obtain an interference entropy value. The interference entropy value is then used to quantify the degree of resource contention in the mixed load scenario. The resource contention degree refers to the degree of mutual interference between the online and offline loads when using the competing resource. This method improves quantification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of performance testing, and in particular to a method and system for quantifying interference for mixed loads. Background Art

[0002] Co-locating latency-sensitive online workloads and low-priority offline workloads on the same data center cluster platform has become an effective method for improving data center resource utilization. While this approach significantly improves data center resource utilization, it also introduces competition between online and offline workloads for shared resources on the same platform. Load performance interference refers to the situation where transient resource competition causes a significant degradation in the QoS (Quality of Service) of online workloads.

[0003] Quantifying load performance interference is a prerequisite for guiding data center system design and optimization. Currently, some technologies quantify load performance interference based on system-level metrics. This approach uses system-level resource metrics related to load performance to measure the degree of performance interference and employs basic statistical methods such as time-series averaging to calculate the metrics. This analysis method is unable to adapt to the diverse characteristics of data center loads, nor can it reflect the transient characteristics of resource contention, making it difficult to accurately reflect the degree of load performance interference. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an interference quantization method, an interference quantization system, and a computer-readable storage medium for a mixed load, which can improve quantization accuracy.

[0005] In one aspect, the present invention provides an interference quantification method for a mixed load, where the mixed load includes an online load and an offline load. The method includes:

[0006] For a characteristic indicator of a competing resource, obtaining first values ​​of the characteristic indicator at multiple time points when the online load and the offline load are running together, the competing resource including hardware resources required for the running of both the online load and the offline load; and

[0007] The degree of discreteness between the multiple first values ​​is calculated to obtain an interference entropy value, and the interference entropy value is used to quantify the resource contention degree of the mixed load, where the resource contention degree refers to the mutual interference degree between the online load and the offline load when using the competing resources.

[0008] Another aspect of the present invention provides a computer-readable storage medium for storing a computer program. When the computer program is executed by a processor, the method described above is implemented.

[0009] Another aspect of the present invention provides an interference quantization system, which includes a processor and a memory. The memory is used to store a computer program. When the computer program is executed by the processor, the method described above is implemented.

[0010] In some embodiments of the present application, when online and offline loads operate together, the first values ​​of the characteristic indicator at multiple time points are obtained, and the degree of dispersion between the multiple first values ​​is calculated to obtain an interference entropy value. The interference entropy value can reflect the instantaneous competition characteristics when the online and offline loads compete for resources. It can more accurately reflect the degree of load performance interference during colocation operation, with high quantification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0012] Figure 1 A schematic diagram of an instruction execution model for a hotspot instruction provided by an embodiment of the present application is shown;

[0013] Figure 2 A schematic diagram of a flow chart of an interference quantification method provided by an embodiment of the present application is shown;

[0014] Figure 3 Shown is a heat map of performance interference affinity based on SLE provided by one embodiment of the present application;

[0015] Figure 4 A schematic diagram of an interference quantization system provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0017] Before explaining the solution of this application, the relevant concepts in this application are first explained.

[0018] In some embodiments, online workloads (also known as online services) typically process user requests and perform computing tasks in the form of services. Examples include web search services, online gaming services, and e-commerce transaction services. Online workloads may have high real-time and stability requirements.

[0019] In some embodiments, offline workloads (also known as offline analysis jobs) are typically compute-intensive batch processing jobs, such as MapReduce and Spark data analysis jobs, and machine learning model training jobs. Offline jobs can tolerate higher latency and support restarting failed tasks.

[0020] In some embodiments, online and offline loads can run in the same system. The system is used to accommodate the hardware resources required by online and offline loads. Offline and online loads deployed in the same system and sharing hardware resources are collectively referred to as mixed loads. That is, mixed loads include both online and offline loads. The hardware resources that may be used during the operation of online and offline loads are called contended resources. A mixed load includes an online load and an offline load. Mixed loads can have different categories. Mixed loads that include the same online load can be classified as mixed loads of the same category. If two mixed loads include different online loads, they can be classified as mixed loads of different categories.

[0021] In some embodiments, load performance interference in a colocation workload can be reflected through application-level and system-level metrics. Load performance interference can refer to a significant degradation in the service quality of the online workload when online and offline workloads compete for resources. From a user perspective, significant degradation in the service quality of the online workload primarily manifests as an inability of the online workload to respond to user requests in a timely manner. For example, when a user searches on a webpage, the webpage may not display the user's search results in a timely manner.

[0022] In some embodiments, application-level metrics are represented by the latency performance of online workloads within a colocation workload. Latency performance can characterize the speed at which an online workload requests access to contended resources and receives a response. Latency performance can be primarily reflected through average latency and tail latency during the online workload's operation, as well as values ​​calculated based on average latency or tail latency. Average latency refers to the average duration from a request to access contended resources to a response received by an online workload during the statistical period. Tail latency refers to the longest duration from a request to access contended resources to a response received by an online workload during the statistical period.

[0023] In some embodiments, system-level indicators are represented by characteristic indicators of competing resources. Characteristic indicators are used to reflect the functional characteristics of competing resources during operation. For example, if memory is a competing resource, memory utilization can be used as a system-level indicator.

[0024] Before executing the method of the present application, you can first select the online load and offline load according to the type of business that may need to be deployed. Based on the selected online load and offline load, you can determine the hardware resources (i.e., competing resources) that may be used when the online load and offline load are running, and extract characteristic indicators of the hardware resources. Finally, based on the extracted characteristic indicators, using the interference quantification method for mixed loads provided by this application, you can perform quantitative detection of load performance interference on the mixed load composed of online loads and offline loads.

[0025] The following first explains the selection of online loads and offline loads.

[0026] In some embodiments, a service type set (BD) and a load latency requirement set (BL) can be first determined. The service type set (BD) includes services that need to be deployed in the same system and share hardware resources. For example, BD = {artificial intelligence, big data, interactive database, HPC}. The load latency requirement set (BL) includes the latency performance requirements of online loads. For example, BL = {1-10ms, 1-20ms, >10s}.

[0027] In some embodiments, online loads and offline loads can be selected in the Benchmark suite according to the range defined by the business type set BD and the load delay requirement set BL. For example, the online load with artificial intelligence business type and a delay performance of 1 to 10ms can be selected. Specifically, the number threshold of online loads can be defined as Bs max , the number threshold of offline loads is Ba max When selecting online and offline loads in the Benchmark suite, the actual number of online loads selected is n on Need to be less than or equal to Bs max , at the same time, the actual number of offline loads n off Need to be less than or equal to Ba max In this way, we can get the load set B = {b i |1≤i≤n on +n off Load set B includes the selected online loads and offline loads. For ease of understanding, Table 1 exemplarily shows the online loads and offline loads included in load set B.

[0028] Table 1 Online and offline loads included in load set B

[0029]

[0030]

[0031] In some embodiments, based on the load set B, n can be determined.on *n off Each co-location load includes an online load and an offline load. Taking Table 1 as an example, we can determine 3*5 co-location loads, totaling 3 types, namely:

[0032] First type of colocation workload (online workload is Imgdnn): {Imgdnn, Union}, {Imgdnn, Multiply}, {Imgdnn, Wordcount}, {Imgdnn, Sort}, {Imgdnn, MD5}

[0033] Second type of hybrid workload (online workload is Masstree): {Masstree, Union}, {Masstree, Multiply}, {Masstree, Wordcount}, {Masstree, Sort}, {Masstree, MD5}

[0034] The third type of mixed workload (online workload is Shore): {Shore, Union}, {Shore, Multiply}, {Shore, Wordcount}, {Shore, Sort}, {Shore, MD5}.

[0035] Based on the above description, the selection of online load and offline load can be completed.

[0036] In some embodiments, after the online load and the offline load are selected, the instruction set BC corresponding to the load set B may be determined. The following describes this process in one embodiment.

[0037] First, we can determine the hotspot functions of each load (including online and offline loads) in the load set B. Take the online load Imgdnn in Table 1 as an example. We can run the online load Imgdnn completely, and then sort the functions in the online load Imgdnn process in descending order according to the hardware resource ratio. Finally, we sort the functions whose hardware resource ratio exceeds the hotspot function resource threshold Rf. max As the hotspot function of online load Imgdnn. For example, assuming the hotspot function resource threshold Rf max= 60% * total CPU resources. The functions called during the online load Imgdnn operation include function A, function B, function C, and function D. Function A occupies 30% of the CPU resources, function B occupies 80% of the CPU resources, function C occupies 65% of the CPU resources, and function D occupies 70% of the CPU resources. Then, in descending order of CPU resource usage, the ranking is: function B, function D, function C, function A. Since the CPU resources occupied by functions B, C, and D exceed the hot function resource threshold Rf, the CPU resources occupied by functions B, C, and D exceed the hot function resource threshold Rf. max , functions B, C, and D can be used as hot functions for the online workload Imgdnn. Using a method similar to that used for the online workload Imgdnn, after determining the hot functions for other loads in workload set B, the hot functions of all loads in load set B can be unioned to obtain the hot function set FP.

[0038] Furthermore, the hot instructions of each function in the hot function set FP are determined. Take one of the functions as an example. The function can be fully run, and then the instructions in the function running process are submitted and sorted according to the hardware resource ratio. Finally, the instructions whose hardware resource ratio exceeds the hot instruction resource threshold Ra are sorted. max The instructions are used as the hot instructions of the function. This process is similar to the hot function selection process described above and will not be described in detail here. After taking the union of the hot instructions of each function in the hot function set FP, the hot instruction set AP can be obtained.

[0039] Furthermore, the instructions in the hot instruction set AP are classified. Specifically, the instructions in the hot instruction set AP can be classified according to the triple instruction feature model. The triple instruction feature model is in the following form:

[0040] (instruction type, source operand type, destination operand type)

[0041] When classifying instructions, instructions with the same instruction type, source operand type, and destination operand type can be considered as instructions of the same category. The following describes the instruction type, source operand type, and destination operand type respectively.

[0042] In some embodiments, instruction types can be divided into data movement instructions, data operation instructions, conditional judgment instructions, jump instructions, and compound instructions consisting of the above-mentioned instructions. Table 2 exemplifies the division of instruction types.

[0043] Table 2 Classification of instruction types

[0044]

[0045]

[0046] In some embodiments, for non-unary operation instructions, the source operand type and the destination operand type are similar, and both can include immediate values, data in registers, and data in memory. For unary operation instructions, the destination operand type can be determined based on the instruction type.

[0047] In some embodiments, after the instructions in the hot instruction set AP are classified according to the triple instruction feature model, the hot instructions can be classified and stored in the instruction set BC.

[0048] Based on the above description, an instruction set BC can be obtained. For ease of understanding, an example of an instruction set BC in an embodiment is given below:

[0049] BC={irMOV,rrMOV,rmMOV,mrMOV,irOP,rrOP,rmOP,mrOP,rCDT,mCDT,JXX,PUSH,POP,CALL,RET,irCMOV,rrCMOV,rmCMOV,mrCMOV}

[0050] In some embodiments, after obtaining the instruction set BC, a contention resource set R can be determined based on the hardware resources (i.e., contention resources) that may be used during the execution of the hotspot instructions. Contention resource set R is used to store the hardware resources that may be used during the execution of the hotspot instructions. The determination of contention resource set R is described below.

[0051] In some embodiments, the life cycle of hotspot instruction execution can be abstracted into an instruction execution model. Each hotspot instruction corresponds to an instruction execution model. Figure 1 , is a schematic diagram of an instruction execution model of a hotspot instruction provided by an embodiment of the present application. Figure 1 It can be seen that the instruction execution model can include the instruction acquisition, decoding, execution, write-back and update stages of the hot instruction. At the same time, the instruction execution model can also include the hardware resources that may be used in each stage of the hot instruction. For example Figure 1 In the example, the hardware resources that may be used during the execution phase of a hot instruction are registers and arithmetic logic units (ALUs). Thus, for each hot instruction's corresponding instruction execution model, the hardware resources that may be used during the execution of each hot instruction can be determined. By taking the union of the hardware resources that may be used during the execution of each hot instruction, we can obtain the set of contended resources R.

[0052] Based on the above description, a contention resource set R can be obtained. For ease of understanding, an example of the contention resource set R in an embodiment is given below:

[0053] R={L1I TLB,L1D TLB,L2 TLB,L1I Cache,L1D Cache,L2 Cache,L3 Cache,Memory,DISK}

[0054] In some embodiments, after obtaining the competitive resource set R, characteristic indicators of the hardware resources in the competitive resource set R can be extracted. The characteristic indicator extraction can be performed based on the functional characteristics of the hardware resources that need to be paid attention to. For example, the characteristic indicator extraction of the hardware resources can be performed based on memory behavior, disk IO status, and computing behavior. The extracted characteristic indicators can be stored in a characteristic indicator set T. For ease of understanding, an example of a characteristic indicator set T is shown in Table 3:

[0055] Table 3 Characteristic index set T

[0056]

[0057] In Table 3, the characteristic indicators numbered 1 to 2 may be characteristic indicators of memory behavior, the characteristic indicators numbered 3 to 4 may be characteristic indicators of disk IO behavior, and the characteristic indicators numbered 5 to 6 may be characteristic indicators of computing behavior.

[0058] Based on the above description, the extraction of feature indicators is completed.

[0059] In some embodiments, by defining a business type set (BD), it is possible to select as many possible colocation loads, contention resources, and characteristic indicators as possible from one or more larger business domains. This ultimately results in a more comprehensive set of characteristic indicators. Furthermore, extracting contention resources based on hot functions and instructions effectively reduces the number of less relevant functions and instructions, thereby reducing analysis costs.

[0060] In some embodiments, based on the extracted characteristic indicators, the interference quantification method for the mixed load provided by this application can be used to quantify the above n on *n off The following conditions must be met when performing quantitative detection of load performance interference on each co-location load:

[0061] 1) Devices (such as computers) running mixed workloads do not have Hyper-Threading or Turbo Boost enabled.

[0062] 2) Collect data using software tools that have negligible performance overhead.

[0063] 3) The input of the colocation load takes into account system resource configuration and does not overload.

[0064] In some embodiments, the interference quantification method for mixed load can be applied to electronic devices, wherein the electronic devices include but are not limited to tablet computers, servers, laptop computers, and desktop computers. Figure 2 , which is a flow chart of an interference quantification method provided by an embodiment of the present application. For each mixed load, the following steps S21 and S22 can be performed respectively.

[0065] Step S21 , for characteristic indicators of competing resources, first values ​​of the characteristic indicators at multiple time points are obtained when online loads and offline loads are running together. The competing resources include hardware resources required for both online loads and offline loads to run.

[0066] In some embodiments, the online load and offline load here are the online load and offline load included in the same mixed load. Competing resources are hardware resources that may be used during the operation of online load and offline load. For example, memory, CPU, cache, register, hard disk, etc. Based on the above relevant description, it can be understood that in addition to the hardware resources required for the operation of both online load and offline load, competing resources may also include the hardware resources required for the operation of one of the online load and offline load, or may also include hardware resources that may not be used when the online load and offline load are running. The characteristic indicators of competing resources are the characteristic indicators in the above-mentioned extracted characteristic indicator set T.

[0067] In some embodiments, the first value of the characteristic indicator is data of the characteristic indicator collected by a data collection software tool when the online load and the offline load are running together. When obtaining the first value of the characteristic indicator, data collection can be performed according to a preset statistical period. Specifically, data collection can be performed every preset time interval within the statistical period. For example, if the statistical period is 1 minute, data collection can be performed every 3 seconds. In this way, multiple first values ​​of each characteristic indicator within the statistical period can be obtained.

[0068] Step S22 : Calculate the degree of dispersion between the multiple first values ​​to obtain an interference entropy value, and use the interference entropy value to quantify the degree of resource contention of the mixed load. The resource contention degree refers to the degree of mutual interference between the online load and the offline load when they use competing resources.

[0069] In some embodiments, the degree of dispersion between multiple first values ​​can be calculated based on entropy calculation. A larger interference entropy value indicates a greater degree of dispersion between the multiple first values, indicating a more unstable value of the characteristic indicator and a greater mutual interference between the online and offline loads when using contention resources. Correspondingly, a smaller interference entropy value indicates a smaller degree of dispersion between the multiple first values, indicating a more stable value of the characteristic indicator and a lesser mutual interference between the online and offline loads when using contention resources.

[0070] In some embodiments, the interference entropy value can be calculated using a performance interference quantification model. Specifically, the first values ​​of the characteristic indicator acquired in step S21 at multiple time points are input into the performance interference quantification model. The performance interference quantification model is then used to calculate the degree of dispersion between the multiple first values ​​to obtain the interference entropy value. The calculation process of the performance interference quantification model is described below.

[0071] In some embodiments, when the contention resource includes multiple characteristic indicators, the degree of dispersion between the multiple first values ​​of each characteristic indicator can be calculated to obtain the first intermediate entropy value corresponding to each characteristic indicator. For example, assuming there are characteristic indicator a, characteristic indicator b, and characteristic indicator c. The first intermediate entropy value corresponding to characteristic indicator a can be obtained based on the first value of characteristic indicator a at multiple time points within the statistical period; the first intermediate entropy value corresponding to characteristic indicator b can be obtained based on the first value of characteristic indicator b at multiple time points within the statistical period; similarly, the first intermediate entropy value corresponding to characteristic indicator c can be obtained.

[0072] In some embodiments, for each feature index, a first intermediate entropy value of each feature index may be calculated based on the following expression (1).

[0073]

[0074] Among them, H(V i ) represents the characteristic index t when the mixed load (i.e., online load and offline load) is running. i The first intermediate entropy value of ;

[0075] V i Represents the characteristic index t i The set of first values ​​at multiple time points in the statistical period (also called characteristic index t i The value range of i is 1≤i≤N t , N t Indicates the number of characteristic indicators;

[0076] v stands for V i One of the first values ​​in ;

[0077] p(v) represents v in Vi The probability of occurrence in

[0078] Num(v) represents v in V i the number in;

[0079] N v Indicates V i The number of first values ​​in .

[0080] In some embodiments, for any characteristic indicator, the first intermediate entropy value of the characteristic indicator is used to represent the degree of discreteness between multiple first values ​​of the characteristic indicator. It can be understood that the characteristic indicator with a larger first intermediate entropy value indicates that the online load and the offline load have a greater impact on the characteristic of the indicator when using competing resources. Judging based on the characteristic of the indicator, it can be determined that the degree of mutual interference between the online load and the offline load when using competing resources is greater. For example, assuming that the first intermediate entropy value of characteristic indicator a is 3 and the first intermediate entropy value of characteristic indicator b is 5, it means that the first value of characteristic indicator b is more discrete and has a larger fluctuation, while the first value of characteristic indicator a has a smaller degree of discreteness and a smaller fluctuation. Judging based on characteristic indicator a, the degree of mutual interference between the online load and the offline load when using competing resources is smaller; judging based on characteristic indicator b, the degree of mutual interference between the online load and the offline load when using competing resources is larger.

[0081] In some embodiments, based on the cumulative entropy characteristic, an interference entropy value can be obtained based on the first intermediate entropy value of the characteristic indicator. Specifically, the first intermediate entropy values ​​of each characteristic indicator can be added together to obtain the interference entropy value. In this way, the first intermediate entropy values ​​of each characteristic indicator are combined to reflect the degree of mutual interference between the online load and the offline load when using competing resources. This process can be expressed by expression (2):

[0082]

[0083] Where f(x, T) represents the interference quantification model established based on the characteristic indicators in the characteristic indicator set T, x represents the current running colocation load, N t Indicates the number of characteristic indicators, H(V i ) represents the characteristic index t i The first intermediate entropy value of .

[0084] In some embodiments, the above-mentioned solution of directly adding the interference entropy value may have the problem of low accuracy. Specifically:

[0085] In some embodiments, the larger the first intermediate entropy value of the characteristic indicator, the greater the degree of mutual interference between the online load and the offline load when using the competing resources; the smaller the first intermediate entropy value of the characteristic indicator, the smaller the degree of mutual interference between the online load and the offline load when using the competing resources. However, in some abnormal cases, when the first intermediate entropy value is relatively small, it does not mean that the degree of mutual interference between the online load and the offline load when using the competing resources is small. For example, when the online load and the offline load use the competing resources, the competing resources are stuck, causing the characteristic indicator of the competing resources to continue to maintain a high first value (for example, the CPU utilization rate continues to be 100%). In this case, although the discrete degree of the first value of the characteristic indicator is small and the obtained first intermediate entropy value is small, it does not mean that the degree of mutual interference between the online load and the offline load when using the competing resources is small. On the contrary, in this case, the degree of mutual interference between the online load and the offline load when using the competing resources is relatively large. In view of this, the method of the present application also includes:

[0086] When the online load is running alone, the second value of the characteristic indicator at multiple time points is obtained, and when the offline load is running alone, the third value of the characteristic indicator at multiple time points is obtained. It can be understood that the second value of the characteristic indicator is the value collected when the offline load does not interfere with the online load, and the discrete degree of the second value can be the minimum discrete degree of the characteristic indicator value when the online load is running. Similarly, the third value of the characteristic indicator is the value collected when the online load does not interfere with the offline load, and the discrete degree of the third value can be the minimum discrete degree of the characteristic indicator value when the offline load is running. Therefore, the entropy value threshold can be determined based on the second and third values ​​of the characteristic indicator, and the target characteristic indicator whose first intermediate entropy value is greater than the entropy value threshold is determined in the characteristic indicator, and the interference entropy value is obtained based on the first intermediate entropy value of the target characteristic indicator. In this way, the abnormal scenario of the above-mentioned competitive resource jamming can be eliminated and the detection accuracy can be improved.

[0087] In some embodiments, when determining the entropy threshold, the degree of dispersion between the multiple second values ​​of each characteristic indicator can be calculated respectively to obtain the second intermediate entropy value corresponding to each characteristic indicator, and the degree of dispersion between the multiple third values ​​of each characteristic indicator can be calculated respectively to obtain the third intermediate entropy value corresponding to each characteristic indicator, and then the minimum value between the second intermediate threshold and the third intermediate threshold is used as the entropy threshold. The calculation formula of the second intermediate entropy value and the third intermediate entropy value can be found in expression (1), which is not repeated here. For the convenience of description, it is assumed that the characteristic indicator t i The second intermediate entropy value is H(V_on i ), the third intermediate entropy value is H(V_off i ), then the first intermediate entropy value of the target feature index can be selected according to expression (3):

[0088] H(V i )≥min(H(V_on i ), H(V_off i )) (3)

[0089] In this way, the interference entropy value can be calculated based on the first intermediate entropy value of the selected target characteristic index to improve the accuracy of the interference entropy value. Expression (3) can be used as a constraint condition for the performance interference quantification model.

[0090] Furthermore, for any characteristic indicator, during the operation of the mixed load (i.e., the online load and the offline load are running at the same time), due to the difference in the characteristics of the online load and the offline load, the online load and the offline load have different effects on the characteristic indicator, which ultimately affects the first intermediate entropy value of the characteristic indicator. That is, there is entropy noise in the first intermediate entropy value. For example, when the online load runs alone, the CPU utilization is 30%, and when the offline load runs alone, the CPU utilization is 20%. Then, when the online load and the offline load run at the same time, the online load has a greater impact on the CPU utilization, and in the first intermediate entropy value of the CPU utilization, the online load has a greater impact. In view of this, in order to eliminate the characteristic differences between the online load and the offline load, the first intermediate entropy value of each target characteristic indicator can be normalized and calculated separately to obtain the optimized entropy value corresponding to each target characteristic indicator. The optimized entropy value is the value obtained after eliminating the entropy noise from the first intermediate entropy value.

[0091] Specifically, for any target characteristic indicator, the first intermediate entropy value of the target characteristic indicator is normalized by taking the sum of the second intermediate entropy value and the third intermediate entropy value of the target characteristic indicator as a reference. The first intermediate entropy value of each target characteristic indicator can be normalized according to expression (4).

[0092]

[0093] Among them, E(V i ) represents the target characteristic index t i The optimized entropy value of

[0094] H(V i ) represents the characteristic index t i The first intermediate entropy value of ;

[0095] H(V_on i ) is the characteristic index t when the online load is running independently. i The second intermediate entropy value of

[0096] H(V_off i ) is the characteristic index t when the offline load runs independently. i The third intermediate entropy value of .

[0097] In this way, the interference entropy value can be calculated based on the optimized entropy value of each target characteristic indicator to improve the accuracy of the interference entropy value.

[0098] Furthermore, considering the cumulative characteristics of entropy, it is necessary for each characteristic indicator to meet the condition of mutual independence. Since the various characteristic indicators of the present application do not meet this condition, weights are introduced for each characteristic indicator. Specifically, when obtaining the interference entropy value based on the optimized entropy values ​​corresponding to the target characteristic indicators, the optimized entropy values ​​of each target characteristic indicator can be multiplied by the weights of the optimized entropy values ​​of each target characteristic indicator and then added together, and the result obtained by the addition is used as the interference entropy value. In this way, the accuracy of the interference entropy value is improved. In some embodiments, for any target characteristic indicator, the weight of the optimized entropy value of the target characteristic indicator can be determined based on the following method:

[0099] When the online load and the offline load are running together, the delay performance of the online load is detected, and then the correlation between the first value of the target characteristic indicator and the delay performance during the operation of the online load is calculated, and the calculated value is used as the weight of the optimized entropy value of the target characteristic indicator, wherein the delay performance characterizes the speed at which the online load requests to use the competing resources and receives the response. Specifically, the first value of the target characteristic indicator reflects the degree of mutual interference between the online load and the offline load when using the competing resources from the system level (i.e., the hardware resource level); the delay performance during the operation of the online load reflects the degree of mutual interference between the online load and the offline load when using the competing resources from the application level (i.e., the delay performance level of the online load). Different target characteristic indicators may have different degrees of correlation with the delay performance during the operation of the online load. In view of this, the correlation between the first value of the target characteristic indicator and the delay performance during the operation of the online load can be calculated based on expression (5):

[0100]

[0101] Among them, w i Represents the target feature index t i The degree of correlation with the delay performance during online load operation, that is, the target characteristic index t i The weight of

[0102] AL is the collection of average delays of online load at multiple time points within the statistical period (also known as the average delay sample);

[0103] V i Represents the characteristic index t i The set of first values ​​at multiple time points in the statistical period (also called characteristic index t i a sample of the first value of );

[0104] μ AL is the average value of the average delay;

[0105] σ AL is the variance of the average delay;

[0106] is the characteristic index t i The variance of the first value of .

[0107] In this way, based on Expression (6), the interference entropy value SLE_o can be calculated.

[0108]

[0109] The interference entropy value (SLE_o) is a hardware resource-level indicator (system-level indicator) that reflects the instantaneous competition between online and offline loads for competing resources. For example, if both online and offline loads have high CPU usage requirements, the dispersion of the first value of CPU utilization will be high, which will be reflected in the interference entropy value (SLE_o), indicating a large SLE_o.

[0110] In some embodiments, the interference entropy value SLE_o can reflect the delay performance (application-level indicator) of the online load. Here, the delay performance of the online load is reflected by the tail-average ratio. The tail-average ratio is the ratio of the tail delay of the online load to the average delay. By reflecting the delay performance of the online load through the tail-average ratio, the difference in delay magnitude caused by the characteristic differences between different online loads can be eliminated. In this way, the application-level indicator of the mixed load can be reflected by the system-level indicator of the mixed load. Specifically, for the same mixed load running in the same system, if the calculated interference entropy value SLE_o is larger, it can be said that the tail-average ratio of the online load is larger; conversely, if the calculated interference entropy value SLE_o is smaller, it can be said that the tail-average ratio of the online load is smaller.

[0111] In some embodiments of the present application, when online and offline loads operate together, the first values ​​of the characteristic indicator at multiple time points are obtained, and the degree of dispersion between the multiple first values ​​is calculated to obtain an interference entropy value. The interference entropy value can reflect the instantaneous competition characteristics when the online and offline loads compete for resources. It can more accurately reflect the degree of load performance interference during colocation operation, with high quantification accuracy.

[0112] Furthermore, in some embodiments, if there are multiple systems, and at least two of the multiple systems have different hardware resources (also referred to as having different system configurations), Table 4 exemplarily lists system configuration information of three systems for ease of understanding.

[0113] Table 4 System configuration information

[0114]

[0115]

[0116] As can be seen from Table 4, systems s1 and s2 have the same system configuration information, and the system configuration information of system s3 is different from the system configuration information of systems s1 and s2. It can be understood that for systems with different system configurations (such as different processing architectures), the extracted feature indicator set T may be different. Specifically, the number of feature indicators in the feature indicator set T may be different. If the same mixed load runs in two systems with different system configurations, the interference entropy value SLE_o calculated based on the first value of the collected feature indicator is not comparable, that is, based on the interference entropy value SLE_o calculated above, load performance interference comparison across loads cannot be achieved. In view of this, in some embodiments, when the same mixed load runs in multiple systems respectively, the method of the present application also includes:

[0117] Determine the interference entropy value of the colocation load when running in each system;

[0118] The interference entropy values ​​of the mixed deployment load when running in each system are normalized and calculated to obtain the second interference entropy values ​​corresponding to the mixed deployment load in each system. This second interference entropy value can be used to compare the resource contention levels when the mixed deployment load runs in each system.

[0119] Specifically, when normalizing the interference entropy values ​​of the mixed load when running in each system, the number of characteristic indicators in each system can be determined for each of the multiple systems. Based on the maximum number of characteristic indicators, the number of characteristic indicators in the multiple systems is normalized to obtain the first normalization coefficient corresponding to each system. The calculation formula of the first normalization coefficient is shown in Expression (7):

[0120]

[0121] Wherein, kn represents the first normalization coefficient;

[0122] NT represents the set of the number of characteristic indicators in each system;

[0123] N t represents the number of characteristic indicators in the t-th system.

[0124] In some embodiments, for any system, the first normalization coefficient corresponding to the system can be multiplied by the interference entropy value of the mixed load when running in the system to perform a normalized calculation on the interference entropy value of the mixed load when running in the system, wherein the calculation formula of the normalized calculation is shown in Expression (8).

[0125] SLE1=kn·SLE_o (8)

[0126] SLE1 is the value obtained by normalizing the interference entropy value SLE_o based on the first normalization coefficient. For the same co-location load, the SLE1 values ​​of the co-location load in different systems can reflect the degree of load performance interference caused by the co-location load in different systems. For example, if the SLE1 value of co-location load A in system 1 is 5 and the SLE1 value in system 2 is 4, it can be determined that the load performance interference caused by co-location load A in system 1 is greater.

[0127] Furthermore, in some embodiments, when multiple co-located loads exist, it is assumed that the multiple co-located loads are running in the same system. Due to the different characteristics of these multiple co-located loads, the same interference entropy value SLE_o does not necessarily mean that the load performance interference levels of these multiple co-located loads are the same. For example, assume that there are two characteristic indicators: CPU utilization and the number of instructions executed per cycle. Suppose that when co-located load A runs, it has a greater impact on CPU utilization, resulting in greater fluctuations in CPU utilization, while when co-located load B runs, it has a greater impact on the number of instructions executed per cycle, resulting in greater fluctuations in the number of instructions executed per cycle. Based on the above-described calculation principle of the interference entropy value SLE_o, it can be seen that the interference entropy values ​​SLE_o calculated for co-located loads A and B may be the same. However, with respect to CPU utilization, a possible interference entropy value SLE_o of 3 indicates a significant level of load performance interference, but with respect to the number of instructions executed per cycle, a possible interference entropy value SLE_o of 5 indicates a relatively low level of load performance interference. That is, the above interference entropy value SLE_o cannot achieve load performance interference comparison across loads. In view of this, for the same system, when there are multiple mixed loads, the method of the present application also includes:

[0128] Determine the interference entropy value for each colocation load.

[0129] Normalizing the interference entropy value of each co-location load to obtain a first interference entropy value corresponding to each co-location load;

[0130] The resource contention levels between the mixed loads are compared based on the first interference entropy values ​​corresponding to the respective mixed loads. The interference entropy values ​​of each mixed load are normalized and calculated, including:

[0131] The delay performance of the online load in each mixed load is detected. The delay performance of the online load can be represented by the tail-average ratio of the online load. Furthermore, to eliminate the magnitude impact caused by the characteristics of different online loads, the tail-average ratios of the online loads of different mixed loads can be normalized to obtain a relative tail-average ratio, and the relative tail-average ratio can be used to reflect the delay performance of the online load in the mixed load. For any mixed load, the tail-average ratio of the online load of the mixed load can be normalized based on Expression (8).

[0132]

[0133] Where l is the tail average ratio of the online load of the colocation load, which indicates the tail average ratio of the online load when the online and offline loads of the colocation load are running simultaneously.

[0134] l on This is the tail average ratio of the online load of the colocation workload when it runs independently.

[0135] l′ is the relative tail-average ratio of the online load of the mixed load.

[0136] Furthermore, for any mixed load, a mapping degree between the interference entropy value SLE_o of the mixed load and the delay performance can be determined, where the mapping degree represents the amount of change in delay performance per unit change in the interference entropy value. For any mixed load, the mapping degree of the mixed load can be determined based on Expression (9).

[0137]

[0138] Here, d1 represents the mapping degree of the colocation load in the same system.

[0139] Furthermore, the interference entropy value of each mixed load can be normalized based on the mapping degree of the mixed load. Specifically, the mapping degree of each mixed load is normalized based on the mapping degree with the largest value to obtain a second normalization coefficient. The second normalization coefficient can be calculated based on expression (10).

[0140]

[0141] kd1 represents the second normalization coefficient when the colocation load is in the same system.

[0142] D1 represents the mapping degree set of all co-located workloads in the same system.

[0143] Furthermore, for any mixed load, the second normalization coefficient of the mixed load can be multiplied by the interference entropy value to perform normalization calculation on the interference entropy value of the mixed load. The interference entropy value can be normalized based on Expression (11).

[0144] SLE2=kd1·SLE_o (11)

[0145] SLE2 is the value obtained by normalizing the interference entropy value SLE_o based on the second normalization coefficient kd1. For different co-located loads running in the same system, the SLE2 value of each co-located load reflects the degree of load performance interference of the co-located load in the system. For example, in system 1, if the SLE2 value of co-located load A is 5 and the SLE2 value of co-located load B is 4, it can be determined that the load performance interference degree of co-located load A in system 1 is greater. In this way, the load performance interference degree can be compared across loads.

[0146] Furthermore, for different mixed loads in different systems, we can first refer to the above formula (7) to obtain the first normalization coefficient kn corresponding to each system. Then, combining the first normalized system kn and the relative tail mean ratio l′ of each mixed load, we can calculate the mapping degree d of each mixed load in different systems using expression (12).

[0147]

[0148] Furthermore, based on expression (13), the mapping degree of each colocation load in different systems can be normalized.

[0149]

[0150] kd represents the second normalization coefficient of the colocation load under different systems.

[0151] D represents the set of mapping degrees of all co-location loads in all systems.

[0152] Furthermore, based on expression (14), the interference entropy value can be normalized.

[0153] SLE=kn·kd·SLE_o (14)

[0154] SLE is the value obtained by normalizing the interference entropy value SLE_o based on the first planning system kn and the second normalization coefficient kd. This allows for load-performance interference comparison between different colocation loads in different systems, that is, cross-system and cross-load load-performance interference comparison.

[0155] For ease of understanding, the technical solution of this application is described below in conjunction with an embodiment.

[0156] In this embodiment, the basic parameters are set as follows: Load set B size threshold B max =8, the number threshold Bs of online loads in load set B max =3, the number threshold of offline loads Ba max =5, hotspot function resource threshold Rf max =60% CPU resources, hot instruction resource threshold Ra max = 80% CPU resources, the threshold value of the number of data collection times when performance interference calculation is performed G max =5. This embodiment is implemented on the X86 platform. For RISC processors, binary machine code and instructions are identical. For CISC processors, instructions are microcodes of binary machine code. Since research has shown that CISC and RISC are now fully integrated, and the differences in their instruction sets are becoming increasingly smaller, the method of the present invention is also applicable to other architectures.

[0157] This embodiment can be divided into the following steps:

[0158] 1. Extract load set B

[0159] 1.1) Initialize the load collection

[0160] 1.2) Define the workload application domain set BD = {artificial intelligence, big data, interactive database, HPC};

[0161] 1.3) Define the load delay requirement set BL = {1-10ms, 1-20ms, >10};

[0162] 1.4) Based on the DCMIX benchmark suite, a typical colocation workload, we obtained load set B = {Imgdnn, Masstree, Shore, Union, Multiply, Wordcount, Sort, MD5}. Load set B is shown in Table 5. The load functions in load set B are shown in Table 6. Load set B can be composed of 3 × 5 colocation workloads.

[0163] Table 5 Load Set B

[0164] Load Name Load Type Application Areas Delay Requirements Imgdnn Online load AI 1~20ms Masstree Online load Big Data 1~10ms Shore Online load Interactive Database 1~10ms Union Offline load Interactive Database >10s Multiply Offline load HPC >10s Wordcount Offline load Big Data >10s Sort Offline load Big Data >10s MD5 Offline load HPC >10s

[0165] Table 6 Load function

[0166]

[0167]

[0168] 2. Extract instruction set BC

[0169] 2.1) Initialization instruction set

[0170] 2.2) For all workloads in workload set B, we use the Linux performance analysis tool Perf to count hot spots. The workload input is configured based on the principle of full load but not overload of system resources. On average, 3.6 hot functions and 9.4 hot instructions meet the corresponding constraints Rf. max 、Ra max , get the instruction set BC={irMOV,rrMOV,rmMOV,mrMOV,irOP,rrOP,rmOP,mrOP,rCDT,

[0171] mCDT, JXX, PUSH, POP, CALL, RET, irCMOV, rrCMOV, rmCMOV, mrCMOV}. MOV stands for Move instruction, OP stands for Operation instruction, CDT stands for Condition instruction, and JXX stands for Jump and Conditional Jump instructions. For example, the instructions movl %eax, (%rsp) and movw %dx, (%rax) both belong to the (Move, Register, Memory) category. This category can be further simplified to irMOV, where i and r represent the first letters of Register and Memory, respectively.

[0172] 3. Extract the competitive resource set R

[0173] 3.1) Based on all instructions in instruction set BC, obtain the set of all potential competing resources: R = {L1ITLB, L1D TLB, L2 TLB, L1I Cache, L1D Cache, L2 Cache, L3 Cache, Memory, DISK}.

[0174] 4. Extract feature index set T

[0175] 4.1) For all resources in the potential competitive resource set R, set MB, DB = {basic behavior indicators, fine-grained behavior indicators, hierarchical behavior indicators reflecting the interaction between resources}, CB = {basic behavior indicators IPC, branch behavior, migration behavior}, and obtain the global feature indicator set T under the typical X86 ISA and the typical Intel Xeon system (Westmere architecture). The specific information is shown in Table 7, where indicators numbered 1-42 are memory behavior indicators, indicators 43-45 are disk IO behavior indicators, and indicators 46-49 are computing behavior indicators. In addition, Table 8 is the result obtained by the general method of selecting feature indicators based on resource behavior under the Skylake processor architecture. The vast majority of feature indicators are implemented by extracting corresponding hardware events using the performance analysis tool Perf. The remaining indicators, such as memory utilization, are calculated by extracting corresponding performance information from the Proc file provided by the Linux system;

[0176] Table 7 Some characteristic indicators of Westmere processor architecture

[0177]

[0178] Table 8 Some characteristic indicators of Skylake processor architecture

[0179]

[0180] 5. Performance Interference Quantification Model Construction

[0181] 5.1) Construct a performance interference quantification model, define the unnormalized output of the model as the native system-level entropy index SLE_o, and define the normalized output as the system-level entropy index SLE (or SLE1, SLE2). After constructing the SLE_o process, starting from the basic entropy quantification model, the accuracy is progressively evaluated to prove the effect of model optimization. The Pearson correlation coefficient of the model output and the tail mean ratio of all categories of mixed loads during the optimization process is shown in Table 9. It can be seen from Table 9 that compared with the basic quantization model, the accuracy of the model has been significantly improved after the progressive optimization process in three aspects, and finally reached the optimal value. The above results prove that the optimization method of the present invention based on the basic performance interference model based on entropy measurement is effective;

[0182] Table 9 Correlation between output and tail mean ratio during model optimization

[0183]

[0184] 6. Performance Interference Calculation

[0185] 6.1) Load set B = {Imgdnn,Masstree,Shore,Union,Multiply,

[0186] Wordcount, Sort, MD5}, the system set is S = {s1, s2, s3}, the specific information of the systems in S is shown in Table 10, calculate the performance interference degree of all co-location loads in load set B under all systems in S. Define the calculation result as set F, initialize

[0187] Table 10 System configuration information

[0188]

[0189] 6.2) For the mixed load set X consisting of load set B, i All x in |1≤i≤15} i , set up cross-colocation load category and cross-system analysis or application for the test space, configure the benchmark input according to Table 11, use Perf to locate events according to Table 12, set the collection interval to 10 seconds to obtain time series sample values ​​of characteristic indicators, and obtain the SLE calculation result F, as shown in Table 13. In addition to the test space consisting of load set B and system set, it also includes the baselines of each type of colocation load on different systems. This is equivalent to running an idle load in the colocation, which is the ideal state with zero performance interference in the current colocation system.

[0190] Table 11 Benchmark configuration

[0191]

[0192]

[0193] Table 12 Event codes and masks of characteristic indicators

[0194] Indicator number Westmere Skylake 1 0185 0185 2 1085 1085 …… …… ……

[0195] Table 13 Calculation results of system level entropy SLE

[0196]

[0197]

[0198] Let's use a specific example to illustrate the calculation method for one item in Table 13: Assume that the currently calculated colocation load is Imgdnn-Wordcount and the current system is s1. First, calculate the entropy value of each characteristic indicator in T for the colocation operation of Imgdnn-Wordcount, the independent operation of Imgdnn, and the independent operation of Wordcount. Formula (3) is used to filter out indicators that do not meet the constraints, and a total of six indicators are eliminated. For example, the entropy of the indicator snoop_hite_PKI in the colocation mode (3.75) is smaller than the entropy of the online service independent mode (5.17) and the entropy of the offline analysis job independent mode (4.38). Next, formula (4) is used to remove the noise in the entropy value. For example, the entropy value of the indicator l2_wb_l3_PKI (4.62) is updated to E (0.60) by the entropy of the online service independent mode (5.25) and the entropy of the offline analysis job independent mode (2.45). Then, the weight w of each feature index is calculated using formula (5), with l3_miss_PKI having the highest weight w, 0.89. The native system-level entropy index SLE_o of Imgdnn-Wordcount is calculated using formula (6) to be 9.91. Further normalization is performed. Since s1 has 49 feature indexes for the Westmere architecture, and system set S also includes system s3 with a Skylake processor architecture, which has 42 feature indexes, the normalization coefficient kn is calculated as 1 using formula (7). According to formula (8), the relative tail mean ratio TL_n of Imgdnn-Wordcount is calculated as 3.37 by combining the tail mean ratio of Imgdnn-Wordcount (5.53) and the tail mean ratio of Imgdnn when running independently (1.64). Therefore, according to formula (12), the mapping degree d of Imgdnn-Wordcount is 2.94. The maximum mapping degree of the additional calculated spatial load set B and system set S is 10.48, which is achieved by Imgdnn-Sort running on s3. Therefore, according to formula (13), the normalization coefficient kd of Imgdnn-Wordcount is calculated as 3.56. Finally, according to formula (14), the value of the system-level entropy index SLE of Imgdnn-Wordcount is 35.32;

[0199] 7. End: Terminates the quantification of the impact on colocation load performance.

[0200] Based on the quantification of colocation workload performance interference according to this invention, the effectiveness of SLE and the performance of the quantification model are evaluated based on the SLE calculation results in Table 13. Furthermore, two SLE application cases are presented: one for SLE-based colocation workload affinity analysis and the other for using SLE to evaluate different isolation mechanisms.

[0201] (1) Equivalence comparison

[0202] By using the system-level entropy indicator SLE, we can make equivalent one-to-one comparisons of performance interference under the same metric.

[0203] Comparing s1 and s2: The average SLE values ​​for s1 and s2 were 23.62 and 19.08, respectively, indicating that s2 had less overall performance impact than s1. Furthermore, 10 of the 15 co-located workloads on s2 had lower SLE values ​​than system s1, including Imgdnn-Wordcount, Imgdnn-Sort, Masstree-Wordcount, Masstree-Sort, Shore-Wordcount, and Shore-Sort. The biggest difference between s1 and s2 lies in their memory resource sizes. s2 has three times the memory capacity of s1, so memory-sensitive workloads or those with high memory resource requirements are affected. Taking Imgdnn-Wordcount as an example, its SLE value on s2 decreased by 69.5% compared to s1. Analysis of the weights of Imgdnn-Wordcount's characteristic indicators on the two systems reveals significant differences that can be divided into two major categories. One category is TLB behavior, such as itlb_miss_PKI and dtlb_miss_PKI, which are 3.6 and 3.3 times higher in s1 than in s2, respectively. The other category is shared cache (L3 Cache) behavior, such as l3_miss_PKI, offcore_rddata_PKI, and snoop_code_PKI, which are 3.2, 2.8, and 6.1 times higher in s1 than in s2, respectively. The former category indicates that the TLB is less stressed in managing page table coherence on s2, while the latter category indicates that the overhead of maintaining cache coherence is lower on s2. In contrast, the SLE values ​​for the other five co-located workloads on s2 were greater than those on s1. These workloads primarily involved running offline analysis jobs, such as Union or MD5. These workloads are Imgdnn-Union, Imgdnn-MD5, Masstree-Union, Shore-Union, and Shore-MD5. Therefore, it can be inferred that a larger memory resource capacity may incur the overhead of longer disk I / O operations. Furthermore, the co-located workload with the highest SLE value across all scenarios was Shore-Union, which consisted of two disk I / O-sensitive workloads and was achieved while running on s2. This suggests that a larger memory resource capacity leads to increased contention for disk I / O resources.

[0204] Comparing s1 and s3: The average SLE values ​​for s1 and s3 are 23.62 and 14.62, respectively, indicating that s3 exhibits less overall performance interference than s1. For the vast majority of s3 workloads, 13 out of 15 have lower SLE values ​​than s1, such as Imgdnn-Union, Imgdnn-Multiply, Masstree-Multiply, Masstree-Sort, Shore-Union, and Shore-Wordcount. Although s3 has fewer CPU cores and smaller memory capacity, it also experiences less CPU contention (such as context switching and thread transfer) and less contention for storage resources (such as L3 cache and memory).

[0205] (2) System performance upper limit

[0206] Using the system-level entropy index SLE, a general method for evaluating the performance upper limit of the system under different types of mixed deployment loads is proposed. For the same type of mixed deployment load, the smaller the difference between the SLE of the mixed deployment load and the baseline, the closer the mixed deployment solution is to the system performance upper limit and the higher its mixed deployment efficiency; the larger the difference, the further the mixed deployment solution deviates from the system performance upper limit and the lower its mixed deployment efficiency. For any mixed deployment load, its system mixed deployment efficiency is defined as the baseline divided by its system-level entropy index SLE and multiplied by 100%, as shown in Expression (15);

[0207]

[0208] Table 14 shows the colocation efficiency under the tested space. The baseline colocation efficiency for all colocation operations running the idle load is 100%. Defining colocation efficiency using a baseline provides an intuitive understanding of the room for system optimization. For example, the Masstree-Wordcount colocation efficiency on three different systems was 72%, 99%, and 73%, respectively. Therefore, it is most suitable for deployment on s2. The performance impact on s2 is very close to the system's upper limit, indicating an ideal state with no performance impact.

[0209] In s1, the average colocation efficiency of Imgdnn-related workloads was 63%, the average colocation efficiency of Masstree-related workloads was 75%, and the average colocation efficiency of Shore-related workloads was 39%, with an overall average colocation efficiency of 59%. In s2, the average colocation efficiency of Imgdnn-related workloads was 85%, the average colocation efficiency of Masstree-related workloads was 87%, and the average colocation efficiency of Shore-related workloads was 41%, with an overall average colocation efficiency of 71%. In s3, the average colocation efficiency of Imgdnn-related workloads was 87%, the average colocation efficiency of Masstree-related workloads was 77%, and the average colocation efficiency of Shore-related workloads was 61%, with an overall average colocation efficiency of 75%. Therefore, Imgdnn-related workloads are most suited for deployment on s3, Masstree-related workloads are most suited for deployment on s2, and Shore-related workloads are most suited for deployment on s3. Overall, the 3×5 colocation workloads are most suited for deployment on s3.

[0210] Table 14 Colocation efficiency calculation results

[0211] Colocation load <![CDATA[s1]]> <![CDATA[s2]]> <![CDATA[s3]]> Imgdnn-Union 92％ 86％ 93％ Imgdnn-Multiply 81％ 82％ 96％ Imgdnn-Wordcount 30％ 97％ 80％ Imgdnn-Sort 27％ 82％ 70％ Imgdnn-MD5 87％ 80％ 93％ Imgdnn-Idle 100％ 100％ 100％ Masstree-Union 93％ 78％ 75％ Masstree-Multiply 72％ 87％ 94％ Masstree-Wordcount 72％ 99％ 73％ Masstree-Sort 60％ 82％ 71％ Masstree-MD5 76％ 88％ 72％ Masstree-Idle 100％ 100％ 100％ Shore-Union 22％ 19％ 48％ Shore-Multiply 81％ 84％ 78％ Shore-Wordcount 22％ 26％ 47％ Shore-Sort 28％ 41％ 59％ Shore-MD5 40％ 35％ 76％ Shore-Idle 100％ 100％ 100％

[0212] (3) Accuracy evaluation

[0213] The accuracy of the performance interference quantification model under different conditions is evaluated by calculating the evaluation index R. R, as shown in formula (16), represents the Pearson correlation coefficient between the output of the performance interference model, namely the native system-level entropy SLE_o, and the application-level indicator tail mean ratio l. If R ≥ 0.6, that is, SLE_o is positively and strongly correlated with the tail mean ratio l, the accuracy of the performance interference quantification model is acceptable, which can ensure the validity of the measurement index.

[0214]

[0215] Table 15 shows the specific values ​​of the evaluation metric R in different scenarios. It can be seen that R in all cases is positive and greater than 0.6, indicating a strong positive correlation. The maximum value is the accuracy of the Imgdnn-related load on s3 (0.95), and the minimum value is the accuracy of the Imgdnn-related load on s2 and the Shore-related load on s3 (0.90). This shows that the performance interference quantification model can objectively and accurately quantify performance interference under various co-location load types and systems. Although the processor architecture of s3 differs from that of s1 and s2, the model input source, namely the feature metric extraction step, is a general and universal method applicable to different processor architectures. Therefore, the accuracy of the performance quantification model is not affected by system differences.

[0216] Table 15 Model accuracy

[0217] Colocation load category <![CDATA[s1]]> <![CDATA[s2]]> <![CDATA[s3]]> Imgdnn related 0.93 0.90 0.95 Masstree related 0.94 0.94 0.93 Shore related 0.93 0.91 0.90

[0218] 4) Scalability

[0219] 4.1) QPS settings for different online services

[0220] For each type of colocation workload, we varied the QPS of the online service to evaluate the accuracy of the performance interference quantification model. The default QPS setting adheres to the principle of full load without overloading, allocating the full quota of CPU cores without exceeding the QoS threshold. Therefore, we set 50% and 25% of the default QPS as the medium and low QPS levels, respectively. The specific values ​​are shown in Table 16.

[0221] Table 16 QPS configuration

[0222] Colocation load category Default QPS Medium-level QPS Low-level QPS Imgdnn related 1100 550 275 Masstree related 550 275 138 Shore related 5 3 1

[0223] Table 17 shows the values ​​of the evaluation metric R at different QPS levels. The overall QPS R is calculated for the default, medium, and low QPS levels. As can be seen, R is positive in all cases and greater than 0.6, indicating a strong positive correlation. The maximum R value is the accuracy of the Shore-related colocation workload on s2 at a low QPS level (0.99), while the minimum value is the accuracy of the Masstree-related colocation workload on s2 at the overall QPS level (0.65). This demonstrates that the accuracy of the performance interference quantification model is unaffected by the QPS of online services in the colocation workload. Because the performance interference quantification model uses the disorder of resource contention at the system level to characterize the degree of performance interference, changes in QPS also reflect this disorder, and the model captures this change, allowing it to accurately quantify performance interference at different QPS levels.

[0224] Table 17 Model accuracy at different QPS

[0225]

[0226]

[0227] 4.2) Ad-hoc offline analysis

[0228] To evaluate the accuracy of the performance interference quantization model in ad-hoc scenarios, a new offline analysis job, FFT, was introduced as an ad-hoc workload. FFT, also from the DCMIX benchmark suite, is used in HPC applications. It computes the Fast Fourier Transform of an input matrix file and writes the result to an output file. This example uses the 400MB file mouse_gene.mtx from MartixMarket as the input file for s1 and s2; and the 10MB file wi2010.mtx as the input file for s3.

[0229] Table 18 shows the specific values ​​of the evaluation index R under the introduction of ad-hoc offline analysis jobs. It can be seen that R in all cases is positive and greater than 0.6, which is a strong positive correlation. The maximum value is the accuracy of the Imgdnn-related mixed load on s3 (0.97), and the minimum value is the accuracy of the Shore-related mixed load on s1 (0.80), which shows that the accuracy of the performance interference quantification model is not affected by the ad-hoc offline analysis job. This is because although there are differences in computing logic and code implementation in the workload at the application level, they are unified at the system level. The present invention uses the ISA instruction-driven method to construct a unified set of competitive resources, and extracts representative and complete resource behavior indicators, thereby selecting all potential performance interference factors. Therefore, the model can accurately quantify performance interference under the introduction of ad-hoc offline analysis jobs.

[0230] Table 18 Model accuracy with ad hoc offline analysis

[0231] Colocation load category System A System B System C Imgdnn related 0.94 0.96 0.97 Masstree related 0.85 0.81 0.82 Shore related 0.80 0.90 0.88

[0232] (5) Application of SLE

[0233] 5.1) Analysis of Performance Interference Affinity in Colocation-Based Load Balancing Based on SLE

[0234] Based on Table 13, the average SLE of each colocation workload across all systems is calculated as the affinity of the colocation solution for performance interference. Figure 3A heatmap of performance interference affinity based on SLE is presented. The higher the SLE of a colocation solution, the darker the color, indicating poor performance interference affinity. The figure shows: a) Shore-Union has the worst performance interference affinity, the darkest among all solutions. Since both the online Shore service and the offline analysis job Union are disk I / O-intensive workloads, colocation between resource-sensitive workloads of the same type should be avoided. b) From the perspective of online services, Shore is the most severely affected by performance interference, compared to Imgdnn and Masstree, with four of the five colocation solutions having a darker color. Due to the hierarchical nature of storage resources, once intense disk I / O resource contention occurs, the affected workloads will incur significant overhead to compensate. Therefore, disk I / O-intensive online services are susceptible to severe performance interference. c) From the perspective of offline analysis jobs, online services colocated with Wordcount and Sort have the worst affinity. Based on workload logic analysis, Sort is a hybrid resource-sensitive workload with distinct behaviors across compute, storage, and disk I / O resources. Similarly, Wordcount first splits the input file into pieces and reads it into memory, then calculates the word frequency on a thread-by-thread basis before writing the results to the output file. Therefore, hybrid resource-intensive offline analysis jobs are not suitable for colocation.

[0235] 5.2) Evaluation of the effectiveness of isolation technology based on SLE

[0236] Two typical isolation technologies, CPU binding and Docker container technology, were evaluated. Their SLE values ​​were compared with the default SLE value without isolation technology to assess the degree of performance interference control.

[0237] Based on the conclusions from the system performance ceiling section of this example, s2's colocation efficiency is moderate, so it was selected as the system for evaluating the effectiveness of isolation technology. Regarding the CPU binding technology settings, since s2 has 12 CPU cores, CPUs 0-1 are bound to the online service, and CPUs 2-11 are bound to the offline analysis job. Regarding the Docker container technology settings, the offline analysis job is deployed in a separate container on s2.

[0238] As shown in Table 19, compared to non-isolation technology, CPU binding reduced the SLE value by an average of 8.47% across all co-located workloads, with the largest reduction being 22.48% for Masstree-Union. Docker container technology reduced the SLE value by an average of 7.16% across all co-located workloads, with the largest reduction being 20.16% for Masstree-Union. Therefore, on s2, CPU binding more effectively mitigated the performance interference of co-located workloads, achieving a higher level of performance interference control.

[0239] Table 19 Comparison of SLE under different isolation technologies

[0240]

[0241]

[0242] See also Figure 4 , is a schematic diagram of an interference quantization system provided by an embodiment of the present application. The interference quantization system includes a processor and a memory, wherein the memory is used to store a computer program. When the computer program is executed by the processor, the interference quantization method described above is implemented.

[0243] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0244] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods described in the embodiments of the present invention. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various processor functions and data processing, thereby implementing the methods described in the aforementioned method embodiments.

[0245] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0246] One embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program. When the computer program is executed by a processor, the above-mentioned interference quantization method is implemented.

[0247] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for quantifying interference for mixed loads, characterized in that: The colocation load includes an online load and an offline load, and the method includes: For a characteristic indicator of a competing resource, obtaining first values ​​of the characteristic indicator at multiple time points when the online load and the offline load are running together, the competing resource including hardware resources required for the running of both the online load and the offline load; and Calculating a degree of dispersion between the plurality of first values ​​to obtain an interference entropy value, and quantifying a degree of resource contention of the mixed load using the interference entropy value, where the degree of resource contention refers to a degree of mutual interference between the online load and the offline load when using the contended resources; Wherein, the competition resources include multiple characteristic indicators; The calculating the discreteness between the plurality of first values ​​to obtain an interference entropy value includes: Calculating the degree of dispersion between the plurality of first values ​​of each characteristic indicator respectively, and obtaining a first intermediate entropy value corresponding to each characteristic indicator respectively; Obtaining the interference entropy value based on the first intermediate entropy value of the characteristic indicator; The obtaining of the interference entropy value based on the first intermediate entropy value of the characteristic indicator includes: When the online load is running alone, obtaining second values ​​of the characteristic indicator at multiple time points; and when the offline load is running alone, obtaining third values ​​of the characteristic indicator at multiple time points; Calculating the degree of dispersion between the plurality of second values ​​of each characteristic indicator respectively to obtain a second intermediate entropy value corresponding to each characteristic indicator respectively, and calculating the degree of dispersion between the plurality of third values ​​of each characteristic indicator respectively to obtain a third intermediate entropy value corresponding to each characteristic indicator respectively; Using the minimum value of the second intermediate entropy value and the third intermediate entropy value as the entropy value threshold; Among the characteristic indicators, determining target characteristic indicators whose first intermediate entropy values ​​are greater than the entropy value threshold, and performing normalization calculation on the first intermediate entropy values ​​of each target characteristic indicator to obtain optimized entropy values ​​corresponding to each target characteristic indicator; According to the weight of the optimized entropy value of each target characteristic indicator, multiplying the optimized entropy value of each target characteristic indicator by the corresponding weight and adding the results, and using the result of the addition as the interference entropy value; Wherein, for any of the target characteristic indicators, the weight of the optimized entropy value of the target characteristic indicator is determined based on the following method: When the online load and the offline load are running together, detecting a delay performance of the online load, where the delay performance represents a speed from when the online load requests to use the contention resource to when it receives a response; A correlation calculation is performed between the first value of the target characteristic indicator and the delay performance during the online load operation, and the calculated value is used as the weight of the optimized entropy value of the target characteristic indicator.

2. The method according to claim 1, wherein The normalizing calculation of the first intermediate entropy value of each target feature indicator includes: For any of the target characteristic indicators, the first intermediate entropy value of the target characteristic indicator is normalized and calculated based on the sum of the second intermediate entropy value and the third intermediate entropy value of the target characteristic indicator.

3. The method according to claim 1, wherein The co-located load runs in a system, and the system is used to accommodate hardware resources required during the operation of the co-located load. In the case where the co-located load runs in multiple systems respectively, the method further includes: Determining interference entropy values ​​when the colocation load runs in each system; Normalized interference entropy values ​​of the mixed load when running in each system are calculated to obtain second interference entropy values ​​corresponding to the mixed load in each system, so as to compare the resource contention levels of the mixed load when running in each system through the second interference entropy values.

4. The method according to claim 3, wherein Among the plurality of systems, at least two systems have different quantities of the characteristic indicators; The normalizing calculation of the interference entropy value when the colocation load is running in each system includes: For the multiple systems, determine the number of characteristic indicators in each system respectively, and based on the maximum number of characteristic indicators as a benchmark, perform normalization calculation on the number of characteristic indicators in the multiple systems respectively to obtain a first normalization coefficient corresponding to each system; For any system, the first normalization coefficient corresponding to the system is multiplied by the interference entropy value of the mixed load when running in the system, so as to perform normalization calculation on the interference entropy value of the mixed load when running in the system.

5. The method according to claim 1, wherein In the case where there are multiple co-located loads, the method further includes: Determining the interference entropy value of each mixed load operation; detecting a delay performance of the online load in each of the co-located loads, where the delay performance represents a speed from when the online load requests to use the contended resource to when it receives a response; For any of the mixed loads, determining a mapping degree between the interference entropy value and the delay performance of the mixed load, wherein the mapping degree represents an amount of change in the delay performance per unit change in the interference entropy value; Taking the mapping degree with the largest value as a reference, normalize the mapping degree of each of the colocation loads to obtain a second normalization coefficient; For any of the mixed loads, multiplying the second normalization coefficient of the mixed load by the interference entropy value to perform normalization calculation on the interference entropy value of the mixed load to obtain a first interference entropy value corresponding to the mixed load; The resource contention levels between the mixed deployment loads are compared based on the first interference entropy values ​​corresponding to the mixed deployment loads.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

7. An interference quantization system, characterized in that: The interference quantification system includes a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

Images