Workload characterization method and analysis system driven by hotspots, combining instrumentation and performance event sampling

Abstract

The application discloses a hotspot-driven workload feature analysis method combining with a probe and performance event sampling, which is characterized in that the analysis method is used for a workload, a hotspot analyzer is used for hotspot positioning, a time-based sampling method is used to obtain workload hotspot feature information, a probe analyzer is used to combine with the start and end addresses of a hotspot code segment and / or a hotspot function to statistically analyze internal dynamic execution instructions of the workload hotspot, and instruction distribution feature information in the workload hotspot is obtained, a performance profiler is used to sample and statistically analyze specific performance events occurring during the execution of the workload hotspot, and the number of performance event occurrences and performance index feature information in the workload are obtained, the hotspot features, the hotspot internal instruction distribution features, the number of hotspot internal performance event occurrences and the performance index features are combined, the start and end addresses of the workload hotspot are used as media, a workload feature synthesizer is used, and hotspot-driven workload feature information is obtained.

Description

Technical Field

[0001] This invention belongs to the fields of computer systems, workload characteristic analysis, performance profiling, and system performance optimization technology, and relates to a hotspot-driven workload characteristic analysis method and analysis system that combines instrumentation and performance event sampling. Background Technology

[0002] Performance optimization has wide applications in computer systems, aiming to improve overall system performance by optimizing hardware and software configurations. Considering the real-world application scenarios of computer systems, system optimization is usually performed for specific workloads; therefore, characteristic analysis of specific workloads is a prerequisite for developing performance optimization solutions.

[0003] To pinpoint system performance bottlenecks, it's crucial to accurately and comprehensively analyze workload characteristics. These characteristics typically include runtime hardware and software behavior patterns, such as instruction distribution during dynamic execution, performance event counts, and hotspot instructions, code segments, and functions. By analyzing these workload characteristics, potential optimization opportunities can be discovered, providing direction and guidance for optimizing the hardware and software systems.

[0004] Due to the uncertainties in practical applications, extracting workloads from real-world application scenarios is not feasible in some situations. Selecting a suitable benchmark performance test program as the workload and conducting performance optimization under its guidance is a feasible alternative.

[0005] Benchmarking is an important method for measuring the performance of computer systems. Benchmarking typically involves running a series of representative programs and evaluating the computer system's performance under similar application workloads based on test scores. By analyzing the computer's behavior during benchmarking, potential software and hardware optimization opportunities can be identified; furthermore, the impact of optimization schemes on system performance can be objectively and quantitatively measured.

[0006] Currently, the methods for workload characteristic analysis mainly fall into the following three categories:

[0007] (1) Workload characteristic analysis based on instrumentation tools. Instrumentation is a technique that inserts probes into code to obtain information about various aspects of program runtime. Instrumentation can be used for performance profiling, such as obtaining information about the statistical distribution characteristics of executed instructions during program runtime, and using these characteristics to guide compiler optimization and the design of instruction set architecture and even microarchitecture;

[0008] (2) Workload characteristic analysis based on hardware performance counters. Hardware performance counters are components of the computer's internal performance monitoring unit (PMU). By setting the performance event selection register inside the performance monitoring unit, the corresponding hardware performance counter can capture specific performance events (such as cache misses, branch prediction failures, etc.), thereby obtaining the characteristics of the number of performance events and performance indicators during program runtime. By deriving corresponding indicators through performance event counting, the performance of each component in the hardware architecture can be reflected. Hardware design optimization can be performed on poorly performing components in order to improve the overall system performance.

[0009] (3) Hotspot-based workload characteristic analysis. Using a performance profiler, sampling is periodically triggered during program execution to capture and save runtime context information (such as the program counter and segment address registers). This context information, combined with the executable binary file, is used to analyze the machine instructions executed at the sampling time, thereby obtaining the call stack of the corresponding workload. Periodic sampling can identify statistically significant hotspot instructions, code segments, or functions during program execution, enabling hardware and software co-optimization for these hotspots.

[0010] However, current workload characteristic analyses are mostly limited to using a single technology or perspective, and a hotspot-driven workload analysis method that integrates multiple technologies is still lacking. The current methods face the following problems or challenges:

[0011] (1) In large-scale applications, using instrumentation, sampling, and other methods to perform global feature analysis on workloads can be extremely time-consuming or require a huge amount of storage space if the analysis requirements are small, and may even be impossible to complete. The challenges of workload feature analysis are how to optimize the process and efficiency of feature analysis by combining the characteristics of the workload itself, and how to achieve a balance between analysis accuracy and overhead.

[0012] (2) Performing global feature analysis on workloads results in an overly broad range of optimization options with a lack of focus, making it difficult to develop efficient optimization solutions. Effectively combining the characteristics of the workload itself to narrow down the range of optimization options is also a challenge faced by workload feature analysis. Summary of the Invention

[0013] To address the shortcomings of existing technologies, the purpose of this invention is to provide a hotspot-driven method for comprehensive analysis of workload characteristics that combines instrumentation and performance event sampling, comprising the following steps:

[0014] (1) For a specific workload, the machine under test runs the corresponding benchmark test program, uses a hotspot analyzer to locate the hotspots of the workload, and outputs the hotspot feature information of the workload. The hotspot feature information includes the number of hotspot code segments and / or hotspot functions, the overhead ratio of hotspot code segments and / or functions, the symbol names and call stacks of hotspot code segments and / or functions, and the start and end addresses of hotspot code segments and / or hotspot functions obtained from binary executable files with function call frame information.

[0015] Steps (2) and (3) below can be performed in parallel, without any order.

[0016] (2) For the hot code segments and / or hot functions of interest, combined with the start and end addresses of the hot code segments and / or hot functions obtained in step (1), use the instrumentation analyzer to count the number of dynamically executed instructions inside them, and output the instruction distribution characteristics of different functional types inside the workload hotspot;

[0017] (3) For the hot code segments and / or hot functions of interest, combined with the start and end addresses of the hot code segments and / or hot functions obtained in step (1), use the performance profiler to sample and count the specific performance events that occur during their internal execution, and output the number of performance events and performance index characteristics within the workload.

[0018] (4) Combining the hot spot feature information, hot spot internal instruction distribution feature information, hot spot internal performance event occurrence frequency and performance index feature information output in steps (1), (2), and (3), use the workload feature synthesizer to output the hot spot driven workload feature information.

[0019] The analysis method proposed in this invention can be used in practical applications and / or benchmark programs.

[0020] In this invention, step (4) outputs the hotspot-driven workload characteristic information, which can be used to perform workload characteristic analysis and provide insights and directions for performance optimization. Based on this, the following can be done: feature comparison between different workloads; if the workload is a benchmark performance program, the benchmark performance test program can be simplified; if the workload is a real application, the benchmark performance test program for that application scenario can be extracted from it.

[0021] The present invention also proposes an analysis system for implementing the above analysis method, the analysis system including a hotspot analyzer, a stud analyzer, a performance profiler, and a workload characteristic synthesizer;

[0022] The hotspot analyzer samples based on time, analyzes the sample information obtained from the sampling, and obtains hotspot feature information.

[0023] The instrumentation analyzer performs instrumentation analysis on the start and end address ranges of the obtained hot spot code segments and / or hot spot functions, and obtains the distribution characteristics of the execution instructions within the hot spot based on the statistics of the instrumented instruction types.

[0024] The performance profiler samples based on performance events, performs performance analysis on the start and end addresses of the obtained hot code segments and / or hot functions, and obtains the number of performance events and performance index characteristics within the hotspots after filtering.

[0025] The workload feature synthesizer is used to correlate and integrate hotspot feature information, the distribution characteristics of executed instructions within the hotspot, the number of performance events occurring within the hotspot, and performance index features, and output complete workload feature analysis information.

[0026] The beneficial effects of this invention include:

[0027] (1) This invention proposes a hotspot-driven comprehensive workload feature analysis method that combines instrumentation and performance event sampling. This method optimizes the feature analysis process and efficiency by incorporating the characteristics of the workload itself, achieving a good balance between analysis accuracy and overhead. Taking a benchmark performance test program workload feature analysis task as an example, traditional methods use instrumentation or profiling techniques to analyze the entire program. This method, however, uses a hotspot-driven approach to identify hotspots and uses instrumentation and performance event profiling techniques to analyze the workload. The proposed method effectively narrows the scope of feature analysis. For example, if a program contains 10 functions, the function consuming the most clock cycles is identified as the hotspot function, consuming a total of 30% of the clock cycles. Feature analysis only needs to be performed on this hotspot function using instrumentation and profiling techniques, without examining other functions. Compared to the traditional method of examining all 10 functions, theoretically, the time overhead of instrumentation and profiling can be reduced by 70% (the specific value depends on the definition of workload and hotspots), effectively improving the efficiency of workload feature analysis and reducing overhead.

[0028] (2) The workload characteristics formed by the method proposed in this invention take hotspots as the main description, which is conducive to narrowing the range of optimization schemes based on the instruction distribution characteristics and performance event characteristics within the hotspots, so as to formulate efficient optimization schemes and improve the efficiency of performance optimization.

[0029] (3) The method proposed in this invention is applicable not only to the workload characteristic analysis of benchmark performance test programs, but also to the workload characteristic analysis extracted from typical real-world scenarios, and has universality.

[0030] (4) Based on the workload characteristics obtained by the hotspot-driven workload analysis method proposed in this invention, workload comparison and workload simplification can be carried out efficiently under the premise that the characteristics remain approximately unchanged, which has good application value. Attached Figure Description

[0031] Figure 1 This is a flowchart of a workload characteristic comprehensive analysis method driven by hotspots and combining instrumentation and performance event sampling.

[0032] Figure 2 This is a schematic diagram of Example 1.

[0033] Figure 3 This is a schematic diagram of Example 2.

[0034] Figure 4 This is a schematic diagram of Example 3. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the present invention are all common knowledge and general knowledge in the art, and the present invention does not have any particular limitations.

[0036] This invention proposes a hotspot-driven comprehensive workload feature analysis method that combines instrumentation and performance event sampling. Specifically, for a specific workload, a hotspot analyzer is used to locate hotspots. A time-based sampling method is used to obtain hotspot feature information of the workload, including the start and end addresses and overhead percentages of hotspot code segments and / or hotspot functions. For the hotspot code segments and / or hotspot functions of interest, on the one hand, an instrumentation analyzer is used to statistically analyze the dynamically executed instructions within the workload hotspot, based on the start and end addresses, to obtain the instruction distribution feature information within the workload hotspot. On the other hand, a performance event-based sampling method is used, and a performance profiler is used to sample and statistically analyze specific performance events that occur during the execution within the workload hotspot, to obtain the frequency of performance events and performance index feature information within the workload. Finally, by combining the hotspot features, the dynamic execution instruction distribution feature within the hotspot, and the frequency of performance events and performance index feature information within the hotspot, and using the start and end addresses of the workload hotspot as a medium, a workload feature synthesizer is used to integrate them into a consistent, hotspot-driven workload feature information that combines instrumentation and performance event sampling.

[0037] The method proposed in this invention can be used for workload characteristic analysis in benchmark performance testing (Example 1), and based on this, workload characteristic comparison can be performed (Example 2), and the benchmark performance test program can be simplified (Example 3).

[0038] A flowchart illustrating the method proposed in this invention is shown below. Figure 1 As shown, the specific implementation methods for each step are described in detail below:

[0039] Step 1: Use a hotspot analyzer to locate workload hotspots

[0040] A workload hotspot is defined as a instruction, code segment, or function that consumes a significant amount of time within the workload. A function contains code segments, and code segments contain instructions. In practical applications, the analysis focuses on instructions, code segments, or functions, determined by the specific precision required for hotspot analysis. Typically, the analysis focuses on hotspot functions or code segments. For a computer system under a given workload, the hotspot analyzer samples the computer's operation at an appropriate sampling frequency, capturing and saving the system's runtime context information. This context information includes the program counter, stack pointer register, stack segment registers, and the workload's call stack in memory. Using this context information in conjunction with the binary executable file, the program counter and segment registers analyze the machine instructions executed by the computer at the sampling time. The function call frame information generated during the executable file's compilation, combined with the stack segment registers and stack pointer register, yields the call stack at the sampling time. The binary executable file originates from the compilation of the input workload program.

[0041] The program counter is used to analyze the address of the instruction currently being executed by the computer system;

[0042] The stack top pointer register is used to analyze the offset of the stack top relative to the stack start address in the memory of the currently running program in the computer system;

[0043] The stack segment register is used to analyze the starting address of the segment corresponding to the stack in the memory of the currently running program in the computer system;

[0044] The workload call stack refers to the memory area of ​​the test program's stack at the sampling time. This memory area contains several stack frames that record the program function state information at the sampling time.

[0045] The appropriate sampling frequency is generally set between 10 and 1000 Hz, such as 99 Hz or 999 Hz. The sampling frequency is related to the specific workload runtime and the desired analysis accuracy. First, a sufficient number of samples must be obtained for analysis to be possible. Specifically, if the workload has a long runtime (e.g., the SPEC CPU benchmark program requiring approximately 10 hours of runtime), the sampling frequency can be appropriately reduced to decrease the amount of data obtained from sampling. This reduces the number of samples collected, and consequently, the amount of data processed in the subsequent analysis process also decreases. However, due to the reduced number of samples, the accuracy of subsequent hotspot analysis may decrease. If more accurate hotspot analysis results are desired, or if the workload runtime is short (e.g., the SPEC JVM benchmark program requiring only 5 minutes of runtime), the sampling frequency can be appropriately increased, provided that the computer system's computing and storage resources can support it. This increases the number of samples collected, resulting in more accurate hotspot analysis results.

[0046] The call stack refers to the memory region of the test program's stack at the sampling time. This memory region contains several stack frames and records the program function state information at the sampling time. Combining the program counter, stack pointer register, and stack segment register, the code address of each sample at the sampling time can be obtained. Specifically, the program counter can be analyzed to obtain the address of the instruction being executed at the sampling time, i.e., the code address of the sample at the sampling time. The stack segment register can obtain the starting address of the corresponding segment in the running program's memory, and the stack pointer register can obtain the offset of the stack top relative to the stack start address in the running program's memory. Combining these two, the addresses of several stack frames contained in the memory region can be analyzed. These memory regions record information about the function call state of the running program at the sampling time. Furthermore, through the function call frame information preserved during the compiler's compilation of the executable file, stack frames can be correlated with function symbols, and the code address of the machine instruction being executed at the sampling time, the function it belongs to, and its function call relationships can be determined.

[0047] After collecting several samples using the above method, the function names pointed to by the call stacks of the samples are counted. Workload hotspots are selected by the number of samples falling into different code segments and / or functions. Specifically, the number of samples falling into each code segment and / or function is counted and sorted in descending order. The code segment and / or function with the most samples is the hotspot code segment and / or function. Similarly, depending on the specific application scenario and feature analysis needs, the n code segments and / or functions with the most samples can also be selected as hotspot code segments and / or functions. Since the hotspot analyzer samples at a fixed frequency, the number of samples contained in a code segment and / or function reflects the time consumed by that code segment and / or function. By counting the samples contained in a code segment and / or function, the more samples contained, the longer the corresponding code segment and / or function consumes time, and the more likely it is to become a workload hotspot.

[0048] Hotspot analyzers, through sampling, can pinpoint statistically significant workload hotspots based on the time consumed by obtained code segments and / or functions. Improving and optimizing these hotspots can effectively enhance overall system performance. Specifically, improvement and optimization methods include: optimizing the algorithms for the hotspot portions, using compiler-level optimization techniques to improve the performance of generated code, and optimizing related instructions within the hotspots at the hardware level to reduce their clock cycle consumption.

[0049] The hotspot analyzer outputs hotspot characteristic information for the workload, which should include the number of samples of hotspot code segments and / or functions, their overhead percentages, symbol names, and call stacks. Specifically, the hotspot characteristic information also includes the start and end addresses of hotspot code segments and / or hotspot functions obtained from binary executables containing function call frame information, providing this information to the instrumentation analyzer and performance profiler for further hotspot-driven workload characteristic analysis.

[0050] The sample size and overhead percentage of the hot code segments and / or functions are used to evaluate the popularity of hot functions and / or code segments;

[0051] The symbolic names and call stacks of the hot code segments and / or functions are used to indicate the names of the code segments and / or functions and their calling functions in the program source code, so as to quickly locate the scope of the hotspot and to find performance bottlenecks and optimize the program code in a targeted manner.

[0052] The following steps, Step 2 (using the piling analyzer to analyze hotspots) and Step 3 (using the performance profiler to analyze hotspots), can be performed in parallel without any specific order.

[0053] Step 2: Analyze hotspots using a piling-in analyzer.

[0054] Instrumentation is a technique that inserts probes into code to obtain various information related to the instruction flow during the dynamic execution of a program. Instrumentation can be used to analyze the behavioral characteristics of binary executable code during runtime. However, if a comprehensive and fine-grained instrumentation analysis of the workload is required, it will cause significant overhead and collect too much information, which will negatively impact the efficiency and effectiveness of the instrumentation analysis results.

[0055] Therefore, in this invention, the instrumentation analyzer can perform instrumentation analysis within the start and end address range of hotspot code segments and / or hotspot functions based on the hotspot feature information obtained in step 1. Within the workload hotspot, i.e., within the start and end address range of hotspot code segments and / or hotspot functions, dynamically executed instructions are classified and counted according to their functional categories (e.g., arithmetic operation instructions, logical operation instructions, memory access instructions, vectorized instructions, etc.), obtaining the number of instructions for each functional category. By leveraging the instruction distribution feature information within the workload hotspot, optimization can be performed at the hardware and software levels for the instruction category with a larger proportion (i.e., a larger number of instructions), which can effectively improve the overall system performance.

[0056] The aforementioned hardware and software optimizations refer to optimizations of software (compilers, linkers, assemblers, etc.) and / or hardware (microarchitecture). Specifically, for example, for a class of instructions with a large proportion of the code, modifying the hardware design and microarchitecture can reduce the number of clock cycles required to execute that instruction; another example is optimizing compilation strategies to use higher-performance instructions for code generation while maintaining semantic integrity; similarly, other optimization methods can be used, which will not be listed here.

[0057] Hotspot-driven instrumentation analysis only performs instrumentation analysis on the start and end address ranges of hotspot code segments and / or hotspot functions. Compared to overall, fine-grained instrumentation analysis of the workload, it collects relatively less information, thus effectively reducing the overhead caused by instrumentation, improving the efficiency of feature analysis, and making workload analysis more targeted.

[0058] Specifically, the instructions supported by the instruction set architecture of the machine under test are divided according to their functional categories (e.g., arithmetic operation instructions, logical operation instructions, memory access instructions, vectorized instructions, etc.). Instrumentation code is injected into one or more workload hotspots obtained in step 1. The instrumentation code can obtain the instructions dynamically executed by the current workload during execution. Statistics are performed according to the functional categories, and finally, the distribution characteristics of dynamically executed instructions within the workload hotspot are output. The instruction distribution characteristics specifically include the number of instructions of various types (functional categories) within the workload hotspot, and the proportion of each instruction to the total number of instructions within the workload hotspot.

[0059] Step 3: Analyze hotspots using a performance profiler.

[0060] The performance profiler also analyzes the workload through sampling, but unlike step 1, it focuses on sampling based on performance events within hot code segments and / or hot functions.

[0061] Performance event-based sampling is achieved using hardware performance counters and corresponding performance event selection registers in the computer's internal performance monitoring unit. The hardware performance counters support triggering overflow interrupts. When an overflow interrupt occurs, the operating system can respond to the interrupt and save the runtime context information for sampling.

[0062] Specifically, the performance profiler sets a performance event selection register to enable the corresponding hardware performance counters to capture specific performance events (such as instruction retirement, cache miss, branch prediction failure, etc.), and sets conditions to trigger overflow interrupts of the hardware performance counters to capture runtime context information. This context information includes the program counter, stack pointer register, stack segment registers, and the call stack of the workload in memory. When the machine runs the workload program, the hardware performance counters are started, and several samples are obtained based on the occurrence of overflow interrupts.

[0063] The condition for triggering an overflow interrupt of the hardware performance counter refers to setting the hardware performance counter to monitor specific performance events, such that an overflow interrupt is triggered when one or more specified performance events occur. This number of times is related to the desired analysis accuracy and performance profiling overhead. If an overflow interrupt is triggered every time a specified performance event occurs, the overhead is high, but the accuracy is guaranteed to reach its maximum. Conversely, to reduce overhead, for some frequently occurring performance events, an overflow interrupt can be set to trigger every 3, 5, or 10 events. Although the accuracy is slightly reduced, both the data volume and overhead can be decreased. For example, for frequently occurring performance events (such as instruction retirement), an overflow interrupt can be set to trigger once every 10 events; similarly, for less frequent performance events (such as branch prediction failure), an overflow interrupt can be set to trigger once every event.

[0064] Specifically, in the performance profiling step, not all samples are retained. The performance profiler only retains samples falling within the hot code segment and / or hot function start and end addresses given in step 1, and performs targeted performance profiling on performance events within the workload hotspots. Specifically, after sampling, the call stack information of each sample can be analyzed using the same call stack analysis method as in step 1 to obtain the address at the time of sampling. It can then be determined whether the sampled address falls within the range of the hot code segment and / or function start and end addresses, thus filtering out samples within this range. Alternatively, at the sampling time, the address of the currently running instruction can be obtained by analyzing the program counter and segment registers, and a judgment can be made to determine whether the current instruction address is within the range of the hot code segment and / or function start and end addresses. If it is, sampling is performed; otherwise, no sampling is performed.

[0065] By analyzing the number of samples falling within workload hotspots, the number of times a specified performance event occurs during the execution of hotspot code segments and / or hotspot functions can be approximated. Furthermore, corresponding performance metrics are derived from the performance event counts to form a correlation between the number of performance events occurring within the hotspot and the performance metric characteristics.

[0066] The performance metrics are calculated using the count of performance events and reflect the system's operating characteristics under workload. The selection of performance metrics needs to be based on the actual analysis requirements. Common performance metrics include, but are not limited to:

[0067] • The average number of clock cycles per instruction is calculated as follows: Average number of clock cycles per instruction = Number of clock cycles / Number of instructions;

[0068] • Branch prediction accuracy is calculated as follows: Branch prediction accuracy = 1 – (Branch prediction failure / Number of branch instructions);

[0069] • The number of cache misses per thousand instructions is calculated as follows: Number of cache misses per thousand instructions = 1000 * (Cache misses / Number of retired instructions).

[0070] The performance indicators can be set according to the actual analysis needs.

[0071] The frequency and performance metrics of performance events within a workload can reflect the performance of corresponding components in the hardware architecture. Optimizing the hardware design of poorly performing components can effectively improve the overall system performance. Specifically, for example, if the branch prediction accuracy is low within a hotspot, the prediction algorithm or logic structure of the processor's branch predictor should be optimized; if the number of cache misses per thousand instructions is high within a hotspot, the size and replacement strategy of the processor cache should be optimized.

[0072] Step 4: Use the workload feature synthesizer to output hotspot-driven workload features

[0073] The workload feature synthesizer can synthesize the hotspot feature information of the workload output by the hotspot analyzer in step 1; the distribution characteristics of the dynamically executed instructions within the hotspot output by the instrumentation analyzer in step 2; and the number of performance events and performance index characteristics during the dynamic execution within the hotspot output by the performance profiler in step 3.

[0074] Specifically, the workload feature synthesizer uses the addresses of hotspot code segments and / or hotspot functions as a medium to associate and integrate the results given by the instrumentation analyzer and the performance profiler. On the one hand, it associates the hotspot functions and / or code segments and / or instructions identified by the hotspot analyzer in machine language with the source code using the debugging information retained during the compilation of the executable file. On the other hand, it integrates the dynamic execution instruction distribution information, performance event occurrence frequency, and performance index information within the hotspots output by the instrumentation analyzer and the performance profiler with the corresponding hotspots. This makes the output comprehensive workload feature analysis results focus on the hotspots, including not only the corresponding source code, overhead ratio, start and end addresses, call stack, and other features of the hotspots themselves, but also, combined with instrumentation and performance event sampling techniques, the dynamic execution instruction distribution characteristics, performance event occurrence frequency, and performance index characteristics within the hotspots, thus forming a more comprehensive and rich workload feature analysis.

[0075] On the other hand, instrumentation analyzers and performance profilers can also explain the causes of hotspots. Specifically, for example, an instrumentation analyzer might identify a hotspot containing a large number of floating-point division instructions. These instructions consume many clock cycles, causing the code segment and / or function to take a long time to execute. Therefore, the hotspot analyzer identifies this hotspot, explaining that its occurrence is due to a large number of clock-cycle-consuming instructions. Similarly, a performance profiler might identify a hotspot with a high number of cache misses per thousand instructions, explaining that the hotspot is caused by multiple cache misses consuming a large number of clock cycles. The information provided by instrumentation analyzers and performance profilers helps explain the causes of hotspots, helps pinpoint performance bottlenecks, and provides direction and suggestions for performance optimization.

[0076] Example 1: Workload Characteristic Analysis of Benchmark Performance Testing

[0077] In performance evaluation scenarios, benchmark performance testing programs can be directly used to evaluate the performance of the computer system under test. Benchmark performance testing programs are a series of representative programs, typically extracted from real-world workloads. Benchmark performance tests assess the performance of a computer system under a specific workload by providing test scores (usually derived from metrics such as runtime and throughput). By observing the behavioral characteristics of the computer during benchmark performance testing, potential software and hardware optimization opportunities for specific application scenarios can be identified.

[0078] Using the hotspot-driven workload characteristic comprehensive analysis method proposed in this invention, which combines instrumentation and performance event sampling, feature analysis is performed on the function-level hotspots of a hypothetical benchmark performance test program. This illustrates the implementation process of the method described in this invention, as follows: Figure 2 As shown. The specific implementation plan is summarized as follows:

[0079] (1) Compile the source code of the benchmark performance test program to generate binary executable code including the symbol table. The symbol table and function call frame information need to be retained during compilation so that subsequent steps can combine the symbol table to resolve function symbols, start and end addresses and call stack, etc.

[0080] (2) Run the program. During the process, a hotspot analyzer is used to periodically sample the system to obtain function-level hotspots. The hotspot analyzer and the following steps can be implemented using performance profiling tools, such as Linux perf, OProfile, etc. The specific steps are as follows:

[0081] a) For example, the sampling interval is set to 99Hz (i.e., 99 samples per second). When sampling occurs, the runtime context information (program counter, stack pointer register, stack segment register, and call stack of the workload in memory, etc.) is captured and saved to form a sample.

[0082] b) After sampling, the code address and call stack of each sample at the sampling time can be obtained by analyzing the context information of the sample. Combined with the symbol table and function call frame information retained at compile time, the function address in the call stack can be matched with the function name in the source code.

[0083] c) By counting the function names pointed to by the call stack of all samples, we can find out the number and proportion of samples falling into each function. When the number of samples is large enough, we can use the proportion of sample number to estimate the proportion of function overhead.

[0084] d) Based on the cost ratio, select several hot functions, such as the n functions with the largest cost ratio (the first two functions are selected in this example), and then obtain the hotspot analysis results of the workload, including the cost ratio and start and end address information of the hot functions.

[0085] Steps (3) and (4) below can be performed simultaneously without any order requirement.

[0086] (3) Use an instrumentation analyzer to obtain the dynamic instruction distribution characteristics within the hotspot. The instrumentation analyzer and the following steps can be implemented using dynamic binary code instrumentation tools, such as Pin, DynamoRIO, etc. The specific steps are as follows:

[0087] a) Based on the hotspot function and its start and end addresses obtained in step (2), the binary executable code is dynamically instrumented, wherein the instrumentation is limited to the hotspot function to reduce the overhead caused by instrumentation.

[0088] b) Classify all or part of the instructions in the instruction set of the machine under test according to the needs of the research. For example, if only memory access instructions are of interest, divide memory access instructions into two types: load and store. Or, divide all instructions into arithmetic, logic, memory access, branch, etc. During instrumentation, obtain the dynamically executed instructions at runtime and classify and statistically analyze them according to the classification method to obtain the characteristic information of the distribution of dynamically executed instructions within each hotspot.

[0089] (4) Use a performance profiler to obtain the characteristics of performance events occurring within the hotspot, including the number of performance events and related metrics. The performance profiler and the following steps can be implemented using performance profiling tools, such as Linux perf, but the machine under test needs to have a performance monitoring unit that supports issuing overflow interrupts, and the operating system needs to support relevant interrupt handling routines. The specific steps are as follows:

[0090] a) Select performance events, such as clock cycles, inst., branch instructions, and branch prediction failures (br-misp). You can also derive corresponding performance metrics by combining the frequency of these performance events. For example, clock cycles and inst.com can be used to derive the average number of clock cycles per instruction (Cycles Per Instructions, CPI), and branch instructions and branch prediction failures can be used to derive the branch prediction miss rate (br-misp-rate).

[0091] b) Perform performance event-based sampling. By setting the performance monitoring unit, it is made to issue an overflow interrupt when a specified number of performance events occur to trigger sampling, capture and save runtime context information (such as program counter, stack top pointer register, stack segment register, and call stack of workload in memory) to form samples.

[0092] c) After sampling, the code address and call stack of each sample at the sampling moment can be obtained by analyzing the context information of the sample;

[0093] d) Based on the hotspot function and its start and end addresses obtained in step (2), the samples are filtered, and only the samples that fall inside the hotspot function are retained. The number of performance events inside the hotspot is obtained by counting the samples inside the hotspot, and the corresponding performance indicators are derived based on the number of occurrences to form the characteristics of the performance events inside each hotspot.

[0094] (5) Using a feature synthesizer, the hotspot features of the workload output by the hotspot analyzer in step (2), the distribution features of dynamically executed instructions inside the hotspot output by the instrumentation analyzer in step (3), and the performance event features of dynamically executed instructions inside the hotspot output by the performance profiler in step (4) are all based on the workload hotspot and can be directly integrated. The final output forms a more comprehensive and rich workload feature analysis, which includes the source code, overhead ratio, start and end addresses, call stack, distribution features of dynamically executed instructions inside the hotspot, and the number of performance events and performance indicators of performance events.

[0095] It should be noted that this embodiment uses a benchmark performance test program as an example to perform hotspot-driven workload characteristic analysis. However, this method is not limited to the analysis of benchmark performance test programs. It is also applicable to actual applications and workloads extracted from actual applications.

[0096] Example 2: Comparison of Workload Characteristics

[0097] In practical applications, it is sometimes necessary to compare the performance of the same workload on different machines under test. Comparative analysis of the workload characteristics on different machines can help to discover the reasons for the performance differences and guide the collaborative optimization of software and hardware.

[0098] For example, using the hypothetical benchmark performance test program described in Example 1, the method proposed in this invention is used to perform a comprehensive analysis of hotspot-driven workload characteristics on two different machines (Machine 1 and Machine 2), and a comparison of workload characteristics is performed. Figure 3 As shown.

[0099] For a given program, it is compiled on two different machines to generate binary executable code (retaining symbol tables and function call frame information). The method described in Example 1 is then used to perform hotspot-driven workload feature analysis, yielding the feature analysis results. Since the method described in this invention is hotspot-driven, the workload feature description is based on hotspots. Therefore, the workload features of the two machines are comparable. By comparing the differences in the workload features of the two machines (e.g., comparing the differences in the hotspot functions with the highest overhead in the two workloads), the reasons for the performance differences can be analyzed.

[0100] It should be noted that this embodiment can also be applied to the verification of third-party workload reduction methods. By analyzing the characteristics of the workload before and after modification and comparing the degree of similarity between the two, it is possible to determine whether the workload reduction is effective and representative.

[0101] Example 3: Simplification of the Benchmark Performance Test Procedure

[0102] Benchmark performance test programs are typically extracted from real-world application scenarios. Some computationally intensive benchmark performance test programs often involve a large number of iterative calculations. Furthermore, to ensure the reliability of the test results, repeated experiments are usually required to guarantee that the measurements accurately reflect the performance of the machine under test. These factors cause some benchmark performance tests to be time-consuming. System performance optimization relies on benchmark performance tests to measure the effectiveness of optimization solutions, and excessively long benchmark performance test program execution times can lead to inefficiencies.

[0103] To address the aforementioned issues, the hotspot-driven workload feature analysis method proposed in this invention is used to perform feature analysis on the workload running benchmark performance tests. While ensuring that the various characteristics within the workload hotspot remain unchanged, the benchmark performance test program is streamlined. For example, unnecessary non-hotspot functions are removed or the number of iterations within hotspot functions is reduced. This helps to reduce the runtime of the benchmark performance test program, thereby improving the efficiency of the performance optimization process.

[0104] The method described in this invention is used to simplify a hypothetical benchmark performance test program, such as... Figure 4 As shown.

[0105] Before the modification, the benchmark performance test program took a long time to run. Using the method proposed in this invention to perform workload characteristic analysis, it can be found that a certain hotspot function in the program (foo function in this example) takes a long time. Through code inspection, it was found that the operation performed by this function is an iterative operation of n times, which is a lot. Combined with the analysis of other characteristics inside the hotspot, it can be found that this function is computationally intensive. Reducing the number of iterations will not affect the workload characteristics of the hotspot function.

[0106] For the reasons mentioned above, the original program was modified to reduce the number of iterations within the function to m (where m is much smaller than n). Next, the modified program was validated using workload characteristic analysis, as shown in Example 2. The analysis yielded the hotspot-driven workload characteristics before and after the modification. If the characteristics within the workload hotspot remain essentially unchanged within certain limits, then the modified program can be used as the workload to reduce runtime and improve efficiency.

[0107] In addition to simplifying the benchmark performance testing procedure, this embodiment can be extended. For example, workloads can be extracted from typical real-world scenarios, their characteristics analyzed, and a simpler workload with approximately unchanged characteristics can be created as a replacement. This improves the efficiency of the performance optimization process while preserving the characteristics of real-world application workloads.

[0108] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of this invention are included in this invention and are protected by the appended claims.

Claims

1. A hotspot-driven workload characteristic analysis method combining instrumentation and performance event sampling, characterized in that, The analysis method targets the workload, using a hotspot analyzer to locate hotspots, and a time-based sampling method to obtain hotspot characteristic information of the workload; using an instrumentation analyzer to combine the start and end addresses of hotspot code segments and / or hotspot functions to statistically analyze the dynamically executed instructions within the workload hotspots to obtain instruction distribution characteristic information within the workload hotspots; and using a performance profiler to sample and statistically analyze specific performance events that occur during execution within the workload hotspots to obtain the frequency of performance events and performance indicator characteristic information within the workload. Finally, by combining hotspot characteristics, hotspot internal instruction distribution characteristics, hotspot internal performance event occurrence frequency and performance index characteristics, and using the start and end addresses of the workload hotspot as a medium, a workload feature synthesizer is used to integrate them into hotspot-driven workload feature information that combines instrumentation and performance event sampling. The analytical method includes the following steps: Step 1: Input the workload, use the hotspot analyzer to locate workload hotspots, and output hotspot characteristic information; Step 2: Use an instrumentation analyzer to analyze the workload hotspots and statistically output the characteristic information of the dynamic execution instruction distribution within the hotspots; And, using a performance profiler to analyze workload hotspots, obtain characteristic information about the number of performance events and performance metrics within the hotspots; In step two, the instrumentation analyzer performs instrumentation analysis within the start and end addresses of the hotspot code segments and / or hotspot functions obtained in step one, obtaining statistical information on the distribution of dynamically executed instructions within the hotspot code segments and / or hotspot functions. The instrumentation operation is implemented by inserting probes into the code. By injecting instrumentation code into one or more workload hotspots, the instructions dynamically executed by the current workload are obtained during execution. The number of instructions is counted according to their functional types, and finally, the characteristic information of the distribution of dynamically executed instructions within the hotspot is output. Instructions of different functional types include arithmetic operation instructions, logical operation instructions, memory access instructions, and vectorized instructions. In step two, the performance profiler analyzes the workload by sampling the hot code segments and / or hot functions based on performance events. The sampling based on performance events is implemented using hardware performance counters and corresponding performance event selection registers in the computer's internal performance monitoring unit. The hardware performance counters support triggering overflow interrupts. When an overflow interrupt occurs, the operating system can respond to the interrupt and save the runtime context information for sampling. In step two, the performance profiler only retains samples falling within the start and end addresses of the hot code segment and / or hot function given in step one. It counts the number of samples falling within the start and end addresses of the hot code segment and / or hot function to obtain the number of specified performance events occurring during the execution of the hot code segment and / or hot function, deriving performance metrics to form a characteristic of the number of performance events occurring within the hotspot and the performance metrics themselves. These performance metrics include the average number of clock cycles per instruction, branch prediction accuracy, and the number of cache misses per thousand instructions. Step 3: Use the workload feature synthesizer to synthesize the hotspot feature information output in Step 1, the feature information of the dynamic execution instruction distribution within the hotspot output in Step 2, and the feature information of the number of performance events and performance indicators within the hotspot, and output the hotspot-driven workload features.

2. The analytical method as described in claim 1, characterized in that, In step one, the hotspot analyzer samples the computer's operation at a sampling frequency of 10~1000Hz, captures and saves the system's runtime context information, analyzes the machine instructions executed by the computer at the sampling time through the program counter and segment registers, and obtains the call stack at the sampling time by combining the function call frame information generated during the compilation of the executable file with the segment registers and stack top pointer registers of the stack at the sampling time.

3. The analytical method as described in claim 2, characterized in that, The context information includes the program counter, stack pointer register, stack segment registers, and the call stack of the workload in memory; the binary executable file is derived from the compilation of the input workload program.

4. The analytical method as described in claim 2, characterized in that, If the hotspot analyzer performs time-based sampling of computer operation and obtains sampled samples, it can count the function names pointed to by the call stack of the samples and select workload hotspots by the number of samples falling in different code segments and / or functions.

5. The analytical method as described in claim 4, characterized in that, The number of samples falling on each code segment and / or function is counted and sorted in descending order. The top 1 or n code segments and / or functions with the most samples are identified as hotspot code segments and / or functions. The hotspot analyzer samples at a fixed frequency. The number of samples contained in a code segment and / or function reflects the time consumed by that code segment and / or function. By counting the samples contained in a code segment and / or function, the more samples contained, the longer the corresponding code segment and / or function consumes time, and the more likely it is to become a hotspot of the workload.

6. The analytical method as described in claim 1, characterized in that, In step one, the hotspot analyzer outputs hotspot feature information of the workload. The hotspot feature information includes the number of hotspot code segments and / or functions, the overhead ratio of hotspot code segments and / or functions, the symbol names and call stacks of hotspot code segments and / or functions, and the start and end addresses of hotspot code segments and / or hotspot functions obtained from binary executable files with function call frame information.

7. The analytical method as described in claim 1, characterized in that, In step three, the workload feature synthesizer uses the addresses of hotspot code segments and / or hotspot functions as a medium to integrate the results given by the instrumentation analyzer and the performance profiler. The output comprehensive workload feature analysis result is based on the workload hotspots, and then adds the dynamic execution instruction distribution characteristics, performance event occurrence frequency, and performance index characteristics within the hotspots to form a workload feature analysis. The workload feature analysis content includes the source code corresponding to the hotspot itself, the overhead ratio of the workload hotspot, the start and end addresses, the call stack, the dynamic execution instruction distribution characteristics, performance event occurrence frequency, and performance index characteristics within the hotspot.

8. The analytical method according to any one of claims 1-7, characterized in that, The analytical method described can be used in real-world applications and / or benchmark programs.

9. The application of the analysis method as described in any one of claims 1-7 in workload characteristic comparison and simplification of benchmark performance test procedures.

10. An analysis system for implementing the analysis method as described in any one of claims 1-7, characterized in that, The analysis system includes a hotspot analyzer, a stud analyzer, a performance profiler, and a workload characteristic synthesizer; The hotspot analyzer samples based on time, analyzes the sample information obtained from the sampling, and obtains hotspot feature information. The instrumentation analyzer performs instrumentation analysis on the start and end address ranges of the obtained hot spot code segments and / or hot spot functions, and obtains the distribution characteristics of the execution instructions within the hot spot based on the statistics of the instrumented instruction types. The performance profiler samples based on performance events, performs performance analysis on the start and end addresses of the obtained hot code segments and / or hot functions, and obtains the number of performance events and performance index characteristics within the hotspots after filtering. The workload feature synthesizer is used to correlate and integrate hotspot feature information, the distribution characteristics of executed instructions within the hotspot, the number of performance events occurring within the hotspot, and performance index features, and output complete workload feature analysis information.

Images