Program performance data collection method and system

Abstract

The invention discloses a program performance data collection method and system, belongs to the field of computer software testing, and aims at instrumentation of different program units, and records performance data by using a data structure of which the size is irrelevant to the program running duration during program running. The low-overhead collection effect is achieved by only storing array related information, meanwhile, instrumentation can be conducted on program units of the same layer or multiple levels, and multi-level program performance data collection is achieved. And the collected array can be subjected to subsequent analysis and comparison, which is of great significance to efficient program performance testing.

Description

Technical Field

[0001] The invention belongs to the field of computer software testing, and designs and implements a program performance data collection method and system. Background Art

[0002] As technology continues to develop, the complexity of modern software is also increasing rapidly. Developers need to deal with multiple programming languages, different platforms, and complex system architectures, and these factors work together to greatly increase the difficulty of software development and maintenance. Performance testing has also become more complicated because testers not only need to evaluate the performance of the software in various usage environments, but also need to simulate the diverse operations of real users and cope with challenges such as massive data processing. In addition, modern software usually requires frequent updates and iterations, which makes performance testing before each version release a more arduous task. Faced with such a complex environment, ensuring the efficient and stable operation of the software is undoubtedly a huge test.

[0003] The existing fine-grained software performance testing technology inserts the program, records and writes the entry and exit time of each function, and performs performance analysis on the written program trace after the program exits to locate the problem function. This technology can accurately record the running time of all functions of the program, but because the recorded data will gradually increase with the running time of the program, especially in the case of a large number of loops, this method will quickly exhaust the hard disk, affecting the operation of the entire system, and also affecting the program under test itself. The traditional performance data recording method writes out the code running time of the program under test one by one, and calculates the running time of each function, basic block or other code unit offline. This method is accurate, but the amount of data tends to become huge, which will not only affect the execution of the program under test itself, thereby disturbing the performance data, but also slow down the subsequent analysis and reduce the efficiency of the analysis. Summary of the invention

[0004] In order to fully or partially solve the above problems, the present invention proposes a program performance data collection method and system.

[0005] Program performance data collection methods, including:

[0006] Modifying a compiler so that the compiler can insert a probe into a program unit of a program to be tested; the program unit is a function or a basic block in the program;

[0007] Inserting a probe into a target program unit by the compiler, wherein the probe generates each single running time of the program unit;

[0008] The single running time is mapped to the array corresponding to the program unit to complete the program performance data collection.

[0009] Furthermore, the compiler determines the program unit where the probe needs to be inserted by reading the compilation options or environment variables of the program to be tested.

[0010] Furthermore, the method for generating a single running time of a program unit includes: obtaining a time when a program unit enters an execution state and a time when the program unit exits the execution state based on the probe, and calculating the difference between the two to obtain the single running time of the program unit.

[0011] Furthermore, the array length is configured based on target requirements, and the array elements are classified according to the set running time as an interval, and each element is the number of times the program unit is run.

[0012] Furthermore, the mapping operation includes: in response to the probe generating a running time, adding 1 to a corresponding element in the array corresponding to the program unit.

[0013] Furthermore, the program units are at the same level or at different levels.

[0014] Program performance comparison methods, including:

[0015] Collecting performance data of program units to be compared by using a program performance data collection method;

[0016] A comparative analysis is performed using the performance data.

[0017] Program performance data collection system, including:

[0018] A compiler modification unit, used for modifying the compiler so that the compiler can insert probes into program units; the program units are functions and basic blocks;

[0019] An inserting unit, used for inserting a probe into the program unit through the compiler, wherein the probe generates a single running time of the program unit;

[0020] A mapping unit is used to map the single running time to an array corresponding to the program unit.

[0021] The beneficial effects of the present invention are as follows:

[0022] The present invention designs a method for collecting program performance data that is independent of program runtime. By only saving array-related information, a low-overhead collection effect is achieved. The method can insert stubs for program units at the same level or multiple levels, thus realizing multi-level program performance data collection. The collected arrays can be subsequently analyzed and compared, which is of great significance for efficient program performance testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Flow chart of an embodiment of the present invention

[0024] Figure 2 Schematic diagram of a single pile insertion in an embodiment of the present invention

[0025] Figure 3 Schematic diagram of multi-level plugging in an embodiment of the present invention

[0026] Figure 4 Schematic diagram of the corresponding arrays of f1 and f2 in an embodiment of the present invention

[0027] Figure 5 Schematic diagram of the time interval configuration for performance data collection in an embodiment of the present invention

[0028] Figure 6 Schematic diagram of time record array length configuration for performance data collection in an embodiment of the present invention

[0029] Figure 7 Schematic diagram of a linearly increasing time array for performance data collection in an embodiment of the present invention

[0030] Figure 8 Schematic diagram of an exponentially increasing time array for performance data collection in an embodiment of the present invention DETAILED DESCRIPTION

[0031] In order to make the various technical features and advantages or technical effects of the above technical solutions of the present invention more obvious and easy to understand, the following Figure 1 Provide detailed explanation.

[0032] Step S1

[0033] Modify the program compiler of the relevant platform so that it can instrument the selected program units.

[0034] In this embodiment, the program unit refers to a program instruction segment such as a function or a basic block that contains program instructions and can be executed multiple times as a whole.

[0035] Step S2

[0036] Instrument the target program unit.

[0037] like Figure 2 As shown in the figure, perf_enter(fid) and __perf_exit(fid) are added at the beginning and end of function f to record the current time. fid is the ID that uniquely identifies the function generated during compilation.

[0038] The above is an operation performed by one-time plugging. This paper further supports plugging program units at the same layer or multiple layers, such as Figure 3 shown. Figure 3 The left side shows the instrumentation for function f. Figure 3The right side shows the instrumentation for each basic block in a function. This function supports the step-by-step location of performance bottlenecks from functions to basic blocks and fine-grained performance analysis at the basic block level, which can be further flexibly adjusted according to target requirements.

[0039] Step S3

[0040] Map the runtime obtained by instrumentation to the corresponding array.

[0041] This method uses an array data structure to record performance. For each program unit in the program, a unique corresponding array is maintained when the program unit is running, and each element of the array records the number of times the program unit is run within a time interval. Since the array length does not change and the number of program units is fixed, the size of the performance data has nothing to do with the program running time, but only with the number of program units and the length of the array.

[0042] Each time the instrumentation generates a running time, the value of the corresponding element in the array is increased by 1. Specifically, it is necessary to maintain a per-thread mapping lastEnterTime to record the last entry time of each function, and a mapping funcCallCounter to maintain the function to its time-consuming array. After obtaining the exit time, the time consumption is calculated based on the last entry time, and the time consumption array index is calculated based on the time consumption, array length and time interval, and the corresponding position count is increased by 1.

[0043] In this embodiment, the array groups elements in intervals of 5000 nanoseconds, and the array length is set to 1024. The time range that can be recorded by the array covers 0 to 515000 nanoseconds, and the last grid can be regarded as recording all calls that take longer than 515000 nanoseconds.

[0044] This embodiment performs data collection of this method on two functions f1 and f2, such as Figure 4 As shown in the figure, f1 runs 7+14+4+1=26 times in total, and f2 runs 92 times in total. Seven of f1's runs took between 0 and 5000 nanoseconds, and 14 of them took between 5000 and 10000 nanoseconds. The same goes for other run data.

[0045] f1 collects data for this method on another platform, and the data are 0, 5, 15, and 6. Then the statistic and pvalue obtained by the ks_2samp distribution comparison algorithm are 0.25 and 1.0 respectively. At this time, pvalue>statistic, so it can be considered that the two distributions are similar and the performance is close. If it is the opposite, the performance gap is large and further analysis is needed.

[0046] Furthermore, in order to flexibly adjust the accuracy of performance data, the performance data record runtime supports the configuration of the time interval size, time interval distribution method, and array length. In order not to affect the command line parameters of the program itself, these configurations are effective through environment variables. The environment variables described below are for reference only, and the actual implementation can be changed according to your own needs.

[0047] (1) Time interval configuration

[0048] The time interval for performance data recording is configured by the environment variable PERF_TIME_INTERVAL. The default value is 5000, in nanoseconds. The reference configurable values ​​are 2500, 5000, 10000, 15000, 20000. Other positive integers are also acceptable. Figure 5 The following table shows the cases where PERF_TIME_INTERVAL is configured as 2500, 5000, and 10000.

[0049] (2) Array length configuration

[0050] The array length during performance data recording is configured by the environment variable PERF_TIME_COUNT. The default value is 1024, and the optional values ​​are 512, 1024, 2048, 4096. Other positive integers are also acceptable. The array length affects the maximum time range that can be recorded. Figure 6 The following table shows the cases where PERF_TIME_COUNT is configured as 512, 1024, and 4096.

[0051] (3) Time interval distribution configuration

[0052] The time interval distribution mode during the performance data recording runtime is configured through the environment variable PERF_TIME_DIST. The default value is linear, which means linear increase. You can also configure expt, which means exponential increase. The time interval distribution mode affects the accuracy of the recorded time.

[0053] For the linear increase method, all time intervals are the same, such as Figure 7 As shown;

[0054] For the exponential increase method, the time interval increases exponentially, the default base is 2, and the time interval represented by the nth time interval is PERF_TIME_INTERVAL*2n-PERF_TIME_INTERVAL~

[0055] PERF_TIME_INTERVAL*2n, such as Figure 8 As shown in the figure, users can choose the corresponding distribution method according to their own needs.

[0056] In the above description, the time interval size, time interval distribution method, and array length configuration jointly affect the accuracy of the recorded performance data. When the time interval distribution method is fixed, the smaller the time interval and the longer the array length, the higher the accuracy of the recorded data, but the data volume will also be larger. When the time interval and array length remain unchanged, the linear time interval distribution method records more accurate data. The exponential growth time interval distribution method has uneven distribution of interval speeds, so the time accuracy decreases as time goes on.

[0057] Table 1 shows the overhead of the fine-grained performance testing tools of the present invention and the prior art when testing on four commonly used software:

[0058] program No instrumentation The present invention Existing tools Brotli 65.235s 75.054s 71.91s Flac 10.336s 25.038s 23.13min Gzip 30.680s 553.034s >2h XZ 4.144s 248.392s >2h

[0059] Table 1

[0060] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. Methods for collecting program performance data, including: Modifying a compiler so that the compiler can insert a probe into a program unit of a program to be tested; The program unit is a function or a basic block in a program; Inserting a probe into a target program unit by the compiler, wherein the probe generates each single running time of the program unit; The single running time is mapped to the array corresponding to the program unit to complete the program performance data collection.

2. The method according to claim 1, characterized in that The compiler determines the program unit where the probe needs to be inserted by reading the compilation options or environment variables of the program to be tested.

3. The method according to claim 1, characterized in that The method for generating a single running time of a program unit includes: obtaining the time when a program unit enters an execution state and the time when the program unit exits the execution state based on the probe, and calculating the difference between the two to obtain the single running time of the program unit.

4. The method according to claim 1, characterized in that: The array length is configured based on target requirements, and each element of the array is classified according to a set running time as an interval, and each element is the number of times the program unit is run.

5. The method according to claim 4, characterized in that The mapping operations include: In response to the probe generating a running time, a corresponding element in the array corresponding to the program unit is incremented by 1.

6. The method according to claim 1, characterized in that The program units are at the same level or at different levels.

7. Methods for comparing program performance, including: Collecting performance data of the program unit to be compared by the method of claim 1; A comparative analysis is performed using the performance data.

8. Program performance data collection system, including: A compiler modification unit, used for modifying the compiler so that the compiler can insert a probe into a program unit; The program units are functions and basic blocks; An inserting unit, used for inserting a probe into the program unit through the compiler, wherein the probe generates a single running time of the program unit; A mapping unit is used to map the single running time to an array corresponding to the program unit.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 8.

10. A storage medium storing a computer program, wherein when the computer program is executed by a computer, the method according to any one of claims 1 to 8 is implemented.

Images