Method and device for obtaining performance test overhead, storage medium and electronic equipment

Abstract

The invention relates to a method and device for obtaining performance test overhead, a storage medium and electronic equipment, and the method comprises the steps: obtaining a test function for the performance test overhead, and setting a switch control function for the test function; the test function is operated based on the switch control function, and the basic overhead of the test function during operation is obtained; aiming at each to-be-tested performance tool, respectively embedding an instrumentation function corresponding to the to-be-tested performance tool into the test function to obtain a to-be-tested overhead test function corresponding to the to-be-tested performance tool; executing the to-be-tested overhead test function based on the switch control function, and respectively obtaining a first overhead for starting the instrumentation function in the to-be-tested overhead test function under the condition of no load and a second overhead for starting the instrumentation function under the load; and obtaining performance test overhead based on the basic overhead, the first overhead and the second overhead. The test accuracy of the to-be-tested application program can be improved.

Description

Technical Field

[0001] This invention relates to the field of operating system performance analysis technology, and in particular to a method, apparatus, storage medium, and electronic device for obtaining performance test overhead. Background Technology

[0002] Performance testing is an essential and crucial step in software and operating system development. Through performance testing, potential performance defects in software and operating systems (applications) can be identified during development, allowing for targeted improvements. Especially with the rapid development of the internet software industry, performance testing has become a vital factor in ensuring the smooth release of software versions. For example, for the Linux operating system, continuous performance testing of the upcoming Linux release allows for continuous optimization. Currently, performance testing typically utilizes performance tools, such as instrumentation tools. Instrumentation involves inserting additional performance testing code (instrument code) into the application under test to perform performance analysis, debugging, and monitoring. This instrumentation code collects various runtime information from the application under test, such as execution time and memory usage, and uses this information to evaluate the application's performance.

[0003] Instrumentation code is placed in the application under test (AUT). Therefore, performance tools based on instrumentation code consume resources during AUT performance testing, incurring performance overhead. This reduces the accuracy of AUT performance testing because the performance tools themselves also consume resources, significantly impacting applications with high real-time response requirements. Therefore, evaluating the performance overhead of various performance tools in AUT performance testing, and selecting the appropriate tool based on these overheads to minimize the impact of performance tools on AUT operation, is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, storage medium, and electronic device for obtaining performance test overhead.

[0005] Specifically, the present invention is achieved through the following technical solution: According to a first aspect of the present invention, a method for obtaining performance test overhead is provided, the method comprising: Obtain the test function used for performance testing overhead, and set a switch control function for the test function; Run the test function based on the switch control function to obtain the basic overhead of the test function during runtime; For each performance tool under test, an instrumentation function corresponding to the performance tool under test is embedded in the test function to obtain the test overhead test function corresponding to the performance tool under test. Based on the switch control function, the overhead test function is executed to obtain the first overhead of starting the instrumentation function under no load and the second overhead of starting the instrumentation function under load. Based on the aforementioned basic overhead, first overhead, and second overhead, the performance test overhead is obtained.

[0006] Optionally, the step of obtaining the test function for performance testing overhead includes: Select kernel test functions from the kernel modules of the target operating system that meet preset requirements; and / or, Based on the preset requirements, construct a self-editable test function.

[0007] Optionally, selecting kernel test functions that meet preset requirements from the kernel modules of the target operating system includes: The kernel test function is obtained by filtering out functions containing input / output, and / or scheduling, and / or lock contention from the kernel module of the target operating system.

[0008] Optionally, the switch control function has a built-in triggering mechanism, which includes: cyclic calling, fixed number of calls, and timed calling.

[0009] Optionally, the test function includes a kernel test function and a custom test function, and obtaining the basic overhead of the test function runtime includes: Edit the basic overhead acquisition function, in which the basic overhead acquisition function is set to synchronously call the kmet_get_ns() function and the clock_gettime() function; Call the kmet_get_ns() function in the kernel module to obtain the kernel-mode basic overhead of the kernel test function during runtime; Synchronously call the clock_gettime() function to obtain the user-space basic overhead of the self-edited test function during runtime; The sum of the kernel-mode basic overhead and the user-mode basic overhead is calculated to obtain the basic overhead of the test function during runtime.

[0010] Optionally, embedding the instrumentation function corresponding to the performance test tool in the test function includes: The instrumentation function corresponding to the performance test tool is embedded in the test function. The instrumentation function includes an acquisition function for characterizing data acquisition using the performance test tool and a processing function for data analysis and processing.

[0011] Optionally, embedding the instrumentation function corresponding to the performance test tool in the test function includes: The test function embeds a first instrumentation function and a second instrumentation function corresponding to the performance test tool. The first instrumentation function is used to characterize data acquisition using the performance test tool, and the second instrumentation function is used to characterize data acquisition, data analysis and processing using the performance test tool.

[0012] Optionally, the performance test overhead includes instrumentation overhead and load overhead. Based on the basic overhead, the first overhead, and the second overhead, the performance test overhead is obtained, including: The difference between the first overhead and the basic overhead is calculated to obtain the instrumentation overhead; The difference between the second cost and the first cost is calculated to obtain the load cost; The sum of the instrumentation cost and the load cost is calculated to obtain the performance test cost.

[0013] Optionally, the performance test overhead includes instrumentation overhead and load overhead. Based on the basic overhead, the first overhead, and the second overhead, the performance test overhead is obtained, including: Calculate the difference between the first cost and the basic cost to obtain the instrumentation cost, and obtain the product of the instrumentation cost and the pre-set instrumentation weight to obtain the first product; Calculate the difference between the second cost and the first cost to obtain the load cost, obtain the product of the load cost and the preset load weight to obtain the second product, and the sum of the instrumentation cost and the load weight is 1; The sum of the first product and the second product is calculated to obtain the performance test cost.

[0014] The method for obtaining performance testing overhead in this technical solution involves: acquiring a test function for performance testing overhead; setting a switch control function for the test function; running the test function based on the switch control function to obtain the basic overhead of the test function's runtime; for each performance tool under test, embedding an instrumentation function corresponding to that performance tool into the test function to obtain a test overhead test function corresponding to that performance tool; executing the test overhead test function based on the switch control function to obtain the first overhead of starting the instrumentation function under no load and the second overhead of starting the instrumentation function under load; and obtaining the performance testing overhead based on the basic overhead, the first overhead, and the second overhead. In this way, by obtaining the overhead of the test function without an instrumentation function, starting the instrumentation function under no load, and starting the instrumentation function under load, the performance tool with the lowest overhead can be selected for performance testing of the application under test, effectively improving the testing accuracy of the application under test.

[0015] According to a second aspect of the present invention, an apparatus for obtaining performance test overhead is provided, the apparatus comprising: The test function setting module is used to obtain the test function used for performance testing overhead and to set the switch control function for the test function; The basic overhead acquisition module is used to run the test function based on the switch control function and acquire the basic overhead of the test function during runtime. The instrumentation module is used to embed the instrumentation function corresponding to each performance test tool into the test function to obtain the test overhead test function corresponding to the performance test tool. An instrumentation triggering module is used to execute the test function of the overhead under test based on the switch control function, and to obtain the first overhead of starting the instrumentation function under no load and the second overhead of starting the instrumentation function under load in the test function of the overhead under test. The test overhead acquisition module is used to acquire the performance test overhead based on the basic overhead, the first overhead, and the second overhead.

[0016] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, wherein when the program is executed by a processor, it implements the steps of the method for obtaining performance test overhead in any possible implementation of the first aspect.

[0017] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for obtaining performance test overhead in any possible implementation of the first aspect. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for obtaining performance test overhead according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the acquisition of basic overhead in a method for obtaining performance test overhead provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the acquisition of the first cost in a method for obtaining performance test costs provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the acquisition of the second overhead in a method for obtaining performance test overhead provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of batch testing in a method for obtaining performance test overhead provided by an embodiment of the present invention; Figure 6 A schematic diagram of an apparatus for obtaining performance test overhead provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] In related technologies, methods that test the application under test (AUT) by setting instrumentation code in the AUT have limitations. Since different instrumentation codes can perform tests on the AUT, and different instrumentation codes (performance tools) consume varying amounts of resources, the degree of performance test accuracy degradation varies accordingly. Therefore, it is necessary to evaluate the performance testing overhead of various performance tools in the AUT to select the appropriate tool that meets the AUT's performance testing requirements, thereby minimizing the impact of performance tools on the AUT's operation and improving the test accuracy.

[0023] This embodiment provides a method for obtaining performance testing overhead. By setting a test function for performance testing overhead and controlling the execution of the test function using a set switch control function, the method obtains the basic overhead of the test function's execution. Furthermore, for each performance tool under test, a corresponding instrumentation function is embedded in the test function. The overhead of starting the instrumentation function in the test function under no load and under load is obtained respectively. Based on the obtained overhead, the performance testing overhead of the corresponding performance tool under test is calculated. Then, based on each performance testing overhead, the performance tool with the lowest overhead is selected for performance testing of the application under test, effectively reducing the impact of performance tools on the operation of the application under test and improving the testing accuracy of the application under test.

[0024] See Figure 1 This invention provides a method for obtaining performance test overhead, which may include the following steps: S101. Obtain the test function used for performance testing overhead, and set a switch control function for the test function; In this embodiment, since the performance testing overhead of the performance tool during application execution is evaluated, the function settings for performance testing based on the performance tool are required for the corresponding test function.

[0025] In this embodiment, as an optional embodiment, obtaining the test function for performance testing overhead includes: Select kernel test functions from the kernel modules of the target operating system that meet preset requirements; and / or, Based on the preset requirements, construct a self-editable test function.

[0026] In this embodiment, as an optional embodiment, a kernel test function that meets preset requirements is selected from the kernel module of the target operating system, including: The kernel test function is obtained by filtering out functions containing input / output, and / or scheduling, and / or lock contention from the kernel module of the target operating system.

[0027] In this embodiment, the kernel test function that meets the preset requirements is a controllable, stable, and recursibly callable test function. Since performance testing overhead measurements generally require multiple repeated measurements to obtain an effective distribution, the kernel test function needs to be repeatedly triggered (recursively callable) to accumulate sufficient sample measurement data, avoiding the influence of the randomness of limited calls on the results. Controllability and stability mean that the selected function is deterministic, ensuring that the comparison of performance testing overhead mainly comes from the instrumentation itself, rather than the random fluctuations in the kernel test function logic. This ensures that the selected kernel test function has triggerability and predictable runtime, preventing large fluctuations in execution time due to I / O, scheduling, and lock contention, which could cause the differences before and after instrumentation to be drowned out by background noise, thus avoiding the impact of uncertain operations on performance testing overhead.

[0028] In this embodiment, the test function can be derived from the kernel module by setting preset requirements, or it can be obtained by directly editing the code using the preset requirements, or it can be obtained by combining the kernel module and self-editing. This embodiment does not limit this.

[0029] In this embodiment, the switch control function includes, but is not limited to, the proc switch function, for triggering the test function. As an optional embodiment, a control node can be created in the / proc file system, for example, the node / proc / probe_test, and a trigger file interface ( / proc / tracepoint_test_trigger) or script text can be created under the node / proc / probe_test to trigger the execution of the test function.

[0030] In this embodiment, as an optional embodiment, the switch control function triggers the test function through a built-in triggering mechanism. The triggering mechanism includes, but is not limited to, cyclic calls, fixed-number calls, and timed calls. The triggering mechanism is used to ensure that the number of calls is controllable within a pre-set measurement period.

[0031] S102. Run the test function based on the switch control function, and obtain the basic overhead of the test function during runtime; In this embodiment, the test function is executed without loading any performance tools (instrumentation) to obtain the basic overhead of the test function's runtime. As an optional embodiment, the basic overhead includes, but is not limited to, execution time and CPU utilization.

[0032] In this embodiment, as an optional embodiment, the test function is a kernel test function, and the basic overhead of the test function during runtime is obtained, including: Call the kmet_get_ns() function in the kernel module to obtain the basic overhead of the kernel test function during runtime.

[0033] In this embodiment, the execution time of the kernel test function is collected using the kmet_get_ns() function.

[0034] In this embodiment, as another optional embodiment, the test function is a self-edited test function, and the basic overhead of the test function during runtime includes: Call the clock_gettime() function to obtain the basic overhead of the self-edited test function during runtime.

[0035] In this embodiment, as another optional embodiment, the test function includes a kernel test function and a custom test function, and the basic overhead of the test function during runtime includes: Edit the basic overhead acquisition function, in which the basic overhead acquisition function is set to synchronously call the kmet_get_ns() function and the clock_gettime() function; Call the kmet_get_ns() function in the kernel module to obtain the kernel-mode basic overhead of the kernel test function during runtime; Synchronously call the clock_gettime() function to obtain the user-space basic overhead of the self-edited test function during runtime; The sum of the kernel-mode basic overhead and the user-mode basic overhead is calculated to obtain the basic overhead of the test function during runtime.

[0036] In this embodiment, by setting synchronous calls to the `ktime_get_ns()` and `clock_gettime()` functions in the basic overhead acquisition function, the execution time obtained by the user-mode `clock_gettime` function and the kernel-mode `ktime_get_ns` function can be based on the same clock source, thereby effectively improving the accuracy of execution time acquisition.

[0037] In this embodiment, if the platform running the application under test has a precise clock or hardware timestamp, as an optional embodiment, the basic overhead of the test function runtime is obtained, including: The underlying overhead is obtained using the Precision Time Protocol (PTP) or the TimeStamp Counter (TSC).

[0038] Figure 2 This diagram illustrates the acquisition of basic overhead in a method for obtaining performance test overhead provided in an embodiment of the present invention. Figure 2As shown in the figure, the application to be tested is the Base test program, the test function is the function under test, the Base test program calls the function under test, the function under test returns the result, and the basic overhead is obtained.

[0039] S103. For each performance tool under test, embed the instrumentation function corresponding to the performance tool under test into the test function to obtain the test overhead test function corresponding to the performance tool under test. In this embodiment, as an optional implementation, the instrumentation function is embedded using the extended Berkeley Packet Filter (Ebpf) or Linux Performance Events (Perf) based on the Hardware Performance Monitoring Unit (PMU). Ebpf allows users to run sandboxed programs safely and efficiently without modifying the kernel source code or loading kernel modules, while Perf, as a user-facing command-line tool interface, can handle various performance events.

[0040] In this embodiment, the performance testing tool can be multiple performance tools used for comparing performance testing overhead. Based on the same test function, for each performance testing tool, the instrumentation function corresponding to that performance testing tool is embedded into the test function to obtain the overhead test function corresponding to that performance testing tool. Each performance testing tool corresponds to one overhead test function. As an optional embodiment, the instrumentation function is embedded into the test function; for details, please refer to relevant technical documents, which will not be elaborated on in this embodiment.

[0041] In this embodiment, as an optional embodiment, the instrumentation function corresponding to the performance test tool is embedded in the test function, including: The instrumentation function corresponding to the performance test tool is embedded in the test function. The instrumentation function includes an acquisition function for characterizing data acquisition using the performance test tool and a processing function for data analysis and processing.

[0042] In this embodiment, the data acquisition function indicates that the instrumentation function is in an unloaded operating state, while the data analysis and processing function indicates that the instrumentation function is in a loaded operating state.

[0043] In this embodiment, as another optional embodiment, the instrumentation function corresponding to the performance test tool is embedded in the test function, including: The test function embeds a first instrumentation function and a second instrumentation function corresponding to the performance test tool. The first instrumentation function is used to characterize data acquisition using the performance test tool, and the second instrumentation function is used to characterize data acquisition, data analysis and processing using the performance test tool.

[0044] S104. Execute the overhead test function based on the switch control function, and obtain the first overhead of starting the instrumentation function under no load and the second overhead of starting the instrumentation function under load in the overhead test function. In this embodiment, the first overhead of running when the test function is instrumented but the instrumentation function does not start the load (no load) is obtained, and the second overhead of running when the test function is instrumented and the instrumentation function starts the load is obtained.

[0045] In this embodiment, the instrumentation function enables the data collection required to obtain performance test overhead under no load, but does not enable the collection load logic (load). The collection load logic includes, but is not limited to: stack sampling, write mapping, parameter recording, and event reporting.

[0046] Figure 3 This is a schematic diagram illustrating the acquisition of the first cost in a method for obtaining performance test costs provided by an embodiment of the present invention. Figure 3 As shown, the instrumentation function is called the Hook instrumentation program under no load. The Base test program calls the function under test, the function under test returns the result, obtains the basic overhead, and then forwards the call to the Hook instrumentation program. The Hook instrumentation program calls the function under test, obtains the return result, and then the Hook instrumentation program returns the return result to the Base test program.

[0047] Figure 4 This is a schematic diagram illustrating the acquisition of the second cost in a method for obtaining performance test costs provided in an embodiment of the present invention. For example... Figure 4 As shown, instrumentation functions are called performance utility programs under load, and... Figure 3 The process is similar, but the Hook instrumentation program is replaced with a performance utility program.

[0048] Figure 5 This is a schematic diagram of batch testing in a method for obtaining performance test overhead provided by an embodiment of the present invention. Figure 5 As shown, this embodiment can realize batch testing functionality. The instrumentation function corresponds to the Hook instrumentation program, the Perf performance tool program, and the ebpf performance tool program. Figure 3 and Figure 4 The testing process is similar.

[0049] S105. Based on the basic overhead, the first overhead, and the second overhead, obtain the performance test overhead.

[0050] In this embodiment, as an optional embodiment, the performance test overhead includes instrumentation overhead and load overhead. Based on the basic overhead, the first overhead, and the second overhead, the performance test overhead is obtained, including: The difference between the first overhead and the basic overhead is calculated to obtain the instrumentation overhead; The difference between the second cost and the first cost is calculated to obtain the load cost; The sum of the instrumentation cost and the load cost is calculated to obtain the performance test cost.

[0051] In this embodiment, by triggering the test function and recording the execution time, the difference between the first overhead and the basic overhead is calculated to obtain the instrumentation overhead; and the difference between the second overhead and the first overhead when the test function is instrumented and the performance tool is running under load is calculated to obtain the load overhead.

[0052] In this embodiment, as an optional embodiment, the instrumentation cost and load cost are calculated using the following formula: Instrumentation cost = Time (duration) corresponding to the first cost - Time corresponding to the basic cost; Load overhead = Time corresponding to the first overhead - Time corresponding to the first overhead

[0053] In this embodiment, as another optional embodiment, the performance test overhead includes instrumentation overhead and load overhead. Based on the basic overhead, the first overhead, and the second overhead, the performance test overhead is obtained, including: Calculate the difference between the first cost and the basic cost to obtain the instrumentation cost, and obtain the product of the instrumentation cost and the pre-set instrumentation weight to obtain the first product; Calculate the difference between the second cost and the first cost to obtain the load cost, obtain the product of the load cost and the preset load weight to obtain the second product, and the sum of the instrumentation cost and the load weight is 1; The sum of the first product and the second product is calculated to obtain the performance test cost.

[0054] In this embodiment, as an optional embodiment, the method further includes: The basic overhead, first overhead, second overhead, instrumentation overhead, and load overhead are visualized and archived. For each performance testing tool, an output chart is plotted based on the displayed data and stored historical data. The data includes the basic overhead, first overhead, second overhead, instrumentation overhead, and load overhead.

[0055] In this embodiment, the results of performance test overhead are visualized and archived, for example, by saving the measurement results in the form of character-separated values ​​(CSV), JSON, database, etc.

[0056] In this embodiment, as an optional embodiment, the output charts include, but are not limited to, line charts and bar charts.

[0057] In this embodiment, as another optional embodiment, the method further includes: Based on the performance testing overhead of each performance testing tool, the performance testing tool with the lowest performance testing overhead is selected for testing the application to be released.

[0058] In this embodiment, as another optional embodiment, the method further includes: The acquired basic overhead, first overhead, and second overhead are uploaded to a remote analysis system in JSON / ProtoBuf format to obtain performance test overhead using the remote analysis system.

[0059] In this embodiment, the test data is output in JSON / ProtoBuf format and uploaded to the remote analysis system.

[0060] The method for obtaining performance test overhead in this embodiment constructs a test function, a switch control function, and an instrumentation function. The switch control function enables batch dynamic triggering of tests, the test function can achieve testing capabilities without recompiling the kernel, and the instrumentation code in the instrumentation function is set to minimize operations, effectively eliminating handle interference. This allows this embodiment to support comparative testing of performance tools under different runtime frameworks, and has the following beneficial effects: 1. Quantitatively evaluate the performance impact of instrumentation functions on the target test function: By precisely controlling the triggering conditions and execution environment before and after instrumentation, the average runtime of the target test function before and after instrumentation (rather than the macroscopic CPU utilization) can be quantitatively calculated, thereby evaluating the impact of instrumentation on performance and providing high-precision performance analysis.

[0061] 2. Evaluate the runtime overhead of different instrumentation functions (such as eBPF and kernel modules) in a unified manner: This embodiment applies the same test function to different performance testing tools, thereby providing a unified interface and measurement method. It can compare and analyze the instrumentation overhead of eBPF (such as using the libbpf framework) and kernel module (such as tracepoint hook) methods under the same test scenario, providing a quantitative basis for instrumentation function selection.

[0062] 3. Possesses a reentrant and highly repeatable testing framework: This embodiment features the characteristics of being executable multiple times and having high consistency, making it suitable for multiple rounds of data collection and statistical analysis. This effectively eliminates errors caused by random fluctuations and enhances the reliability of the evaluation conclusions.

[0063] 4. Instrumentation functions can be dynamically loaded / unloaded to adapt to different testing scenarios: The instrumentation logic in this embodiment can be dynamically attached and detached. For example, eBPF programs support pinning and continuous attachment to tracepoints, and kernel modules support dynamic loading and unloading, allowing the instrumentation functions in this embodiment to be flexibly inserted into test paths without interfering with the continuous operation of the original system.

[0064] 5. Suitable for closed-loop automated testing environments: The method in this embodiment is automatically triggered through a trigger file interface (such as / proc / tracepoint_test_trigger) or script, making it suitable for integration into an automated testing framework to achieve continuous evaluation and regression analysis of the performance impact of instrumentation.

[0065] 6. High versatility, compatible with multiple kernel versions and architecture platforms: The method described in this embodiment is adaptable to different Linux kernel versions and various architectures, such as x86_64 and arm64, making it easy to deploy in embedded and server environments.

[0066] 7. Provide standardized evaluation indicators: All evaluation results provide standardized indicator outputs and support archiving of results in the form of tables, logs, charts, etc., which facilitates horizontal comparison, trend analysis and result reproduction.

[0067] The method in this embodiment can be widely applied to performance impact assessment scenarios for various runtime monitoring mechanisms such as eBPF and Perf. For example, it can be used in performance tool development to clarify the performance impact of performance tools on application runtime when developing eBPF or Perf-based performance tools, thus supporting the promotion and deployment of these tools. Another example is its application in performance verification for automotive operating systems and embedded systems: since operating systems and embedded systems are extremely sensitive to time overhead, if instrumentation functions introduce significant additional time delays, it will affect system real-time performance. Using the method in this embodiment, through automated batch testing, multi-round comparisons, and statistical analysis, the overhead of instrumentation functions (tools) can be assessed in advance.

[0068] Based on the same inventive concept, such as Figure 6 As shown, this embodiment of the invention also provides an apparatus for obtaining performance test overhead, the apparatus comprising: The test function setting module 601 is used to obtain the test function used for performance testing overhead and set the switch control function for the test function; In this embodiment, as an optional embodiment, the test function setting module 601 is specifically used for: Select kernel test functions from the kernel modules of the target operating system that meet preset requirements; and / or, Based on the preset requirements, construct a self-editable test function.

[0069] In this embodiment, as an optional embodiment, a kernel test function that meets preset requirements is selected from the kernel module of the target operating system, including: The kernel test function is obtained by filtering out functions containing input / output, and / or scheduling, and / or lock contention from the kernel module of the target operating system.

[0070] The basic overhead acquisition module 602 is used to run the test function based on the switch control function and acquire the basic overhead of the test function during runtime. In this embodiment, the switch control function has a built-in triggering mechanism, which includes: cyclic calling, fixed number of calls, and timed calling.

[0071] In this embodiment, as an optional embodiment, the test function includes a kernel test function and a self-edited test function, and the basic overhead acquisition module 602 is specifically used for: Edit the basic overhead acquisition function, in which the basic overhead acquisition function is set to synchronously call the kmet_get_ns() function and the clock_gettime() function; Call the kmet_get_ns() function in the kernel module to obtain the kernel-mode basic overhead of the kernel test function during runtime; Synchronously call the clock_gettime() function to obtain the user-space basic overhead of the self-edited test function during runtime; The sum of the kernel-mode basic overhead and the user-mode basic overhead is calculated to obtain the basic overhead of the test function during runtime.

[0072] The instrumentation module 603 is used to embed the instrumentation function corresponding to the performance tool under test into the test function for each performance tool under test, so as to obtain the test overhead test function corresponding to the performance tool under test. In this embodiment, as an optional embodiment, the insertion module 603 is specifically used for: The instrumentation function corresponding to the performance test tool is embedded in the test function. The instrumentation function includes an acquisition function for characterizing data acquisition using the performance test tool and a processing function for data analysis and processing.

[0073] In this embodiment, as another optional embodiment, the insertion module 603 is specifically used for: The test function embeds a first instrumentation function and a second instrumentation function corresponding to the performance test tool. The first instrumentation function is used to characterize data acquisition using the performance test tool, and the second instrumentation function is used to characterize data acquisition, data analysis and processing using the performance test tool.

[0074] The instrumentation triggering module 604 is used to execute the test function of the overhead under test based on the switch control function, and to obtain the first overhead of starting the instrumentation function under no load and the second overhead of starting the instrumentation function under load in the test function of the overhead under test. In this embodiment, the instrumentation function enables the data collection required to obtain performance test overhead under no load, but does not enable the collection load logic (load). The collection load logic includes, but is not limited to: stack sampling, write mapping, parameter recording, and event reporting.

[0075] The test overhead acquisition module 605 is used to acquire the performance test overhead based on the basic overhead, the first overhead, and the second overhead.

[0076] In this embodiment, as an optional embodiment, the test overhead acquisition module 605 is specifically used for: The difference between the first overhead and the basic overhead is calculated to obtain the instrumentation overhead; The difference between the second cost and the first cost is calculated to obtain the load cost; The sum of the instrumentation cost and the load cost is calculated to obtain the performance test cost.

[0077] In this embodiment, as another optional embodiment, the performance test overhead includes instrumentation overhead and load overhead, and the test overhead acquisition module 605 is specifically used for: Calculate the difference between the first cost and the basic cost to obtain the instrumentation cost, and obtain the product of the instrumentation cost and the pre-set instrumentation weight to obtain the first product; Calculate the difference between the second cost and the first cost to obtain the load cost, obtain the product of the load cost and the preset load weight to obtain the second product, and the sum of the instrumentation cost and the load weight is 1; The sum of the first product and the second product is calculated to obtain the performance test cost.

[0078] In this embodiment, as an optional embodiment, the device further includes: A visualization module (not shown in the figure) is used to visualize and archive the basic overhead, first overhead, second overhead, instrumentation overhead, and load overhead. For each performance testing tool, an output chart is plotted based on the displayed data and stored historical data. The data includes the basic overhead, first overhead, second overhead, instrumentation overhead, and load overhead.

[0079] In this embodiment, as another optional embodiment, the device further includes: The tool selection module is used to select the performance testing tool with the lowest performance testing cost for testing the application to be released, based on the performance testing cost of each tool.

[0080] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, wherein when the program is executed by a processor, it implements the steps of the method for obtaining performance test overhead in any of the above possible implementations.

[0081] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0082] Based on the same inventive concept, see [link to inventive concept] Figure 7 This invention also provides an electronic device, including a memory 101 (e.g., non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, it implements the steps of the method for obtaining performance test overhead in any of the above possible implementations, which can be equivalent to the aforementioned apparatus for obtaining performance test overhead. Of course, the processor can also be used to process other data or perform calculations. This electronic device can be a PC, server, terminal, or other similar device.

[0083] like Figure 7 As shown, the electronic device may also include: memory 103, network interface 104, and internal bus 105. In addition to these components, other hardware may also be included, which will not be described in detail here.

[0084] It should be noted that the aforementioned device for obtaining performance test overhead can be implemented by software. As a logical device, it is formed by the processor 102 of the electronic device in which it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0085] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0086] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by special-purpose logic circuitry—such as FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), and the device can also be implemented as special-purpose logic circuitry.

[0087] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0088] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0089] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0090] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0091] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0092] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0093] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method of obtaining performance test overhead, characterized by, The method comprises the following steps: obtaining a test function for performance test overhead, setting a switch control function for the test function; running the test function based on the switch control function, and obtaining a basic overhead when the test function is running; for each performance tool to be tested, embedding a corresponding plug-in function of the performance tool to be tested in the test function to obtain a test function of the performance tool to be tested corresponding to the overhead; based on the switch control function, executing the test function of the overhead to be tested, and respectively obtaining a first overhead of the plug-in function under no load and a second overhead of the plug-in function under load in the test function of the overhead to be tested; based on the basic overhead, the first overhead and the second overhead, obtaining a performance test overhead.

2. The method of acquiring performance test overhead of claim 1, wherein, The method comprises the following steps: selecting a kernel test function meeting a preset requirement from a kernel module of a target operating system; and / or constructing a self-editing test function according to the preset requirement.

3. The method of acquiring performance test overhead of claim 2, wherein, The method comprises the following steps: filtering out functions containing input / output, scheduling and / or lock contention from the kernel module of the target operating system to obtain the kernel test function.

4. The method of claim 1, wherein, The switch control function has a trigger mechanism, and the trigger mechanism comprises loop calling, fixed-time calling and timing calling.

5. The method of claim 1, wherein, The test function comprises the kernel test function and the self-editing test function, and the method comprises the following steps: editing a basic overhead obtaining function, and setting synchronous calling of a ktime_get_ns() function and a clock_gettime() function in the basic overhead obtaining function; calling the ktime_get_ns() function in the kernel module to obtain a kernel basic overhead when the kernel test function is running; synchronously calling the clock_gettime() function to obtain a user basic overhead when the self-editing test function is running; calculating a sum of the kernel basic overhead and the user basic overhead to obtain the basic overhead when the test function is running.

6. The method of acquiring performance test overhead according to any one of claims 1 to 5, characterized in that, The method comprises the following steps: embedding a first plug-in function and a second plug-in function corresponding to the performance tool to be tested in the test function, wherein the first plug-in function is used for representing data collection by using the performance tool to be tested, and the second plug-in function is used for representing data collection and data analysis and processing by using the performance tool to be tested.

7. The method of obtaining performance test overhead according to any one of claims 1 to 5, characterized in that, The performance test overhead comprises plug-in overhead and load overhead, and the method comprises the following steps: calculating a difference between the first overhead and the basic overhead to obtain the plug-in overhead.

8. The method of acquiring performance test overhead according to any one of claims 1 to 5, characterized in that, ​ ​ calculating a difference between the second overhead and the first overhead to obtain the load overhead; calculating a sum of the instrumentation overhead and the load overhead to obtain the performance test overhead.

9. The method of obtaining performance test overhead according to any one of claims 1 to 5, characterized in that, The performance test overhead includes an instrumentation overhead and a load overhead, and the performance test overhead is obtained based on the base overhead, the first overhead and the second overhead, including: calculating a difference between the first overhead and the base overhead to obtain the instrumentation overhead, obtaining a product of the instrumentation overhead and a pre-set instrumentation weight to obtain a first product; calculating a difference between the second overhead and the first overhead to obtain the load overhead, obtaining a product of the load overhead and a pre-set load weight to obtain a second product, and a sum of the instrumentation overhead and the load weight being 1; calculating a sum of the first product and the second product to obtain the performance test overhead.

10. An apparatus for obtaining performance test overhead, the apparatus comprising: The apparatus for obtaining the performance test overhead includes: a test function setting module configured to obtain a test function for the performance test overhead and set a switch control function for the test function; a base overhead obtaining module configured to run the test function based on the switch control function and obtain a base overhead of the test function; an instrumentation module configured to embed, for each performance tool to be tested, an instrumentation function corresponding to the performance tool to be tested into the test function to obtain a test function of a to-be-tested overhead corresponding to the performance tool to be tested; an instrumentation triggering module configured to execute the test function of the to-be-tested overhead based on the switch control function and obtain a first overhead of the instrumentation function in the test function of the to-be-tested overhead under no load and a second overhead of the instrumentation function in the test function of the to-be-tested overhead under load; a test overhead obtaining module configured to obtain the performance test overhead based on the base overhead, the first overhead and the second overhead.

11. A storage medium, characterized by A program or instruction is stored on a storage medium, and the program or instruction is run by a processor to implement the steps of the method for obtaining the performance test overhead according to any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method for obtaining the performance test overhead according to any one of claims 1 to 9 when the processor executes the program.

Images