Performance analysis method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a performance analysis method and device, electronic equipment and a storage medium, which are used for real-time analysis of internal running states of a test object and provide a performance test method with smaller granularity. The method comprises the following steps: determining a to-be-tested object related to a test requirement; sampling events generated by the to-be-tested object during running at a set period to obtain sampling data, wherein the sampling data is used for representing running conditions of the to-be-tested object at each sampling moment; and performing performance analysis on the sampling data to determine a performance analysis result of the to-be-tested object.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a performance analysis method, apparatus, electronic device, and storage medium. Background Technology

[0002] Traditional benchmark services can only measure an application's average response time and average throughput in a test scenario. If the test scenario does not meet the predetermined requirements, performance analysis or the creation of more granular test scenarios is necessary. Meanwhile, general performance analysis tools perform performance statistics based on the application itself, but are unaware of the application's context and require manual configuration by the user.

[0003] Therefore, current performance testing methods based on Benchmark services can only analyze the overall application situation under the entire preset application scenario, but they are unaware of what happens inside the application scenario and lack effective solutions. Summary of the Invention

[0004] This application provides a performance analysis method, apparatus, electronic device, and storage medium for real-time analysis of the internal operating state of a test object, providing a more granular performance testing method.

[0005] Firstly, this application provides a performance analysis method, including:

[0006] Identify the objects to be tested that are relevant to the testing requirements;

[0007] According to a set period, the events generated by the object under test during operation are sampled to obtain sampled data, wherein the sampled data is used to characterize the operation of the object under test at each sampling time.

[0008] The sampled data is subjected to performance analysis to determine the performance analysis results of the object under test.

[0009] In one possible embodiment, sampling events generated by the object under test during operation to obtain sampled data includes:

[0010] Based on the events generated by the object under test during operation, determine the event data corresponding to the events, wherein the event data includes snapshot data and / or telemetry data;

[0011] The event data is sampled to obtain the sampled data corresponding to the object under test.

[0012] In one possible embodiment, sampling events generated by the object under test during operation to obtain sampled data includes:

[0013] Based on at least one of the tick interrupt function and performance analysis tools, the events generated by the object under test during operation are sampled to obtain the sampled data.

[0014] In one possible embodiment, the step of performing performance analysis on the sampled data to determine the performance analysis result of the object under test includes:

[0015] The Benchmark service is invoked to perform performance analysis on the sampled data, and the performance analysis results of the object under test are determined.

[0016] In one possible embodiment, the step of invoking the Benchmark service to perform performance analysis on the sampled data and determine the performance analysis results of the object under test includes:

[0017] After determining the test requirements, the BenchMark service is invoked to obtain the initialization parameters related to the test requirements;

[0018] The test range is determined based on the initialization parameters;

[0019] The Benchmark service is used to obtain the sampling data corresponding to the test range, and the sampling data is used to perform performance analysis to determine the performance analysis results of the object under test.

[0020] In one possible embodiment, the sampling data includes any one or more of the following:

[0021] The function being executed;

[0022] Function calling relationships and called relationships;

[0023] Functions whose execution count exceeds a threshold;

[0024] Function runtime information;

[0025] Memory information allocated for objects created by the function.

[0026] In one possible embodiment, the event includes any one or more of the following:

[0027] Hardware events, wherein the hardware events characterize the hardware usage of the object under test;

[0028] Software events, wherein the software events characterize events generated by the object under test;

[0029] A tracepoint event, wherein the tracepoint event characterizes an event triggered by a static tracepoint of the object under test.

[0030] Secondly, this application provides a performance analysis device, comprising:

[0031] The determination module is used to identify the objects to be tested that are relevant to the testing requirements;

[0032] The sampling module is used to sample the events generated by the object under test during operation at a set period to obtain sampling data, wherein the sampling data is used to characterize the operation of the object under test at each sampling time.

[0033] The analysis module is used to perform performance analysis on the sampled data and determine the performance analysis results of the object under test.

[0034] In one possible embodiment, the sampling module is specifically used for:

[0035] Based on the events generated by the object under test during operation, determine the event data corresponding to the events, wherein the event data includes snapshot data and / or telemetry data;

[0036] The event data is sampled to obtain the sampled data corresponding to the object under test.

[0037] In one possible embodiment, the sampling module is specifically used for:

[0038] Based on at least one of the tick interrupt function and performance analysis tools, the events generated by the object under test during operation are sampled to obtain the sampled data.

[0039] In one possible embodiment, the analysis module is specifically used for:

[0040] The Benchmark service is invoked to perform performance analysis on the sampled data, and the performance analysis results of the object under test are determined.

[0041] In one possible embodiment, the analysis module is specifically used for:

[0042] After determining the test requirements, the BenchMark service is invoked to obtain the initialization parameters related to the test requirements;

[0043] The test range is determined based on the initialization parameters;

[0044] The Benchmark service is used to obtain the sampling data corresponding to the test range, and the sampling data is used to perform performance analysis to determine the performance analysis results of the object under test.

[0045] In one possible embodiment, the sampling data includes any one or more of the following:

[0046] The function being executed;

[0047] Function calling relationships and called relationships;

[0048] Functions whose execution count exceeds a threshold;

[0049] Function runtime information;

[0050] Memory information allocated for objects created by the function.

[0051] In one possible embodiment, the event includes any one or more of the following:

[0052] Hardware events, wherein the hardware events characterize the hardware usage of the object under test;

[0053] Software events, wherein the software events characterize events generated by the object under test;

[0054] A tracepoint event, wherein the tracepoint event characterizes an event triggered by a static tracepoint of the object under test.

[0055] Thirdly, this application provides an electronic device, comprising:

[0056] Memory, used to store program instructions;

[0057] A processor is configured to invoke program instructions stored in the memory and execute the steps of the method described in any one of the first aspects according to the obtained program instructions.

[0058] Fourthly, this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a computer, cause the computer to perform the method described in any one of the first aspects.

[0059] Fifthly, this application provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of the first aspects.

[0060] The performance analysis method provided in this application has at least the following beneficial effects:

[0061] After identifying the object under test (DUT) related to the testing requirements, this application periodically samples the events generated by the DUT during its operation. Since the DUT generates at least one event during its operation, and the generated events characterize the operation of the DUT during its operation, performance analysis of various operating conditions during operation can reveal the state of the DUT during operation, providing a more granular performance analysis method.

[0062] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 A flowchart illustrating the implementation of a performance analysis method provided in this application embodiment;

[0065] Figure 2 A flowchart illustrating the implementation of a performance analysis method based on BenchMark service, provided in this application embodiment;

[0066] Figure 3 A flowchart illustrating an example performance analysis method provided in this application embodiment;

[0067] Figure 4 A schematic diagram of a performance analysis device provided in an embodiment of this application;

[0068] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0070] The term "comprising" and any variations thereof in this application are intended to cover non-exclusive protection. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The term "multiple" in this application may mean at least two, for example, two, three, or more, and the embodiments of this application do not impose any limitation.

[0071] The data collection, dissemination, and use in this application all comply with relevant national laws and regulations.

[0072] The application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0073] Before introducing the performance analysis method provided in the embodiments of this application, for ease of understanding, the technical background of the embodiments of this application will be described in detail below.

[0074] Traditional benchmark services can only calculate average response time and average throughput in test scenarios. If the test scenario does not meet the predetermined requirements, performance analysis or the creation of smaller-granular test scenarios is required. Meanwhile, general performance analysis tools such as perf (Linux performance analysis tool), TPTP (Test and Performance Tools Platform), CodePro Profiler, YourKit Profiler, JProfiler, valgrind, and performance profilers are all based on the application using PMU (performance monitor unit), tracepoints, kernel counters, or JVM (Java Virtual Machine) for performance statistics, but they are unaware of the application scenario and require manual configuration by the user.

[0075] For example, given an application capable of order placement and trading, the current requirement is to analyze its performance from order placement to delivery to the exchange. This involves analyzing the performance of order processing within the internal trading system. The order placement and delivery process can be assumed to have four steps: placement, recording, conversion, and delivery. The current testing approach involves calling the Benchmark service when the order is placed (placed). The Benchmark service records the sending time of the test message. Then, when the order is ready to be sent to the exchange (delivered), a feedback message is sent to the Benchmark service. Based on this, the Benchmark service performs statistical analysis of various parameters to obtain the system performance test results. However, if the performance test results are unsatisfactory, the Benchmark report cannot specify the exact location of the problem (e.g., whether it's a recording or conversion issue) from order placement to delivery. It only provides a general timeframe, such as the average time taken. If further analysis of the recording and conversion steps is needed, the Benchmark service must be called again for re-analysis, making the process cumbersome.

[0076] Therefore, current performance testing methods based on Benchmark services can only analyze the overall application situation under the entire preset application scenario, but they are unaware of what happens inside the application scenario and lack effective solutions.

[0077] In view of this, embodiments of this application provide a performance analysis method for periodically sampling events generated by the test object during its operation after determining the test object related to the test requirements. Since the test object generates at least one event during its operation, and the generated events characterize the operation status of the test object during its operation, by performing performance analysis on various operation statuses during operation, the state of the test object during operation can be known, thus providing a more granular performance analysis method.

[0078] It should be noted that the objects to be tested in this application include, but are not limited to, at least one of applications and kernels.

[0079] The performance analysis method of this application embodiment will be described below with reference to the accompanying drawings and specific embodiments.

[0080] Reference Figure 1 As shown in the embodiment of this application, a performance analysis method can be executed by a server, specifically including the following steps S100-S102:

[0081] Step S100: Identify the objects to be tested that are relevant to the testing requirements;

[0082] In some embodiments, the object under test in this embodiment includes, but is not limited to, applications and / or kernels.

[0083] In practice, this application can perform performance analysis on the object under test during operation based on testing requirements, and can customize test scenarios according to user needs, thereby performing performance analysis on the object under test in test scenarios corresponding to the testing requirements, and realizing customized performance analysis.

[0084] Step S101: Sample the events generated by the object under test during operation according to a set period to obtain sampled data, wherein the sampled data is used to characterize the operation of the object under test at each sampling time.

[0085] Optionally, the sampling data in this embodiment is obtained by sampling the events generated by the object under test during its operation at each sampling time according to a set period. It can be represented by a tree structure, where each node in the tree structure represents the sampling data used by the application, such as the function corresponding to the application at the sampling point, and lists the running time information of each node, including running time or running time percentage. It can also list the number of objects generated by the function corresponding to the node, as well as the memory size allocated to the generated objects.

[0086] In practice, this application can sample events generated by the object under test during operation, analyze the performance during operation based on the sampled data, reduce the granularity of current performance analysis of applications, perceive the overall operation of the application during operation, as well as various details during operation, and thus make an accurate judgment on performance symptoms.

[0087] Optionally, this embodiment samples the events generated by the object under test during its operation in the following manner to obtain the sampled data corresponding to the object under test:

[0088] Based on the events generated by the object under test during operation, determine the event data corresponding to the events, wherein the event data includes snapshot data and / or telemetry data; sample the event data to obtain the sampled data corresponding to the object under test.

[0089] In this embodiment, the snapshot data is obtained by statistically analyzing the operation of the object under test based on snapshot technology. The snapshot data includes the application's execution information over a period of time. Using snapshot technology, after the application starts, the profiler tool begins to collect various execution data of the application. Some of this data can be directly displayed in the telemetry view, while most of the data is stored internally until the user requests a snapshot, at which point the statistical information based on this stored data is displayed.

[0090] The telemetry data in this embodiment is obtained by analyzing the operation of the object under test based on telemetry technology. Telemetry is a method used to view the operation of an application. Typically, multiple views are displayed in real time to show CPU usage, memory usage, thread status, and other useful information, so that users can quickly identify the key to the problem.

[0091] Optionally, the events generated by the object under test during runtime in this embodiment can be statistically analyzed in the following way:

[0092] Utilize at least one of the PMU (performance monitor unit), tracepoint, and kernel counters to count the events generated by the object under test during runtime.

[0093] Optionally, the events generated by the object under test during runtime in this embodiment include, but are not limited to, any one or more of the following:

[0094] The first type is hardware events, which characterize the hardware usage of the object under test.

[0095] In practice, hardware events can include events generated by the PMU, such as cache-misse, CPU-cycles, cache-references, cache-misses, task-clock-msecs, and IPC. These are typically used when analyzing application hardware usage. A cache-misse indicates that the data the CPU needs to access is cached, called a "hit," while a miss indicates a "miss." When the CPU misses data, it typically accesses hardware, triggering a hardware event. CPU-cycles represent the CPU's frequency and cycles, usually in GHz. Cache-references indicate the number of cache hits; cache-misses indicate the number of cache misses.

[0096] Cycles refers to the processor clock cycle; a single machine instruction may require multiple cycles.

[0097] Task-clock-msecs represents CPU utilization. A high value indicates that the application spends most of its time on CPU computation rather than I / O.

[0098] IPC is the ratio of Instructions to Cycles. A higher IPC value indicates that the application is making full use of the processor's capabilities. Instructions represent the number of machine instructions.

[0099] The second type is a software event, where the software event represents an event generated by the object under test.

[0100] In practice, software events include events generated by the core programs of the application, such as context-switches, page-faults, CPU clock, and CPU migrations.

[0101] Context switching can be understood as process switching or task switching in an application. The CPU switches from one thread to another. While the operating system can run multiple processes simultaneously, a CPU can only execute one task at a time. The operating system uses time-slicing to make it appear to the user that these tasks are running concurrently. The CPU serves each task for a certain period, then saves the current task's state, loads the state of the next task, and continues serving the next task. This process of saving and reloading task states is called context switching.

[0102] A page fault occurs when a process accesses a page in the virtual address space, but this page is not yet in physical memory. In this case, the CPU cannot execute the process, and Linux generates a hard page fault interrupt. The system needs to read the corresponding data page from a slow device (such as a disk) into physical memory and establish a mapping between the physical memory address and the virtual address space page. Only then can the process access this part of the virtual address space. This process is called a page fault. Page faults are further divided into several types: major page fault, minor page fault, invalid page fault, and segment fault.

[0103] cpu-migrations represent the number of CPU migrations that occurred during the execution of a process t1, that is, the number of times the scheduler transferred the process from one CPU to another for execution.

[0104] The third type is the tracepoint event, where the tracepoint event represents the event triggered by the static tracepoint of the object under test.

[0105] In practice, a tracepoint event is an event triggered by a static tracepoint in the core program of the application. These tracepoints are used to determine the behavior details of the core program during the execution of the application, such as the number of times the slab memory allocator is configured.

[0106] In this context, a tracepoint is a pre-inserted instrument at the insertion point of a function in the application. When the execution reaches the insertion point of the function, the instrumented function is executed, which in turn triggers the probe function pre-bound to the insertion point. There can be one or more probe functions, and the probe functions can be defined with arbitrary behavior, thereby enabling the observation of the function's internal workings.

[0107] In practice, any of the above events can generate corresponding event data, such as snapshot data, which can then be sampled to obtain sampled data.

[0108] In some embodiments, the sampling methods include any one or more of the following:

[0109] (1) Tick interrupt function;

[0110] (2) Performance analysis tools; for example, perf, Oprofile, etc. Performance analysis tools can be embedded in the Benchmark service.

[0111] Based on at least one of the tick interrupt function and performance analysis tools, the events generated by the object under test during its operation are sampled to obtain the sampled data corresponding to the object under test.

[0112] Specifically, the above sampling method is used to sample events of the object under test. For example, sampling can be performed based on tick interrupts, that is, sampling points are triggered within tick interrupts, and the context of the object under test at that time is determined within the sampling points. For example, if the previous application (transaction service) spent 90% of its time on the function translate(), then 90% of the sampling points should fall within the context of the function translate().

[0113] In some embodiments, the sampling data includes any one or more of the following:

[0114] a) The function being executed;

[0115] b) Function calling relationships and called relationships;

[0116] c) Functions whose execution count exceeds the threshold;

[0117] d) Function runtime information;

[0118] e) Memory information allocated for objects generated by the function.

[0119] Step S102: Perform performance analysis on the sampled data to determine the performance analysis results of the object under test.

[0120] In some embodiments, the BenchMark service is invoked to perform performance analysis on the sampled data and determine the performance analysis results of the object under test.

[0121] Optionally, after determining the test requirements, the Benchmark service is invoked to obtain initialization parameters related to the test requirements. These initialization parameters include information related to the test requirements, such as test scenario information (e.g., the environment in which the test object runs, the trading platform on which the test object runs), test object information (e.g., user group characteristic information), the test scope of the test scenario (e.g., the start and end times of the test), and performance analysis metrics (e.g., throughput, maximum rate, minimum rate, performance bottlenecks, etc.). In this embodiment, the test scope can represent the start and end of the test, or it can represent limitations imposed on the test scenario by the test platform, test object, etc. This embodiment does not impose excessive limitations on this.

[0122] Optionally, the test scope of the test scenario in this embodiment can be represented by a structure, thereby obtaining the running status of the object under test within the test scope through the structure; wherein, the structure can be sampled and sent to the BenchMark service to generate a performance analysis report.

[0123] The test range is determined based on the initialization parameters; the sampled data corresponding to the test range is obtained using the Benchmark service; the performance of the sampled data is analyzed to determine the performance analysis result of the object under test.

[0124] During implementation, when a test request message carrying test requirements is received, the Benchmark service is invoked to obtain initialization parameters related to the test requirements. Based on the initialization parameters, the Benchmark service is initialized to obtain an initial Benchmark service. At this time, the Benchmark service records the start time of the test. At the end time, the sampled data collected from the start time to the end time is sent to the initial Benchmark service for statistical analysis of various data to obtain the performance analysis results of the test object. The end time is determined based on the test scope of the test scenario included in the initialization parameters.

[0125] Reference Figure 2 As shown in the figure, this application also provides a performance analysis method based on Benchmark service, and the specific implementation process is as follows:

[0126] Step S200: Obtain test requirements;

[0127] Step S201: Call the BenchMark service to obtain the initialization parameters related to the test requirements;

[0128] Step S202: The Benchmark service determines the test range based on the initialization parameters;

[0129] The BenchMark service determines the start and end times of the test based on the initialization parameters.

[0130] Step S203: Within the test range, the events generated by the object under test during operation are sampled according to a set period to obtain the sampled data corresponding to the object under test;

[0131] The events include hardware events, which characterize the usage of hardware by the object under test; software events, which characterize events generated by the object under test; and tracepoint events, which characterize events triggered by static tracepoints of the object under test.

[0132] The sampled data includes running functions; function call relationships and called-by-function relationships; functions whose execution counts exceed the threshold; function execution time information; and memory allocation information for objects generated by functions.

[0133] Specifically, event data corresponding to the events generated by the object under test during its operation is determined; the event data is sampled according to at least one of the tick interrupt function and performance analysis tools to obtain sampled data corresponding to the object under test.

[0134] Step S204: Use the BenchMark service to perform performance analysis on the sampled data corresponding to the test range, and determine the performance analysis results of the object under test.

[0135] Reference Figure 3 As shown, this application embodiment takes the performance analysis of an application from internal order placement to order delivery to the exchange as an example. It assumes that the process from internal order placement to order delivery includes four steps: order task assignment, recording, conversion, and order delivery. The performance analysis method provided in this application will be described in detail below:

[0136] Step S300: When the order task is issued, the Benchmark service is called to obtain the initialization parameters related to the test requirements. At this time, the Benchmark service records the test start time.

[0137] Step S301: The Benchmark service determines the test end time based on the initialization parameters;

[0138] Step S302: During the period from the start time of the test to the end time of the test, the events generated by the application during its operation are sampled according to a set period to obtain sampled data;

[0139] Step S303: When the order is shipped, perform performance analysis on the sampled data collected during this period through the Benchmark service.

[0140] The above process can not only perform performance analysis on each step of the application process from internal order placement to order delivery to the exchange, but also perform overall analysis, thus improving the efficiency of performance analysis.

[0141] This embodiment uses snapshot data or telemetry data of CPU and memory obtained through periodic sampling in Benchmark analysis to obtain the function call relationships, function call relationships, and hotspot list (including functions whose execution counts exceed a threshold) of the object under test during runtime. All collected sampled data is represented in a tree structure, where nodes in the tree structure represent sampled data, and nodes are filtered based on initialization parameters related to the test requirements. For each node in the tree, the corresponding runtime or percentage of runtime, the number of objects generated by that node, and the memory size allocated to these objects are listed. Optionally, the Benchmark service can embed performance analysis tools such as perf and OProfile, or directly perform periodic sampling processing on data such as CPU, hard disk, and memory. Optionally, the perf or OProfile functionality can be ported to the Benchmark service to reduce interactions between services and increase analysis speed.

[0142] Specifically, applications can utilize PMU, tracepoints, and kernel counters for performance statistics. This allows for analysis not only of application performance issues (per thread) but also of kernel performance issues. By combining the Benchmark service receiving the test start request message, recording initialization parameters, and subsequently recording the tracepoint generation time, when the Benchmark receives the test end request message, analysis of the reports from the start to the end of the test can reveal various details of the application or kernel during that period, enabling accurate diagnosis of performance symptoms. Furthermore, by combining the initialization parameters, analytical metrics for the current scenario can be calculated, such as throughput, maximum and minimum rates, performance bottlenecks, and other comprehensive performance indicators.

[0143] In some embodiments, the data transmission in this embodiment (including initialization parameters, sampling data, analysis indicators, analysis reports, etc.) is based on the RPC (Remote Procedure Call) protocol. The RPC protocol is built upon a transport protocol, such as TCP (Transmission Control Protocol) or UDP (User Datagram Protocol), to carry information data between communicating programs. Optionally, the RPC protocol in this embodiment is based on TCP binary stream data transmission, and the transmission uses BSON (Binary Serialized Document Format) for packing and unpacking.

[0144] It should be noted that this embodiment samples the events generated by the object under test during its operation to obtain sampled data. The specific situation of the object under test during operation is determined by performing performance analysis on the collected sampled data. Let's assume x1, x2, ..., x... k , ..., x n These are sampled data from the same application during runtime, and each has a different expected value (Ex). k And variance Dx k ;

[0145] If there exists a constant C such that Dx k <= C (where k = 1, 2, ..., n), that is, the variance is bounded. Then, according to Chebyshev's law of large numbers, for any small positive number ε, the following formula (1) is satisfied:

[0146]

[0147] Where n represents the number of samples, Dx k Indicates the sampled data x k The variance, Ex k Indicates the sampled data x k The expectation.

[0148] This demonstrates that as long as the sampling frequency is high enough and the sampling time is long enough, the above inference indicates that the average value of the sampled data is close to the average value of all data in the application. Therefore, the performance analysis method in this embodiment can effectively and accurately analyze the performance of the object under test during operation.

[0149] To illustrate the case more fully, we will use some simple examples. Note that in actual use, the methods will be more complex. Based on an Intel 64-bit CPU, the documentation shows that the following events can be monitored, as shown in Table 1:

[0150] Table 1

[0151] General performance counter Event Name UMask Event Selection 0 instruction retire 00H 3CH 1 unhalted core cycles 00H C0H 2 unhalted reference cycles 01H 3CH 3 LLC reference 4FH 2EH 4 LLC misses 41H 2EH 5 branch instruction retired 00H C4H 6 branch misses retired 00H C5H 7 topdown slots 01H A4H

[0152] Then, general performance counters 0 and 1 can be configured via shell scripts. After configuration, the general performance counters will record instruction data and CPU cycles respectively. Of course, in addition to the eight events in Table 1, more events can be selected, but this embodiment does not impose too many limitations on this. When the above general performance counters are enabled, performance analysis can be performed and initialization parameters such as scene parameters can be recorded in the design code corresponding to the method in this embodiment.

[0153] First, a Benchmark class is designed. The Benchmark class is mainly responsible for recording data, generating reports, and sending data. During the initialization process, the CPU needs to be configured. In this embodiment, the time can also be configured. Additional scenario parameters are configured through a parameter list. After the test is completed, a report is generated and data is sent, so the running time and instruction data of the scenario can be obtained. More internal running information of the program can also be obtained through other events.

[0154] Optionally, this embodiment can also design a test order application, orderApp, to analyze the application's performance from internal order placement to order delivery to the exchange. For example, assume the process from internal order placement to order delivery includes four steps: order task assignment, recording, conversion, and order delivery. In implementation, only Benchmark recording needs to be configured in the order placement and delivery steps. During actual operation, a general performance counter can be used to fully understand the entire operation process of orderApp.

[0155] This embodiment obtains sampled data by sampling events generated by the test object during its operation. Based on the initialization parameters related to the test requirements and the method of Benchmark service to perform performance analysis on the sampled data, it can obtain the real-time running status of the program and the overall running test status of the program, which greatly improves the ability to perceive the whole field of certain particularly important scenarios.

[0156] Based on the same inventive concept, embodiments of this application provide a performance analysis device, please refer to... Figure 4 The device includes:

[0157] Module 400 is used to identify the objects to be tested related to the test requirements;

[0158] The sampling module 401 is used to sample the events generated by the object under test during operation at a set period to obtain sampling data, wherein the sampling data is used to characterize the operation of the object under test at each sampling time.

[0159] The analysis module 402 is used to perform performance analysis on the sampled data and determine the performance analysis results of the object under test.

[0160] In one possible embodiment, the sampling module 401 is specifically used for:

[0161] Based on the events generated by the object under test during operation, determine the event data corresponding to the events, wherein the event data includes snapshot data and / or telemetry data;

[0162] The event data is sampled to obtain the sampled data corresponding to the object under test.

[0163] In one possible embodiment, the sampling module 401 is specifically used for:

[0164] Based on at least one of the tick interrupt function and performance analysis tools, the events generated by the object under test during operation are sampled to obtain the sampled data.

[0165] In one possible embodiment, the analysis module 402 is specifically used for:

[0166] The Benchmark service is invoked to perform performance analysis on the sampled data, and the performance analysis results of the object under test are determined.

[0167] In one possible embodiment, the analysis module 402 is specifically used for:

[0168] After determining the test requirements, the BenchMark service is invoked to obtain the initialization parameters related to the test requirements;

[0169] The test range is determined based on the initialization parameters;

[0170] The Benchmark service is used to obtain the sampling data corresponding to the test range, and the sampling data is used to perform performance analysis to determine the performance analysis results of the object under test.

[0171] In one possible embodiment, the sampling data includes any one or more of the following:

[0172] The function being executed;

[0173] Function calling relationships and called relationships;

[0174] Functions whose execution count exceeds a threshold;

[0175] Function runtime information;

[0176] Memory information allocated for objects created by the function.

[0177] In one possible embodiment, the event includes any one or more of the following:

[0178] Hardware events, wherein the hardware events characterize the hardware usage of the object under test;

[0179] Software events, wherein the software events characterize events generated by the object under test;

[0180] A tracepoint event, wherein the tracepoint event characterizes an event triggered by a static tracepoint of the object under test.

[0181] Based on the same inventive concept, this application provides an electronic device that can realize the function of the performance analysis device described above. Please refer to... Figure 5 The device includes a processor 501 and a memory 502, the memory 502 being used to store program instructions; the processor 501 is used to call the program instructions stored in the memory 502 and execute the following steps according to the obtained program instructions:

[0182] Identify the objects to be tested that are relevant to the testing requirements;

[0183] According to a set period, the events generated by the object under test during operation are sampled to obtain sampled data, wherein the sampled data is used to characterize the operation of the object under test at each sampling time.

[0184] The sampled data is subjected to performance analysis to determine the performance analysis results of the object under test.

[0185] In one possible embodiment, the processor 501 is configured to perform:

[0186] Based on the events generated by the object under test during operation, determine the event data corresponding to the events, wherein the event data includes snapshot data and / or telemetry data;

[0187] The event data is sampled to obtain the sampled data corresponding to the object under test.

[0188] In one possible embodiment, the processor 501 is configured to perform:

[0189] Based on at least one of the tick interrupt function and performance analysis tools, the events generated by the object under test during operation are sampled to obtain the sampled data.

[0190] In one possible embodiment, the processor 501 is configured to perform:

[0191] The Benchmark service is invoked to perform performance analysis on the sampled data, and the performance analysis results of the object under test are determined.

[0192] In one possible embodiment, the processor 501 is configured to perform:

[0193] After determining the test requirements, the BenchMark service is invoked to obtain the initialization parameters related to the test requirements;

[0194] The test range is determined based on the initialization parameters;

[0195] The Benchmark service is used to obtain the sampling data corresponding to the test range, and the sampling data is used to perform performance analysis to determine the performance analysis results of the object under test.

[0196] In one possible embodiment, the sampling data includes any one or more of the following:

[0197] The function being executed;

[0198] Function calling relationships and called relationships;

[0199] Functions whose execution count exceeds a threshold;

[0200] Function runtime information;

[0201] Memory information allocated for objects created by the function.

[0202] In one possible embodiment, the event includes any one or more of the following:

[0203] Hardware events, wherein the hardware events characterize the hardware usage of the object under test;

[0204] Software events, wherein the software events characterize events generated by the object under test;

[0205] A tracepoint event, wherein the tracepoint event characterizes an event triggered by a static tracepoint of the object under test.

[0206] This application embodiment does not limit the specific connection medium between the memory 502 and the processor 501. For example, the memory 502 and the processor 501 are connected via a bus, which can be divided into an address bus, a data bus, a control bus, etc.

[0207] The memory 502 may include read-only memory (ROM) and random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0208] The processor 501 mentioned above can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processing (DSP), an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc.

[0209] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium. The computer program product includes computer program code, which, when executed on a computer, causes the computer to perform any of the performance analysis methods discussed above. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to that of the performance analysis methods, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method; repeated details will not be elaborated further.

[0210] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0211] Based on the same inventive concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the performance analysis methods discussed above. Since the principle by which the above computer program product solves the problem is similar to that of the performance analysis method, the implementation of the above computer program product can be referred to the implementation of the method, and repeated details will not be described again.

[0212] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0213] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0214] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0215] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0216] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0217] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A performance analysis method characterized by, The method comprises the following steps: determining a to-be-tested object related to a test requirement; sampling events generated by the to-be-tested object during running at a set period to obtain sampling data, wherein the sampling data is used to represent running conditions of the to-be-tested object at each sampling time point; performing performance analysis on the sampling data to determine a performance analysis result of the to-be-tested object, comprising: calling a BenchMark service to perform performance analysis on the sampling data to determine the performance analysis result of the to-be-tested object, wherein, after the test requirement is determined, the BenchMark service is called to obtain initialization parameters related to the test requirement; a test range is determined according to the initialization parameters; and the sampling data corresponding to the test range is obtained by using the BenchMark service; performing performance analysis on the sampling data to determine a performance analysis result of the to-be-tested object.

2. The method of claim 1, wherein, The sampling of the events generated by the to-be-tested object during running to obtain the sampling data comprises the following steps: determining event data corresponding to the events generated by the to-be-tested object during running, wherein the event data comprises snapshot data and / or telemetry data; sampling the event data to obtain the sampling data corresponding to the to-be-tested object.

3. The method of claim 1, wherein, The sampling of the events generated by the to-be-tested object during running to obtain the sampling data comprises the following steps: sampling the events generated by the to-be-tested object during running according to at least one of a tick interrupt function and a performance analysis tool to obtain the sampling data.

4. The method according to any one of claims 1 to 3, characterized in that, The sampling data comprises any one or any multiple of the following: a running function; a calling relationship and a called relationship of a function; a function whose execution times exceed a threshold value; running time information of a function; memory information allocated for an object generated for a function.

5. The method according to any one of claims 1 to 3, characterized in that, The events comprise any one or any multiple of the following: a hardware event, wherein the hardware event represents a use condition of the to-be-tested object on hardware; a software event, wherein the software event represents an event generated by the to-be-tested object; a tracepoint event, wherein the tracepoint event represents an event triggered by a static tracepoint of the to-be-tested object.

6. A performance analysis device, characterized by The method comprises the following steps: a determining module is configured to determine a to-be-tested object related to a test requirement; a sampling module is configured to sample events generated by the to-be-tested object during running at a set period to obtain sampling data, wherein the sampling data is used to represent running conditions of the to-be-tested object at each sampling time point; performance analysis is performed on the sampling data to determine a performance analysis result of the to-be-tested object, comprising: calling a BenchMark service to perform performance analysis on the sampling data to determine the performance analysis result of the to-be-tested object, wherein, after the test requirement is determined, the BenchMark service is called to obtain initialization parameters related to the test requirement; a test range is determined according to the initialization parameters; and the sampling data corresponding to the test range is obtained by using the BenchMark service; An analysis module is configured to analyze the sampling data to determine a performance analysis result of the object under test.

7. An electronic device, comprising: The method comprises the steps of: a memory configured to store program instructions; a processor configured to invoke the program instructions stored in the memory and execute the steps of the method according to the obtained program instructions.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program comprises program instructions, which, when executed by a computer, cause the computer to execute the method according to any one of claims 1-5.

9. A computer program product, characterised in that, The computer program product comprises computer program code, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1-5.

Images