Performance test method and device, equipment and storage medium

Abstract

The invention provides a performance testing method and device and a storage medium. The method comprises the steps that all executable programs of a software system submitted by a user and modified program files in the software system are received; based on the program file and all executable programs of the software system, performing a performance test to obtain a test report; and sending the test report to the user. According to the application, a user only needs to send all the executable programs of the software system and the currently modified program file to the server. And the server automatically completes the performance test of the software system and returns a test report to the user. Automatic testing of the whole process of the software system is achieved, the automation degree is high, and the testing efficiency is improved. And the test is performed only when the modified program file appears, so that the incremental performance test is realized, the performance test is more targeted, and the test efficiency and accuracy are improved. And tests in the aspects of off-line analysis, on-line analysis, integration test and the like can be carried out, and the performance test is more comprehensive, efficient and accurate.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of software development, and particularly relates to a performance testing method and device, equipment and a storage medium. BACKGROUND

[0002] Performance control of a software system is a very tricky and important thing, especially in a scenario such as automatic driving that has a high requirement for software system performance. Therefore, the performance of the software system needs to be evaluated.

[0003] In the related art, system performance testing of a large-scale software mostly still stays in the stage of manual testing by a developer, and has a low degree of automation, low efficiency and low accuracy of testing. SUMMARY

[0004] The application provides a performance testing method, device, equipment and storage medium, which can alleviate the technical problem of low degree of automation, low efficiency and low accuracy of software system performance testing in the related art.

[0005] The first aspect of the application provides a performance testing method, comprising:

[0006] receiving all executable programs of a software system submitted by a user and a modified program file in the software system;

[0007] performing performance testing based on the program file and all executable programs of the software system to obtain a test report;

[0008] sending the test report to the user.

[0009] In some embodiments of the application, the performance testing based on the program file and all executable programs of the software system comprises:

[0010] performing offline analysis based on the program file and all executable programs of the software system, wherein the offline analysis is used to analyze code problems without running programs of the software system; and / or,

[0011] performing online analysis based on the program file and all executable programs of the software system, wherein the online analysis is used to test performance with running programs of the software system; and / or,

[0012] performing integration testing based on the program file and all executable programs of the software system.

[0013] In some embodiments of the application, the offline analysis based on the program file and all executable programs of the software system comprises:

[0014] detecting whether there is a code writing error in the program file based on syntax rules of a programming language used by the software system; and / or

[0015] detecting, by a preset memory analysis tool, whether there is a memory exception problem in all executable programs of the software system after the program file is modified.

[0016] In some embodiments of the present application, the online analysis based on the program file and all executable programs of the software system comprises:

[0017] obtaining an old system version and a new system version of the software system before and after modification and a test tool used in the performance test;

[0018] determining program objects of the software system affected by the program file before and after modification based on the program file, the old system version, the new system version and the test tool;

[0019] performing performance test on the program objects by using the test tool.

[0020] In some embodiments of the present application, the determination of the program objects of the software system affected by the program file before and after modification based on the program file, the old system version, the new system version and the test tool comprises:

[0021] if the minimum program scale targeted by the test tool is a system-level program, all programs of the software system under the old system version and all programs of the software system under the new system version are determined as the program objects;

[0022] if the minimum program scale targeted by the test tool is a non-system-level program, target executable programs depended by the program file before and after modification are determined based on the program file, the old system version and the new system version.

[0023] In some embodiments of the present application, the determination of the target executable programs depended by the program file before modification based on the program file, the old system version and the new system version comprises:

[0024] determining whether there is a first executable program to which the program file belongs under the old system version;

[0025] if there is no first executable program, it is determined that there is no target executable program depended by the program file before modification;

[0026] If the first executable program exists, traversing the calling relationship of all executable programs of the software system under the old system version, it is determined whether a second executable program having a direct or indirect calling relationship with the first executable program exists;

[0027] If the second executable program exists, the first executable program and the second executable program are determined as target executable programs relied on by the program file before modification; if the second executable program does not exist, the first executable program is determined as a target executable program relied on by the program file before modification.

[0028] In some embodiments of the present application, based on the program file, the old system version and the new system version, the target executable program relied on by the program file after modification is determined, comprising:

[0029] A third executable program to which the program file belongs under the new system version is determined.

[0030] Traversing the calling relationship of all executable programs of the software system under the new system version, it is determined whether a fourth executable program having a direct or indirect calling relationship with the third executable program exists.

[0031] If the fourth executable program exists, the third executable program and the fourth executable program are determined as target executable programs relied on by the program file after modification; if the fourth executable program does not exist, the third executable program is determined as a target executable program relied on by the program file before modification.

[0032] In some embodiments of the present application, the performance test of the program object by the test tool comprises:

[0033] In the case that the minimum program scale targeted by the test tool is a system-level program, the test tool is used to perform performance tests on all programs under the old system version and all executable programs under the new system version respectively.

[0034] In some embodiments of the present application, the performance test of the program object by the test tool comprises:

[0035] In the case that the minimum program scale targeted by the test tool is a non-system-level program and it is determined that there exist target executable programs relied on by the program file before and after modification, the test tool is used to perform performance tests on the target executable programs relied on before and after modification.

[0036] In some embodiments of the present application, the performance testing of the program object by the testing tool comprises:

[0037] In the case where the minimum program scale targeted by the testing tool is a non-system-level program, and it is determined that there is no target executable program that the program file depends on before modification, the performance testing of the target executable program that the modified program depends on is performed by the testing tool.

[0038] In some embodiments of the present application, the online analysis based on the program file and all executable programs of the software system further comprises:

[0039] sending the program file and all executable programs of the software system to the target running platform specified by the user;

[0040] receiving the analysis result obtained by the target running platform based on the online analysis of the program file and all executable programs.

[0041] In some embodiments of the present application, the integration testing based on the program file and all executable programs of the software system comprises:

[0042] running all executable programs of the modified software system based on the program file and all executable programs of the software system;

[0043] analyzing the system performance of the modified software system during the running.

[0044] In some embodiments of the present application, in the case where the performance testing comprises the offline analysis and the online analysis, the method further comprises:

[0045] generating a test report in the case where the offline analysis result indicates that there is an abnormal problem;

[0046] performing the online analysis in the case where the offline analysis result indicates that there is no abnormal problem.

[0047] In some embodiments of the present application, in the case where the performance testing comprises the online analysis and the integration testing, the method further comprises:

[0048] generating a test report in the case where the online analysis result indicates that there is an abnormal problem;

[0049] performing the integration testing in the case where the online analysis result indicates that there is no abnormal problem.

[0050] Embodiments of the second aspect of the present application provide a performance testing device, comprising:

[0051] receiving module, configured to receive all executable programs of a software system submitted by a user and program files modified in the software system;

[0052] a testing module, configured to perform performance testing based on the program files and all executable programs of the software system to obtain a test report;

[0053] a report sending module, configured to send the test report to the user.

[0054] An embodiment of the third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method in the first aspect.

[0055] An embodiment of the fourth aspect of the present application provides a computer readable storage medium, having a computer program stored thereon, and the program is executed by a processor to implement the method in the first aspect.

[0056] The technical scheme provided in the embodiments of the present application has at least the following technical effects or advantages:

[0057] In the embodiments of the present application, a platform capable of automatically performing performance testing on a software system is provided, and a user only needs to send all executable programs of the software system and program files modified in the current software system to the platform. The platform can automatically complete performance testing on the software system and return a test report to the user. The full-process automatic testing on the software system is realized, and the degree of automation of the performance testing is very high, thereby improving the performance testing efficiency. The performance testing is incremental in the case that the program files modified in the software system are tested, and the performance testing is more targeted, thereby improving the efficiency and accuracy of the software performance testing. In the testing process, the software system can be tested in terms of offline analysis, online analysis, and integrated testing, and the performance testing is more comprehensive, efficient, and accurate.

[0058] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0059] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description, with reference to the drawings in which:

[0060] In the drawings:

[0061] Figure 1 A flowchart of a performance testing method provided in an embodiment of this application is shown;

[0062] Figure 2 A schematic diagram of an online performance test process provided in an embodiment of this application is shown;

[0063] Figure 3 A schematic diagram of the executable program on which the modified program file provided in one embodiment of this application depends is shown;

[0064] Figure 4 This paper illustrates another schematic diagram of the online performance testing process provided in one embodiment of this application;

[0065] Figure 5 Another flow chart of a performance test provided in one embodiment of this application is shown;

[0066] Figure 6 A schematic diagram of the analysis tool used for performance testing provided in an embodiment of this application is shown;

[0067] Figure 7 This illustration shows a structural schematic diagram of a performance testing device provided in one embodiment of this application;

[0068] Figure 8 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0069] Figure 9 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation

[0070] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0071] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0072] The following description, in conjunction with the accompanying drawings, describes a performance testing method, apparatus, device, and storage medium according to embodiments of this application.

[0073] Performance testing of software systems is crucial for their development and maintenance, especially in scenarios like autonomous driving where high performance requirements are paramount. However, current technologies for large-scale software system performance testing largely rely on manual debugging by developers, resulting in low automation, efficiency, and accuracy.

[0074] Based on this, this application provides a performance testing method. This method offers a platform capable of automatically performing performance testing on a software system. Users who need to perform performance testing on a software system only need to send all executable programs of the software system, as well as any modified program files in the current software system, to the platform. The platform can then automatically complete the performance testing of the software system based on all executable programs and modified program files, and return a test report to the user. This achieves automated testing of the entire software system process, with a high degree of automation, thus improving performance testing efficiency. Furthermore, testing is only performed when modified program files are found in the software system, enabling incremental performance testing of the software system. This makes performance testing more targeted and improves the efficiency and accuracy of software performance testing.

[0075] The software system can be any system requiring performance testing. This system consists of multiple functional modules, each with at least one program file. During the development and maintenance of the software system, it is often necessary to modify these program files for upgrades or maintenance. Modifying program files may alter the software system's functionality and impact its performance.

[0076] The program file can be an executable program in the software system, a function in the executable program, or several lines of code in the executable program.

[0077] See Figure 1 The method specifically includes the following steps:

[0078] Step 101: Receive all executable programs and modified program files in the software system submitted by the user.

[0079] The execution entity in this application embodiment can be a server used for software performance testing, which is equipped with the aforementioned platform capable of automatically performing performance testing on the software system. A client application corresponding to this platform can be installed on the user's mobile phone or computer, allowing the user to log in to the platform through this client.

[0080] If modified program files appear in a user's software system, the user can log in to the platform via the client and transmit all executable programs of the software system, along with the currently modified program files, to the server. The server receives all executable programs and modified program files uploaded by the user.

[0081] In some embodiments of this application, after a user modifies a program file, the program file can be stored in a code repository. The code repository can automatically detect the modified program file and automatically call the client to send the modified program file and all executable programs of the software system to the server. The modified program file can be a modified, deleted, or added program file.

[0082] Step 102: Based on the modified program files and all executable programs of the software system, perform performance testing and obtain a test report.

[0083] In this embodiment of the application, the performance testing services that the server can provide may include, but are not limited to, one or more of offline performance analysis, online performance analysis, and system integration testing.

[0084] For offline performance analysis, the server performs offline analysis based on the modified program files and all executable programs of the software system. This offline analysis is used to analyze code problems without running the software system's programs.

[0085] In some embodiments, offline analysis may include the server detecting whether there are code errors in the modified program files based on the syntax rules of the programming language used by the software system.

[0086] The programming language used in the software system can be C, C++, Java, etc. Detecting code errors in modified program files can be done through static code analysis based on the syntax rules of the programming language. Static code analysis refers to performing a syntax check on the code of a program file without running it. Alternatively, the compiler corresponding to the programming language can be invoked to perform dynamic code analysis on the modified program file. This involves compiling the code in the modified program file and checking for errors in syntax, function dependencies, naming conventions of function-level data structures, etc.

[0087] In other embodiments of this application, code inspection tools can also be used to check for coding errors in the modified program files. These code inspection tools may include, but are not limited to, unit-test and benchmark testing.

[0088] In other embodiments, offline analysis may also include using a preset memory analysis tool to detect whether there are memory anomalies in all executable programs of the software system after the program file has been modified.

[0089] The aforementioned preset memory analysis tools may include, but are not limited to, ASAN (a memory error detection tool), MSAN (a tool for detecting uninitialized memory reads in a program), and TSAN (a tool for detecting data contention between threads).

[0090] The above-mentioned offline analysis methods, which detect whether there are code errors in the modified program files and whether there are memory exceptions in all executable programs, can both be executed, or either one can be executed.

[0091] Offline analysis can quickly determine whether modified program files contain coding errors, and / or quickly screen whether modifications to these program files have caused memory errors in the executable program of the software system. If problems are found during offline analysis, a test report is generated based on the offline analysis results, and then this test report is sent to the user through step 103. If no problems are detected by offline analysis, a test report indicating that there are no coding errors or memory errors can be generated, and this test report is subsequently sent to the user. In some embodiments, if no problems are found by offline analysis, online performance analysis can continue.

[0092] If offline analysis detects a problem, a test report is generated, and the performance testing process exits. Online performance analysis only proceeds if no problems are detected offline. This saves time on subsequent online analysis or integration testing if problems are detected offline.

[0093] For online performance analysis, the server performs online analysis based on program files and all executable programs of the software system. Online analysis is used to test performance while the software system's programs are running.

[0094] Specifically, such as Figure 2 As shown, the server can perform online performance analysis through the following steps S1-S3, including:

[0095] S1: Obtain the old and new system versions of the software system before and after the modification, as well as the testing tools used in this performance test.

[0096] Every time a program file is modified, the software system version changes from the old system version before the program file was modified to the new system version after the program file was modified.

[0097] The server receives all executable programs from the software system, including the original program file. After receiving the modified program file and all executable programs from the software system, the server obtains the old system version of the software system before the program file was modified, and the new system version of the software system after the program file was modified. The old system version includes all program code before the program file was modified, and the new system version includes all program code after the program file was modified.

[0098] The server receives all executable programs of the software system as the old system version. It then replaces the original program file in each executable with the modified program file, thus obtaining the new system version of the software system.

[0099] In addition to the aforementioned old and new system versions, the server also obtains the testing tools used in this performance test, which can be preset testing tools. Alternatively, in some embodiments, the server can display a test configuration interface to the user through a client. This test configuration interface may include a setting component for configuring the testing tools. The user can submit their desired testing tools through this setting component. The setting component may include one or more of the following: selection controls, drop-down lists, input boxes, buttons, etc.

[0100] Testing tools perform performance testing on a program-by-program scale. This processing unit can be the entire system, a functional module within the software system, a specific program file within a functional module, a function within a program file, or even a single line of code within a program file. In other words, the processing unit of a testing tool can be at different scales, such as system-level, module-level, function-level, or line-of-code level.

[0101] Users can configure the testing tools they need through the test configuration interface, which greatly facilitates software performance testing and meets users' personalized needs for using testing tools.

[0102] S2: Based on the modified program files, the old system version, the new system version, and the testing tools, determine the program objects in the software system affected by the program files before and after the modification.

[0103] Corresponding to different levels of processing units in testing tools, program objects can also be divided according to different granularities. For example, program objects can include, but are not limited to, system-level, functional modules, functions, and lines of code. For functional module-level program objects, testing tools such as Linux Trace can be used to perform performance testing on a module-by-module basis. For function-level program objects, testing tools such as Google Benchmark can be used to perform performance testing on a function-by-function basis. For line-of-code-level program objects, testing tools such as Perf can be used to perform performance testing on a line-by-line basis.

[0104] The program objects affected by the program file are those that the program file depends on. Modification of the program file may affect the performance of the program objects that the program file depends on. The identification of the program objects affected by the program file is related to the testing tools used in this performance test.

[0105] In some embodiments of this application, if the minimum program size targeted by the testing tool is a system-level program, then all executable programs of the software system under the old system version and all executable programs of the software system under the new system version are identified as the program objects on which the modified program file depends.

[0106] In some other embodiments of this application, if the minimum program size targeted by the testing tool is a non-system-level program, then based on the program file, the old system version, and the new system version, the target executable program that the program file depends on before and after modification is determined as the aforementioned program object.

[0107] Non-system-level programs include, but are not limited to, programs of different sizes such as functional modules, program files, functions, and lines of code.

[0108] When the testing tool targets a non-system-level program, the process of determining the target executable programs that the program files depend on under older system versions may include:

[0109] Determine if a first executable program exists in the old system version to which the program file belongs; if the first executable program does not exist, determine if a target executable program that the program file depends on did not exist before the modification; if the first executable program exists, traverse the call relationships of all executable programs in the software system under the old system version to determine if a second executable program has a direct or indirect call relationship with the first executable program; if a second executable program exists, determine the first executable program and the second executable program as the target executable programs that the program file depends on before the modification; if a second executable program does not exist, determine the first executable program as the target executable program that the program file depends on before the modification.

[0110] The process of determining the target executable programs that a program file depends on in a new system version may include:

[0111] Identify the third executable program to which the program file belongs under the new system version; traverse the call relationships of all executable programs in the software system under the new system version to determine whether there is a fourth executable program that has a direct or indirect call relationship with the third executable program; if a fourth executable program exists, then identify the third executable program and the fourth executable program as the target executable programs that the modified program file depends on; if no fourth executable program exists, then identify the third executable program as the target executable program that the original program file depends on.

[0112] In the embodiments of this application, the executable program can be understood as a functional module in a software system. The following is in conjunction with... Figure 3 Describe the process for determining the target executable programs that this program file depends on. For example... Figure 3 As shown, modules A, B, C, D, and F are executable programs in the software system. file1.cc and file1.h are two program files of module A, file2.cc and file2.h are two program files of module B, file3.cc and file3.h are two program files of module C, file4.cc and file4.h are two program files of module D, and file5.cc and file5.h are two program files of module F. Assuming that file5.cc has been modified, in the old system version, the executable program to which file5.cc belongs is module F. Executable programs with direct or indirect calling relationships with module F include modules A, B, and C. Therefore, in the old system version, the target executable programs that file5.cc depends on are modules A, B, C, and F. Following the same processing method, it can be determined that in the new system version, the target executable programs that file5.cc depends on are also modules A, B, C, and F.

[0113] S3: Use this testing tool to perform performance testing on program objects.

[0114] In some embodiments of this application, when the minimum program size targeted by the testing tool is a system-level program, the testing tool is used to perform performance tests on all executable programs under the old system version and all executable programs under the new system version, respectively, to obtain a first test result for the old system version and a second test result for the new system version. If problems are detected by the online test, a test report is generated based on the first and second test results. The test report is then sent to the user.

[0115] In other embodiments of this application, when the minimum program size targeted by the testing tool is a non-system-level program, and it is determined that there are target executable programs that the program files before and after modification depend on, the testing tool performs performance testing on the target executable programs that depend on the program before modification to obtain a first test result, and performs performance testing on the target executable programs that depend on the program after modification to obtain a second test result. If the online test detects a problem, a test report is generated based on the first and second test results. The test report is then sent to the user.

[0116] When a program file is modified, performance tests are performed on all executable programs that the modified program file depends on. This implements incremental performance testing triggered by program file modifications, and performs performance tests on all dependent executable programs, resulting in comprehensive and highly accurate performance testing. The tests obtain the test results for the same executable program that the modified program file depends on before and after the modification. By comparing these two test results, the impact of modifying the program file on system performance can be clearly seen, which helps guide R&D or operations personnel in further development or maintenance of the software system.

[0117] In some other embodiments of this application, when the minimum program size targeted by the testing tool is a non-system-level program, and it is determined that there is no target executable program that the program file before modification depends on, the testing tool is used to perform performance testing on the target executable program that the modification depends on.

[0118] In one example, if the modified program file is a new file, the executable program that the new version depends on may not exist in the old system version. In this case, performance testing is only performed on the executable program that the modified program file depends on.

[0119] In some other embodiments of this application, when there are multiple modified program files, if the program objects determined based on each modified program file contain the same program object, then the number of times the performance test is performed on the same program object is one.

[0120] In other words, performance tests will not be performed repeatedly on the same program objects, thus saving performance testing resources and improving performance testing efficiency.

[0121] like Figure 3 As shown, if both file4.cc and file5.cc are modified, then module B is an executable program that both file4.cc and file5.cc depend on. Therefore, only one performance test will be performed on module B under the old system version, and only one performance test will be performed on module B under the new system version.

[0122] If the modified program file has dependent executables in the old system version but not in the new system version, then performance testing can be performed only on the executables that the program file depends on in the old system version. Alternatively, performance testing can be omitted. Since the new system version does not have any dependent executables for this program file, modifying the program file will not affect the performance of the new system version.

[0123] In other embodiments of this application, the method also provides a cross-platform online performance testing method. This cross-platform testing is based on two platforms; for ease of description, one platform is referred to as the current running platform, and the other as the target running platform. The processor architectures of the current running platform and the target running platform can be the same or different. For example, the processor of the current running platform can be an x86 processor, while the processor of the target running platform can be an ARM processor.

[0124] In cross-platform application scenarios, after the current operating platform receives the modified program file and all executable programs of the software system submitted by the user, it can send the program file and all executable programs of the software system to the target operating platform. The target operating platform receives the modified program file and all executable programs of the software system, performs online performance testing based on the program file and all executable programs of the software system according to any of the above embodiments, and sends the analysis results of the online analysis to the current operating platform.

[0125] The above methods enable online performance testing in cross-platform scenarios, improving the efficiency and accuracy of online performance testing in cross-platform scenarios.

[0126] To facilitate understanding of online performance testing, the following explanation is provided in conjunction with accompanying diagrams. Figure 4 As shown, after detecting that the program file file5.cc has been modified, the modified program file file5.cc is displayed. Based on this program file file5.cc, the upstream dependencies of the program file file5.cc in the software system under the old system version are analyzed to determine the executable programs A, B, C, and F that the program file file5.cc depends on. The executable programs A', B', C', and F' that the program file file5.cc depends on under the new system version are also analyzed. Using the executable programs A, B, C, and F in the old system version as the base versions, and the executable programs A', B', C', and F' in the new system version as the evaluation versions, performance tests are performed on the executable programs A and A', B and B', C and C', and F and F' respectively, and a final test report is generated based on the obtained test results.

[0127] In this application embodiment, online performance testing may include, but is not limited to, lock / latency analysis, CPU usage analysis, I / O resource analysis, and dynamic memory analysis. CPU usage analysis includes, but is not limited to, CPU time consumption and memory usage. This application embodiment can perform online performance evaluation at the system level, module level, function level, and line-of-code level. Taking CPU time consumption as an example, if function-level performance testing is performed, the time consumption of each function in the executable program can be tested; if line-of-code-level performance testing is performed, the time consumption of each line of code in the executable program can be tested. This allows for accurate identification of functions or lines of code with performance problems, thereby providing more targeted guidance for developers to modify and maintain the program code to improve the performance of the software system.

[0128] If the online analysis does not detect any problems, further integration testing can be performed based on the modified program files and all executable programs of the software system. Specifically, based on the modified program files and all executable programs of the software system, all executable programs of the modified software system are run, and the system performance of the modified software system is analyzed during the running process.

[0129] Integration testing allows for the evaluation of a software system's overall latency, performance, and stability. For example, it involves running the software system for a certain period (e.g., a day) to check for abnormal system crashes. It also examines the latency of task processing within the software system during operation.

[0130] After integration testing is completed, a test report is generated based on the test results and then sent to the user.

[0131] In this embodiment, offline performance analysis, online performance analysis, and integrated analysis are used; in practical applications, one or more of these three methods can be employed. As an example, performance testing includes offline and online analysis. In this case, if the offline analysis results indicate an anomaly, a test report is generated. If the offline analysis results indicate no anomalies, the online analysis is performed.

[0132] In other embodiments, where performance testing includes online analysis and integration testing, a test report is generated if the online analysis results indicate an anomaly. If the online analysis results indicate no anomalies, integration testing is performed.

[0133] In some other embodiments, performance testing includes offline analysis, online analysis, and integration testing. If the offline analysis results indicate an anomaly, a test report is generated. If the offline analysis results indicate no anomalies, online analysis is performed. If the online analysis results indicate an anomaly, a test report is generated. If the online analysis results indicate no anomalies, integration testing is performed.

[0134] In other words, offline analysis is performed before online analysis and / or integration testing. Online analysis is performed after offline analysis, and / or online analysis is performed before integration testing. Integration testing is performed after offline analysis and / or online analysis. Each section proceeds to the next stage only if no issues are detected during execution. If an issue is detected in the current section, subsequent testing is skipped, and a test report is generated directly. This saves time in subsequent testing processes and improves performance testing efficiency.

[0135] Step 103: Send the test report to the user.

[0136] After the test report is generated in step 102, the server sends the test report to the user's client. The client then displays the test report to the user.

[0137] To facilitate understanding of a specific performance testing process proposed in the embodiments of this application, the following is in conjunction with the appendix. Figure 5 Please provide an explanation. For example... Figure 5 As shown, the user's client sends the modified program files and all executable programs of the software system to the server. The server first performs offline analysis, conducting static or dynamic code checks on the modified program files. It uses code inspection tools such as unittest or benchmark to perform code checks on the modified program files, and uses pre-defined memory analysis tools such as ASAN, MSAN, and TSAN to check for memory anomalies in all executable programs of the software system after the modification. If the offline analysis detects problems, subsequent online analysis and integration testing are not performed. Instead, a test report is generated based on the offline analysis and sent back to the client, providing code and memory test results for the user to filter for initial code issues.

[0138] If offline analysis finds no problems, online analysis continues. Online analysis uses testing tools such as benchmark, trace, and perf to perform performance tests on the executable programs that the modified program files depend on under both the old and new system versions of the software. Figure 5As shown, online performance testing can include lock / latency analysis, CPU usage analysis (function hotspots, performance bottlenecks), I / O resource analysis, and dynamic memory analysis. If online analysis does not find any issues, subsequent integration testing continues. If online analysis detects issues, a test report is generated based on the analysis results and sent to the client to provide users with performance test results at different granularities, such as function-level, module-level, system-level, and line-of-code-level. In practical applications, one or more of these different granularity performance tests can be performed during online analysis.

[0139] like Figure 5 As shown, if no issues are found during online analysis, integration testing continues. During integration testing, the entire modified software system is run, and latency / performance and stability issues are analyzed. Based on the integration test results, a test report is generated and sent to the client.

[0140] exist Figure 5 The performance analysis process shown uses many analysis tools. Figure 6 This illustration demonstrates some possible analysis tools. For example, regarding locks, locks are widely used as a safety mechanism for multi-threaded concurrency, but improper use can lead to excessively long thread mutual exclusion times, affecting frame rate and latency stability. Lock analysis tools that can be used in this embodiment include, but are not limited to, lock data statistics, lock call stack tracing, IO resource analysis, and dynamic memory analysis. Regarding high load issues, high CPU load (e.g., many cores exceeding 80% load) and thread preemption are common problems in software development projects; loads above 70% often result in frame drops and stability issues. System load analysis tools that can be used in this embodiment include, but are not limited to, hotspot function analysis, thread state analysis, IO resource analysis, and logging-specific analysis. Regarding memory issues, if the system's dynamic memory allocation efficiency is low, a large number of dynamic memory allocations can lead to a significant increase in CPU load and system lag. Dynamic memory analysis tools that can be used in this embodiment include, but are not limited to, dynamic memory statistics and dynamic memory profiling. Regarding frame rate issues, frame drops, unstable frame rates, and large latency fluctuations can reduce the system's real-time performance and smoothness. Frame rate analysis tools that can be used in this embodiment include, but are not limited to, frame rate stability analysis.

[0141] This application provides a platform capable of automatically performing performance testing on a software system. Users simply need to send all executable programs of the software system, along with any modified program files currently in the system, to the platform. The platform can then automatically complete the performance testing of the software system and return a test report to the user. This achieves fully automated testing of the entire software system process, with a high degree of automation, thus improving performance testing efficiency. Furthermore, testing is only performed when modified program files are found in the software system, enabling incremental performance testing that is more targeted and improves both efficiency and accuracy. During the testing process, offline analysis, online analysis, and integration testing can be performed on the software system, resulting in more comprehensive, efficient, and accurate performance testing.

[0142] This application also provides a performance testing apparatus for performing the performance testing method provided in any of the above embodiments. Figure 7 As shown, the device includes:

[0143] The receiving module 201 is used to receive all executable programs of the software system and modified program files in the software system submitted by the user.

[0144] Test module 202 is used to perform performance tests on all executable programs of the program file and software system and obtain test reports;

[0145] The report sending module 203 is used to send test reports to users.

[0146] Test module 202 is used for offline analysis based on program files and all executable programs of the software system, whereby offline analysis is used to analyze code problems without running the program of the software system; and / or, for online analysis based on program files and all executable programs of the software system, whereby online analysis is used to test performance while the program of the software system is running; and / or, for integration testing based on program files and all executable programs of the software system.

[0147] Test module 202 is used to detect whether there are code writing errors in the program file based on the syntax rules of the programming language used by the software system; and / or, to detect whether there are memory abnormalities in all executable programs of the software system after the program file is modified by a preset memory analysis tool.

[0148] Test module 202 is used to obtain the old system version and the new system version of the software system before and after the modification, as well as the test tools used in this performance test; based on the program files, the old system version, the new system version, and the test tools, it determines the program objects in the software system before and after the modification that are affected by the program files; and it uses the test tools to perform performance tests on the program objects.

[0149] Test module 202 is used to determine all programs of the software system under the old system version and all programs of the software system under the new system version as program objects if the minimum program size targeted by the test tool is a system-level program; if the minimum program size targeted by the test tool is a non-system-level program, it determines the target executable program that the program files before and after modification depend on based on the program files, the old system version and the new system version.

[0150] Test module 202 is used to determine whether a first executable program to which the program file belongs exists under the old system version; if the first executable program does not exist, it is determined that there is no target executable program that the program file depends on before the modification; if the first executable program exists, the call relationship of all executable programs in the software system under the old system version is traversed to determine whether there is a second executable program that has a direct or indirect call relationship with the first executable program; if the second executable program exists, the first executable program and the second executable program are determined as the target executable programs that the program file depends on before the modification; if the second executable program does not exist, the first executable program is determined as the target executable program that the program file depends on before the modification.

[0151] Test module 202 is used to determine the third executable program to which the program file belongs under the new system version; it iterates through the call relationships of all executable programs in the software system under the new system version to determine whether there is a fourth executable program that has a direct or indirect call relationship with the third executable program; if there is a fourth executable program, then the third executable program and the fourth executable program are determined as the target executable program that the modified program file depends on; if there is no fourth executable program, then the third executable program is determined as the target executable program that the program file before modification depends on.

[0152] Test module 202 is used to perform performance tests on all programs under the old system version and all executable programs under the new system version, when the minimum program size targeted by the test tool is a system-level program.

[0153] Test module 202 is used to perform performance tests on the target executable programs that the test tool targets as non-system-level programs, and to determine that there are target executable programs that the program files before and after modification depend on.

[0154] Test module 202 is used to perform performance tests on the modified target executable program when the minimum program size targeted by the test tool is a non-system-level program and it is determined that there is no target executable program that the program file before modification depends on.

[0155] The test module 202 is used to send the program files and all executable programs of the software system to the target running platform specified by the user; and to receive the analysis results obtained by the target running platform through online analysis based on the program files and all executable programs.

[0156] Test module 202 is used to run all executable programs of the modified software system based on program files and all executable programs of the software system; and to analyze the system performance of the modified software system during the running process.

[0157] In some embodiments, where performance testing includes offline analysis and online analysis, the testing module 202 is configured to generate a test report if the offline analysis results indicate the presence of an anomaly, and to perform online analysis if the offline analysis results indicate the absence of an anomaly.

[0158] In other embodiments, where performance testing includes online analysis and integration testing, the test module 202 is configured to generate a test report if the online analysis results indicate an anomaly, and to perform integration testing if the online analysis results indicate no anomaly.

[0159] The performance testing device and the performance testing method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0160] This application also provides an electronic device for performing the above-described performance testing method. Please refer to... Figure 8 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802, and a communication interface 803. The processor 800, the communication interface 803, and the memory 801 are connected via the bus 802. The memory 801 stores a computer program that can run on the processor 800. When the processor 800 runs the computer program, it executes the performance testing method provided in any of the foregoing embodiments of this application.

[0161] The memory 801 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 803 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0162] Bus 802 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs. After receiving an execution instruction, the processor 800 executes the program. The performance testing method disclosed in any of the foregoing embodiments of this application can be applied to the processor 800, or implemented by the processor 800.

[0163] The processor 800 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 800 or by instructions in software form. The processor 800 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 801. Processor 800 reads the information in memory 801 and, in conjunction with its hardware, completes the steps of the above method.

[0164] The electronic device provided in this application embodiment and the performance testing method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0165] This application also provides a computer-readable storage medium corresponding to the performance testing method provided in the foregoing embodiments. Please refer to... Figure 9 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the performance testing method provided in any of the foregoing embodiments.

[0166] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0167] The computer-readable storage medium provided in the above embodiments of this application and the performance testing method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0168] It should be noted that:

[0169] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0170] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting a schematic diagram in which the claimed application requires more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0171] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0172] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A performance testing method, characterized in that, include: Receive all executable programs of the software system submitted by the user, as well as the modified program files in the software system; Based on the program files and all executable programs of the software system, performance testing is performed to obtain a test report; The test report is sent to the user.

2. The method according to claim 1, characterized in that, The performance testing based on the program file and all executable programs of the software system includes: Offline analysis is performed based on the program files and all executable programs of the software system. This offline analysis is used to analyze code issues without running the programs of the software system; and / or, Based on the program files and all executable programs of the software system, online analysis is performed to test performance while the programs of the software system are running; and / or, Integration testing is performed based on the program files and all executable programs of the software system.

3. The method according to claim 2, characterized in that, The offline analysis based on the program file and all executable programs of the software system includes: Based on the syntax rules of the programming language used by the software system, detect whether there are any code errors in the program files; and / or, The software system uses a preset memory analysis tool to detect whether there are memory anomalies in all executable programs after the program files have been modified.

4. The method according to claim 2, characterized in that, The online analysis based on the program file and all executable programs of the software system includes: Obtain the old and new system versions of the software system before and after the modification, as well as the testing tools used in this performance test; Based on the program file, the old system version, the new system version, and the testing tool, determine the program objects in the software system affected by the program file before and after the modification; The performance of the program object is performed using the aforementioned testing tool.

5. The method according to claim 4, characterized in that, The process of determining the program objects in the software system affected by the program file before and after modification, based on the program file, the old system version, the new system version, and the testing tool, includes: If the minimum program size targeted by the testing tool is a system-level program, then all programs of the software system under the old system version and all programs of the software system under the new system version are identified as the program objects. If the minimum program size targeted by the testing tool is a non-system-level program, then based on the program file, the old system version, and the new system version, the target executable program that the program file depends on before and after the modification is determined.

6. The method according to claim 5, characterized in that, Based on the program file, the old system version, and the new system version, determine the target executable program that the program file depends on before modification, including: Determine whether a first executable program to which the program file belongs exists under the old system version; If the first executable program does not exist, it is determined that the target executable program that the program file depends on did not exist before the modification; If the first executable program exists, traverse the call relationships of all executable programs in the software system under the old system version to determine whether there is a second executable program that has a direct or indirect call relationship with the first executable program; If the second executable program exists, then the first executable program and the second executable program are determined as the target executable program on which the program file depends before modification; if the second executable program does not exist, then the first executable program is determined as the target executable program on which the program file depends before modification.

7. The method according to claim 5, characterized in that, Based on the program file, the old system version, and the new system version, determine the target executable program that the modified program file depends on, including: Determine the third executable program to which the program file belongs under the new system version; Traverse the call relationships of all executable programs in the software system under the new system version to determine whether there is a fourth executable program that has a direct or indirect call relationship with the third executable program; If the fourth executable program exists, then the third executable program and the fourth executable program are determined as the target executable program on which the modified program file depends; if the fourth executable program does not exist, then the third executable program is determined as the target executable program on which the original program file depends.

8. The method according to claim 5, characterized in that, The performance testing of the program object using the testing tool includes: When the minimum program size targeted by the testing tool is a system-level program, the testing tool is used to perform performance tests on all programs under the old system version and all executable programs under the new system version.

9. The method according to claim 5, characterized in that, The performance testing of the program object using the testing tool includes: When the minimum program size targeted by the testing tool is a non-system-level program, and it is determined that there are target executable programs that the program files before and after the modification depend on, the testing tool is used to perform performance tests on the target executable programs that depend on before the modification and the target executable programs that depend on after the modification.

10. The method according to claim 5, characterized in that, The performance testing of the program object using the testing tool includes: If the minimum program size targeted by the testing tool is a non-system-level program, and it is determined that there is no target executable program that the program file depends on before modification, the testing tool is used to perform performance testing on the target executable program that the program depends on after modification.

11. The method according to claim 2, characterized in that, The online analysis based on the program file and all executable programs of the software system also includes: Send the program file and all executable programs of the software system to the target operating platform specified by the user; Receive the analysis results obtained by the target operating platform through online analysis based on the program file and all executable programs.

12. The method according to claim 2, characterized in that, The integration test, based on the program file and all executable programs of the software system, includes: Based on the program file and all executable programs of the software system, run all executable programs of the modified software system; The system performance of the modified software system was analyzed during operation.

13. The method according to any one of claims 2-12, characterized in that, In the case that the performance test includes both offline analysis and online analysis, the method further includes: If the offline analysis results indicate an anomaly, a test report is generated. If the offline analysis results indicate that there are no abnormalities, then perform the online analysis.

14. The method according to any one of claims 2-12, characterized in that, In the case that the performance testing includes the online analysis and the integration testing, the method further includes: If the online analysis results indicate an anomaly, a test report is generated. If the online analysis results indicate that there are no abnormal issues, then perform the integration test.

15. A performance testing device, characterized in that, include: The receiving module is used to receive all executable programs of the software system submitted by the user, as well as the modified program files in the software system. The testing module is used to perform performance tests based on the program file and all executable programs of the software system and obtain a test report. The report sending module is used to send the test report to the user.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-15.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-14.

Images