A method and apparatus for performance testing of an application

Abstract

This invention provides a method, apparatus, electronic device, and computer-readable storage medium for performance testing of applications. The method includes: running the application based on its installation package and injecting test code corresponding to a performance testing scenario into the running application; running a simulated service of the application through the test code and sending a pseudo-random user event stream to the application through the simulated service to trigger the application to run according to the pseudo-random user event stream; collecting data generated by the application in real time during the application's operation according to the pseudo-random user event stream; and analyzing and processing the data to obtain performance test results. This invention enables efficient and accurate testing of application performance.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for testing the performance of applications. Background Technology

[0002] Performance testing plays a crucial role in ensuring the quality of applications (APPs). APP performance testing includes client-side performance testing, network performance testing, and server-side performance testing. Performance testing primarily measures various metrics during APP operation, such as CPU (Central Processing Unit) utilization.

[0003] Currently, there are two main methods for app performance testing. The first is manual testing, where testers manually trigger performance tests, and different metrics need to be tested one by one. Manual testing is time-consuming and inefficient. The second method is to modify the underlying code of the app under test to achieve performance testing. However, modifying the underlying code of the app under test may affect the original performance data of the app under test, resulting in lower accuracy of performance testing. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for testing the performance of applications, which can efficiently and accurately test the performance of applications.

[0005] The technical solution of this invention is implemented as follows:

[0006] This invention provides a method for testing the performance of an application, the method comprising:

[0007] The application is run based on the application's installation package, and test code corresponding to the performance test scenario is injected into the running application.

[0008] The test code runs a simulated service of the application, and the simulated service sends a pseudo-random stream of user events to the application.

[0009] The application is triggered to run based on the pseudo-random user event stream;

[0010] During the process of the application running according to the pseudo-random user event stream, the data generated by the application is collected in real time;

[0011] The data is analyzed and processed to obtain performance test results.

[0012] This invention provides an application performance testing device, comprising:

[0013] The injection module is used to run the application based on the application's installation package and inject test code corresponding to the performance test scenario into the running application;

[0014] The sending module is used to run a simulated service of the application through the test code, and send a pseudo-random user event stream to the application through the simulated service to trigger the application to run according to the pseudo-random user event stream;

[0015] The data acquisition module is used to collect data generated by the application in real time during the process of the application running according to the pseudo-random user event stream;

[0016] The analysis module is used to analyze and process the data to obtain performance test results.

[0017] In the above scheme, the injection module is further configured to inject test code corresponding to the performance test scenario into the running application by executing a performance test script, so as to obtain the control tree information of the application; based on the control tree information of the application, establish a communication channel between the simulation service and the application, so that the simulation service sends the pseudo-random user event stream to the application through the communication channel; wherein, the performance test scenario includes at least one of the following: startup test scenario; homepage swiping test scenario; entering live room test scenario; memory usage test scenario; CPU usage test scenario; GPU usage test scenario; smoothness test scenario; power consumption test scenario.

[0018] In the above scheme, the sending module is further configured to send the pseudo-random user event stream to the application through the simulation service and via the communication channel, so that the pseudo-random user event stream is transmitted in the control tree of the application, and the application responds to the pseudo-random user event stream transmitted in the control tree.

[0019] In the above scheme, the acquisition module is further configured to record the application's running log in real time during the application's operation according to the pseudo-random user event stream; create a data acquisition task for the type of the performance test scenario; and extract corresponding data from the running log according to the data acquisition task corresponding to the performance test scenario; wherein the data includes at least one of the following information: startup time information; memory usage information; CPU usage information; GPU usage information; smoothness information; and power consumption information.

[0020] In the above scheme, the analysis module is further used to extract data values ​​corresponding to each time point from the data and write the data values ​​corresponding to each time point to a database table; call the interface of the database table to obtain the data values ​​of the application at each time point; and determine the performance test results of the application in the form of a coordinate system with the time point as the horizontal axis and the data values ​​of the application at each time point as the vertical axis.

[0021] In the above scheme, the application performance testing device further includes: an installation package acquisition module, used to pull the installation package of the application from the branch corresponding to the application in the application building platform by executing the installation package acquisition script; or, to obtain the installation package of the application from the application store by executing the installation package acquisition script.

[0022] In the above scheme, the application performance testing device further includes: a display module, used to call the interface of the database table to obtain data values ​​of multiple applications at various time points; wherein, the multiple applications are multiple applications of the same type but different applications, or multiple applications of the same type but different application versions; based on the data values ​​of the multiple applications at various time points, a performance comparison chart of the multiple applications is determined and displayed.

[0023] In the above scheme, the application performance testing device further includes: a reminder module, used to periodically send the performance test results and / or performance test links by executing a reminder script; wherein, the performance test link is a network link used to obtain the performance test results.

[0024] In the above scheme, the application performance testing device further includes: an on-chain module, used to store the performance test results in a blockchain network, so that the blockchain network responds to the performance test data request for the application based on the stored performance test results.

[0025] This invention provides an electronic device, comprising:

[0026] Memory, used to store executable instructions;

[0027] The processor, when executing executable instructions stored in the memory, implements the performance testing method for the application provided in this embodiment of the invention.

[0028] This invention provides a computer-readable storage medium storing executable instructions for inducing a processor to execute, thereby implementing the performance testing method for the application provided in this invention.

[0029] The embodiments of the present invention have the following beneficial effects:

[0030] By injecting test code corresponding to the performance test scenario into the running application, performance testing of the application can be carried out without modifying the underlying code of the application, thereby improving the accuracy of performance testing. By collecting data generated by the application in real time during the performance test and analyzing and processing the collected data to obtain performance test results, performance testing can be completed without human intervention, thus improving the efficiency of performance testing. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the architecture of the application performance testing system 100 provided in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in an embodiment of the present invention;

[0033] Figure 3 This is a flowchart illustrating the application performance testing method provided in an embodiment of the present invention;

[0034] Figure 4 This is a flowchart illustrating the application performance testing method provided in an embodiment of the present invention;

[0035] Figure 5 This is a flowchart illustrating the application performance testing method provided in an embodiment of the present invention;

[0036] Figure 6 This is an application diagram illustrating the performance testing method for applications provided in this embodiment of the invention;

[0037] Figure 7 This is a schematic diagram illustrating an application scenario of the performance testing method for applications provided in this embodiment of the invention;

[0038] Figure 8 This is a schematic diagram illustrating an application scenario of the performance testing method for applications provided in this embodiment of the invention;

[0039] Figure 9 This is a schematic diagram illustrating an application scenario of the performance testing method for applications provided in this embodiment of the invention;

[0040] Figure 10 This is a schematic diagram of the structure of the application performance testing system provided in this embodiment of the invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0044] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0045] 1) An application is a computer program designed to perform one or more specific tasks. An application is a single executable file or a single program. Applications run in user mode, can interact with the user, and have a visual user interface.

[0046] 2) Program instrumentation is a method of inserting probes (or detectors) into the program while ensuring the original logical integrity of the program under test. By executing the probes and releasing characteristic data of the program's operation, the control flow and data flow information of the program can be obtained through the analysis of this data, thereby obtaining dynamic information such as logic coverage, and thus achieving the testing purpose.

[0047] 3) An installer pack, also known as a software installer, is a collection of self-extracting files, including all the files required for the software installation. Running this installer pack (executable file) will extract all the software's files to your hard drive, allowing you to modify the registry, change system settings, create shortcuts, and perform other tasks.

[0048] 4) A script is an extension of batch processing files. It's a program saved as plain text. Generally, a computer script is a defined series of actions that control the computer to perform calculations and operations, and it can implement logical branches, etc. Scripts can usually be temporarily called and executed by applications. Various scripts are widely used in web design because they can not only reduce the size of web pages and improve browsing speed, but also enrich the presentation of web pages, such as with animations and sounds. For example, automatically launching email software when clicking an email address on a webpage is achieved through script functionality.

[0049] 5) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0050] 6) Blockchain is an encrypted, chain-like storage structure for transactions formed by blocks.

[0051] 7) A blockchain network is a collection of nodes that incorporate new blocks into a blockchain through consensus.

[0052] 8) Ledger is a collective term for the blockchain (also known as ledger data) and the state database synchronized with the blockchain. The blockchain records transactions in the form of files in a file system; the state database records transactions in the blockchain in the form of key-value pairs of different types, supporting fast querying of transaction data in the blockchain.

[0053] 9) Smart Contracts, also known as chaincode or application code, are programs deployed in nodes of a blockchain network. Nodes execute smart contracts called in received transactions to update or query key-value pairs in the state database.

[0054] 10) Consensus is a process in a blockchain network used to reach an agreement on transactions in a block among multiple involved nodes. The agreed-upon block is appended to the end of the blockchain and used to update the state database.

[0055] In related technologies, application performance testing can be achieved by modifying the underlying code of the application under test. However, these technologies have the following technical problems: First, while modifying the underlying code can obtain performance test data, it cannot guarantee that the modification will not affect the original performance test data. Second, the obtained performance test data cannot be optimally analyzed by testers quickly, requiring significant manpower for secondary analysis. Third, it does not support performance comparisons between similar applications or between different versions of the same application.

[0056] To address the aforementioned technical problems, embodiments of the present invention provide a method, apparatus, electronic device, and computer-readable storage medium for testing application performance, which can efficiently and accurately test application performance. The following describes exemplary applications of the application performance testing method provided by embodiments of the present invention. This method can be implemented by a server, for example, by a single server or by multiple servers (i.e., a server cluster) working together.

[0057] The following describes an embodiment of the present invention using a single server as an example. See [link to documentation]. Figure 1 , Figure 1 This is a schematic diagram of the architecture of the application performance testing system 100 provided in an embodiment of the present invention. The application performance testing system 100 includes a server 200, a network 300, and a terminal 400, which will be described separately.

[0058] Server 200 is used to obtain the installation package of the application to be tested, and to perform performance testing based on the installation package of the application to be tested (the performance testing process will be described in detail below) to obtain the performance test results, and then send the performance test results to terminal 400 for display.

[0059] Network 300 is used as a medium for communication between server 200 and terminal 400, and can be a wide area network, a local area network, or a combination of both.

[0060] Terminal 400 is used to receive and display the performance test results of the application sent by server 200, so that testers can fix the application by viewing the performance test results to improve the application's performance; or, so that users can select the best-performing application from multiple applications of the same type by viewing the performance test results.

[0061] The embodiments of this invention can be widely applied to the application development stage and the user usage stage. For example, in the application development stage, before the application is released, the performance of the application to be released is tested to identify the problems that exist in the application, and the functions of the application to be released are improved based on the problems, so as to improve the user experience of the application. In the user usage stage, before a user wants to select the best performing application among applications of the same type, performance tests can be performed on multiple applications of the same type, and the best performing application can be selected based on the performance test results.

[0062] The structure of the electronic device for performing application performance testing provided in the embodiments of the present invention will be described next. The electronic device may be... Figure 1 Server 200 shown, see Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 provided in an embodiment of the present invention. Figure 2 The illustrated electronic device 500 includes at least one processor 510, a memory 530, and at least one network interface 520. The various components in the electronic device 500 are coupled together via a bus system 540. It is understood that the bus system 540 is used to implement communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 540.

[0063] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0064] The memory 530 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 530 described in this embodiment is intended to include any suitable type of memory. The memory 530 may optionally include one or more storage devices physically located away from the processor 510.

[0065] In some embodiments, memory 530 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0066] Operating system 531 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0067] The network communication module 532 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0068] In some embodiments, the application performance testing device provided in this invention can be implemented using a combination of hardware and software. As an example, the application performance testing device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the application performance testing method provided in this invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0069] In other embodiments, the application performance testing device provided in this invention can be implemented in software. Figure 2 A performance testing apparatus 533 for an application stored in memory 530 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: an injection module 5331, a sending module 5332, a data acquisition module 5333, and an analysis module 5334. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0070] In other embodiments, the application performance testing device provided in this invention can be implemented in hardware. As an example, the application performance testing device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the application performance testing method provided in this invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0071] Below, by Figure 1 The embodiment of the present invention is illustrated using a server 200 implementing the application performance testing method provided in this embodiment as an example. See also... Figure 3 , Figure 3 This is a flowchart illustrating the application performance testing method provided in this embodiment of the invention, which will be combined with... Figure 3 The steps shown are explained.

[0072] In step S101, the application installation package is obtained.

[0073] In some embodiments, the installation package of an application is pulled from the corresponding application branch in the application build platform (RDM) by executing an installation package retrieval script.

[0074] This application building platform includes a vast number of application installation packages, encompassing both the latest and older versions. It also includes multiple branches, such as social networking branches, game branches, and live streaming branches.

[0075] In other embodiments, the application's installation package is obtained from the App Store by executing an installation package retrieval script.

[0076] Here, the app store contains a huge number of app installation packages, including not only the latest versions of apps but also older versions.

[0077] As an example, server 200 responds to an application test request, which includes the name of the application to be tested; server 200 retrieves the installation package of the application corresponding to the name of the application to be tested from the application building platform or application store.

[0078] In this embodiment of the invention, by executing an installation package acquisition script, the installation package of the application is automatically obtained from the application building platform or application store, which speeds up the acquisition time of the application installation package and improves the efficiency of the application testing process.

[0079] In step S102, the application is run based on the application's installation package.

[0080] Here, the application's installation package is a collection of self-extracting files, including all the files needed to install the application. Installation can be completed by extracting the files from the application's installation package.

[0081] In some embodiments, the application is run in a corresponding terminal environment based on the application's installation package, wherein the terminal environment is a terminal hardware environment.

[0082] As an example, all files in the application's installation package are extracted to the appropriate terminal environment, and modifications are made to the registry and system settings to run the application in the appropriate terminal environment.

[0083] In step S103, test code corresponding to the performance test scenario is injected into the running application.

[0084] Here, the performance test scenarios include at least one of the following: startup test scenario; homepage scrolling test scenario; memory usage test scenario; CPU usage test scenario; GPU (Graphics Processing Unit) usage test scenario; smoothness test scenario; power consumption test scenario. When the application type is a live streaming application, the performance test scenarios may also include a live streaming room entry test scenario.

[0085] In some embodiments, by executing a performance test script, test code (or test stubs) corresponding to the performance test scenario is injected into the running application to obtain the application's control tree information; based on the application's control tree information, a communication channel is established between the simulation service and the application so that the simulation service can send pseudo-random user event streams to the application through the communication channel.

[0086] As an example, by executing a performance test script, test code is injected into the process of a running application. Through the reflection function in the test code, the control tree information of the running application process, as well as the properties and methods of related classes and objects, can be obtained. During the application's execution, a mock service (or mock server) is created in the test code. Based on the control tree information of the running application process, as well as the properties and methods of related classes and objects, a communication channel can be established between the mock service and the application, so that the mock service can send pseudo-random user event streams to the application through the communication channel.

[0087] In this embodiment of the invention, injecting test code corresponding to the performance test scenario into the running application does not modify the underlying implementation code of the application. This differs from related technologies that achieve application performance testing by changing the underlying code of the application under test. This solves the technical problem in related technologies that cannot guarantee whether changing the underlying code of the application under test will affect the original performance test data of the application under test, thereby improving the accuracy of performance testing.

[0088] In step S104, the application's simulation service is run through test code, and a pseudo-random user event stream is sent to the application through the simulation service to trigger the application to run according to the pseudo-random user event stream.

[0089] Here, the pseudo-random user event stream includes key events, touch events, and gesture input events, which can simulate the user's actual operation, allowing the application to run in the terminal environment and enabling the testing of the application.

[0090] In some embodiments, a simulated service of the application is run by test code, and a pseudo-random user event stream is sent to the application through the simulated service and via a communication channel, so that the pseudo-random user event stream is passed through the application's control tree, and the application responds to the pseudo-random user event stream passed through the control tree.

[0091] As an example, during application runtime, a mock service (or mock server) is created in the test code. Based on the application's control tree information, a communication channel can be established between the mock service and the application. The mock service sends a pseudo-random user event stream to the application through the communication channel, so that the pseudo-random user event stream is transmitted in the application's control tree. Thus, the application can perform corresponding operations based on the pseudo-random user event stream transmitted in the control tree. For example, when the pseudo-random user event stream is a key event, the application responds to the key event and makes a corresponding response. The application's corresponding response to the pseudo-random user event stream is reflected in the application's runtime log.

[0092] In this embodiment of the invention, a pseudo-random user event stream is sent to the application through a simulated service, causing the application to run according to the pseudo-random user event stream. This allows for application testing without human intervention, thereby saving human resources and improving the efficiency of performance testing.

[0093] In step S105, during the process of the application running according to the pseudo-random user event stream, the data generated by the application (or performance test data) is collected in real time.

[0094] Here, the data includes at least one of the following: startup time information; memory usage information; central processing unit (CPU) usage information; graphics processing unit (GPU) usage information; smoothness information; power consumption information.

[0095] The startup time information includes the time it takes to start the application; the CPU usage information includes the CPU usage value at each time point, the maximum peak CPU usage, and the average CPU usage; the memory usage information includes the memory usage value at each time point, the peak memory usage, and the average memory usage; the smoothness information includes the frame rate (FPS) value at each time point, the peak FPS, and the average FPS; the power consumption information includes the power consumption at each time point, the power consumption before the test, and the power consumption after the test, and so on. The information contained in the above data will not be listed one by one here.

[0096] In some embodiments, the application's runtime logs are recorded in real time as the application runs according to a pseudo-random stream of user events; data collection tasks are created for different types of performance testing scenarios; and corresponding data is extracted from the runtime logs based on the data collection tasks corresponding to the performance testing scenarios.

[0097] Taking the smoothness test scenario as an example, the data collection task created for the smoothness test scenario is to collect smoothness information. The smoothness information includes frame rate (FPS) information and stuttering rate (i.e., the number of stutters within 1 second) information (or stuttering information), etc.

[0098] For example, when the application is a game, it's necessary to collect the average number of game screen refreshes (i.e., frame rate information) and the time taken to display the current game screen frame within one second from the application's runtime log. When the time taken to display the current game screen frame is more than twice the average time taken to display the previous three game screen frames, it can be considered as one instance of stuttering, and the number of stutters within one second (i.e., stutter rate) can be counted. In this way, the smoothness information of game applications can be collected.

[0099] In some embodiments, after collecting the data generated by the application, performance analysis can be performed based on the collected data to obtain performance analysis results. For example, after collecting the lag information generated by the application, the reasons for the application's lag can be analyzed based on the lag information, and the reasons for the lag can be shown to the testers, along with solutions to the lag, so that the testers can improve the performance of the application to address the lag issues.

[0100] This invention, based on a data acquisition task, automatically collects data generated by the application during performance testing. This solves the technical problems in related technologies where performance testing requires manual triggering by testers, and different metrics need to be tested individually, resulting in time-consuming and inefficient manual testing. This saves human resources and improves the efficiency and accuracy of performance testing. Furthermore, this invention performs performance analysis based on the collected data to obtain analysis results and provide solutions. This addresses the technical problem in related technologies where performance test data cannot be quickly analyzed by testers, allowing testers to quickly identify application problems and improve application performance.

[0101] In step S106, the data is analyzed and processed to obtain performance test results.

[0102] In some embodiments, data values ​​corresponding to each time point are extracted from the data and written to a database table; the database table interface is called to obtain the application's data values ​​at each time point; the application's performance test results are determined in the form of a coordinate system with time points as the horizontal axis and the application's data values ​​at each time point as the vertical axis.

[0103] As an example, the data collected in step S105 is stored in the form of a spreadsheet (Excel). Therefore, it is necessary to extract the data values ​​corresponding to each time point from the spreadsheet and write the data values ​​corresponding to each time point to a database table. The database table contains data from multiple applications. By calling the database table interface, the data values ​​of the applications at each time point are obtained. The performance test results of the applications are plotted in a coordinate system with the time points as the horizontal axis and the data values ​​of the applications at each time point as the vertical axis. The performance test results can be a curve graph.

[0104] Furthermore, server 200 can output the aforementioned performance test results. Optionally, server 200 can display the performance test results on a specific interface for testers to view, or it can export the performance test results to a file for saving, so that testers can view them at any time when needed.

[0105] This invention improves the efficiency of performance testing by collecting data generated by the application during performance testing in real time and analyzing and processing the collected data.

[0106] See Figure 4 , Figure 4 This is a flowchart illustrating the application performance testing method provided in this embodiment of the invention, based on... Figure 3 Steps S107 and S108 may be included after step S106.

[0107] In step S107, the interface of the database table is called to obtain data values ​​of multiple (i.e., at least two) applications at various points in time.

[0108] Here, multiple applications refer to multiple applications of the same type but different applications, or multiple applications of the same type but different versions.

[0109] In some embodiments, the database table contains data from multiple applications. By calling the database table's interface, the data values ​​of the multiple applications at various points in time can be obtained.

[0110] As an example, the database table contains data for versions 1 and 2 of application A, and version 1 and 2 of application B. By calling the database table's interface, it is possible to obtain at least the data for versions 1 and 2 of application A; at least the data for version 1 of application A and version 1 of application B; and at least the data for version 1 of application A and version 2 of application B.

[0111] In step S108, a performance comparison chart of multiple applications is determined and displayed based on the data values ​​of multiple applications at various time points.

[0112] In some embodiments, a performance comparison chart of multiple applications is plotted and displayed in the form of a coordinate system, with time points as the horizontal axis and data values ​​of multiple applications at each time point as the vertical axis.

[0113] Taking memory usage information as an example, by calling the database table interface, the memory usage information of application A and application B obtained includes the memory usage data values ​​of application A and application B at various time points. With the time point as the horizontal axis and the data values ​​of application A and application B at various time points as the vertical axis, a performance comparison chart of the memory usage information of application A and application B can be drawn in the form of a coordinate system, and the performance comparison chart of the memory usage information of application A and application B can be displayed.

[0114] This invention not only supports performance comparison between applications of the same type, but also between different versions of the same application. Furthermore, this invention provides testers with performance comparison charts to quickly analyze the problems existing in the application, thereby helping testers to further improve the application's performance.

[0115] See Figure 5 , Figure 5 This is a flowchart illustrating the application performance testing method provided in this embodiment of the invention, based on... Figure 3 Step S109 may be included after step S106.

[0116] In step S109, a reminder script is executed to periodically send performance test results and / or performance test links.

[0117] Here, the performance test link is a web link used to obtain performance test results. Clicking the performance test link will allow you to obtain (or view) the performance test results through a webpage.

[0118] In some embodiments, the recipient of performance test results and / or performance test links can be set in a reminder script before the application's performance test; the recipient of performance test results and / or performance test links can be an email address, social network, or SMS message (belonging to the tester or user).

[0119] In this embodiment of the invention, by running a script on the server and pre-setting the recipient of the performance test results and / or performance test links in the script, the server automatically sends the performance test results and / or performance test links to the recipient (e.g., testers or users) after the test is completed. This allows the recipient to obtain the performance test results and / or performance test links as soon as possible, thereby improving testing efficiency and human-computer interaction efficiency.

[0120] In some embodiments, after step S106, the method may further include: storing the performance test results in a blockchain network so that the blockchain network responds to a performance test data request for the application based on the stored performance test results.

[0121] Next Figure 1 Taking the example of server 200 uploading performance test results to a blockchain network for storage, this illustrates an exemplary application of the application performance testing method provided in this embodiment of the invention. See also... Figure 6 , Figure 6 This is an application diagram illustrating the performance testing method for an application provided in this embodiment of the invention, including a blockchain network 600 (consensus nodes 610-1 to 610-3 are shown as an example), an authentication center 700, and business entities 800 / 900, which will be described below.

[0122] The type of blockchain network 600 is flexible and diverse, and can be any of the following: public chain, private chain, or consortium chain. Taking a public chain as an example, any electronic device of any business entity (e.g., server 200) can access the blockchain network 600 and become a client node without authorization; taking a consortium chain as an example, after obtaining authorization, the electronic devices under the business entity can access the blockchain network 600 and become client nodes.

[0123] As an example, when blockchain network 600 is a consortium blockchain, business entities 800 / 900 register with certification authority 700 to obtain their respective digital certificates. The digital certificates include the business entity's public key and the digital signature signed by certification authority 700 with the public key and identity information of business entities 800 / 900. This digital signature is attached to the transaction along with the business entity's digital signature for the transaction (e.g., for storing performance test results on the blockchain or querying performance test results) and sent to blockchain network 600. Blockchain network 600 then retrieves the digital certificate and digital signature from the transaction to verify the reliability of the transaction (i.e., whether it has been tampered with) and the identity information of the business entity that sent the message. Blockchain network 600 will verify the identity, such as whether it has the authority to initiate the transaction.

[0124] In some embodiments, client nodes may act only as observers of the blockchain network 600, providing support for business entities to initiate transactions. For the functions of the consensus nodes 610 of the blockchain network 600, such as sorting, consensus services, and ledger functions, the client nodes may implement them by default or selectively (e.g., depending on the specific business needs of the business entity). This allows for the maximum migration of the business entity's data and business processing logic to the blockchain network 600, achieving trustworthiness and traceability of data and business processing through the blockchain network 600.

[0125] Consensus nodes in blockchain network 600 receive data from different business entities (e.g.) Figure 6The business entity (800 / 900) submits transactions, executes transactions to update or query the ledger, and various intermediate or final results of the transaction execution can be returned to the business entity's client node for display.

[0126] The following example illustrates an exemplary application of a blockchain network, using server 200 uploading performance test results to the blockchain network for storage. (See also: [link to relevant documentation]). Figure 6 Client node 810 can be server 200.

[0127] First, the logic for uploading performance test results to the blockchain is set on client node 810. For example, when the performance test results are obtained, client node 810 sends the performance test results to blockchain network 600, generating a corresponding transaction. The transaction includes: the smart contract that needs to be called to upload the performance test results to the blockchain, and the parameters passed to the smart contract; the transaction also includes the digital certificate of client node 810, the signed digital signature, and broadcasts the transaction to consensus node 610 in blockchain network 600.

[0128] Here, client node 810 can send performance test results to blockchain network 600 periodically or in real time. For example, it can send the results to blockchain network 600 whenever a performance test result is obtained; or it can send all the performance test results obtained within a certain time interval (e.g., 2 hours or 3 days) to blockchain network 600 to ensure the reliability of the performance test results obtained by blockchain network 600.

[0129] Then, when consensus node 610 in blockchain network 600 receives a transaction, it verifies the digital certificate and digital signature carried in the transaction. If the verification is successful, it confirms whether business entity 800 has the right to conduct the transaction based on the identity of the business entity 800 carried in the transaction. Any verification judgment in the digital signature and permission verification will cause the transaction to fail. After successful verification, consensus node 610 signs its own digital signature (for example, by encrypting the transaction digest using the private key of node 610-1) and continues to broadcast it in blockchain network 600.

[0130] Finally, after receiving the successfully verified transaction, consensus node 610 in blockchain network 600 populates the transaction into a new block and broadcasts it. When broadcasting a new block, consensus node 610 in blockchain network 600 verifies the new block, for example, verifying whether the digital signatures of the transactions in the new block are valid. If the verification is successful, the new block is appended to the end of its stored blockchain, and the state database is updated according to the transaction results. The transactions in the new block are then executed: for transactions that submit storage performance test results, key-value pairs including the performance test results are added to the state database.

[0131] Let's take the example of server 210 querying performance test results in blockchain network 600 to illustrate an exemplary application of blockchain networks. See [link to relevant documentation] Figure 6 Client node 910 can be server 210.

[0132] Here, server 210 is the server that needs to test the performance of the application, and the application to be tested by servers 200 and 210 is the same. Therefore, the performance test results required by servers 200 and 210 are the same. Thus, server 210 can directly obtain the performance test results through the blockchain network 600, thereby avoiding the need for server 210 to re-perform the performance test of the application to be tested in order to obtain the results, and thus saving server resources.

[0133] In some embodiments, the types of data that client node 910 can query in blockchain network 600 can be determined by consensus node 610 through restricting the permissions of transactions that the client phase of the business entity can initiate. When client node 910 has the permission to initiate a query for performance test results, client node 910 can generate a transaction for querying performance test results and submit it to blockchain network 600. Consensus node 610 executes the transaction to query the corresponding performance test results from the state database and returns them to client node 910.

[0134] The following will continue to combine Figure 2 The implementation of the application performance testing device 533 provided in this embodiment of the invention is an exemplary structure of a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the performance testing device 533 for the application stored in the memory 530 may include: an injection module 5331, a sending module 5332, an acquisition module 5333, and an analysis module 5334.

[0135] The injection module 5331 is used to run the application based on the application's installation package and inject test code corresponding to the performance test scenario into the running application;

[0136] The sending module 5332 is used to run a simulation service of the application through the test code, and send a pseudo-random user event stream to the application through the simulation service to trigger the application to run according to the pseudo-random user event stream;

[0137] The acquisition module 5333 is used to acquire data generated by the application in real time during the process of the application running according to the pseudo-random user event stream;

[0138] The analysis module 5334 is used to analyze and process the data to obtain performance test results.

[0139] In some embodiments, the injection module 5331 is further configured to inject test code corresponding to the performance test scenario into the running application by executing a performance test script to obtain the control tree information of the application; based on the control tree information of the application, establish a communication channel between the simulation service and the application, so that the simulation service sends the pseudo-random user event stream to the application through the communication channel; wherein, the performance test scenario includes at least one of the following: startup test scenario; homepage swiping test scenario; entering live room test scenario; memory usage test scenario; CPU usage test scenario; GPU usage test scenario; smoothness test scenario; power consumption test scenario.

[0140] In some embodiments, the sending module 5332 is further configured to send the pseudo-random user event stream to the application via the simulation service and through the communication channel, so that the pseudo-random user event stream is transmitted in the control tree of the application, and the application responds to the pseudo-random user event stream transmitted in the control tree.

[0141] In some embodiments, the acquisition module 5333 is further configured to record the application's running log in real time during the application's operation according to the pseudo-random user event stream; create a data acquisition task for the type of the performance test scenario; and extract corresponding data from the running log according to the data acquisition task corresponding to the performance test scenario; wherein the data includes at least one of the following information: startup time information; memory usage information; central processing unit usage information; graphics processing unit usage information; smoothness information; and power consumption information.

[0142] In some embodiments, the analysis module 5334 is further configured to extract data values ​​corresponding to each time point from the data and write the data values ​​corresponding to each time point to a database table; call the interface of the database table to obtain the data values ​​of the application at each time point; and determine the performance test results of the application in the form of a coordinate system with the time point as the horizontal axis and the data values ​​of the application at each time point as the vertical axis.

[0143] In some embodiments, the application performance testing device 533 further includes: an installation package acquisition module, configured to pull the installation package of the application from the branch corresponding to the application in the application building platform by executing an installation package acquisition script; or to obtain the installation package of the application from an application store by executing the installation package acquisition script.

[0144] In some embodiments, the application performance testing device 533 further includes: a display module, configured to call the interface of a database table to obtain data values ​​of multiple applications at various points in time; wherein, the multiple applications are multiple applications of the same type but different applications, or multiple applications of the same type but different application versions; and based on the data values ​​of the multiple applications at various points in time, determine and display a performance comparison chart of the multiple applications.

[0145] In some embodiments, the application performance testing device 533 further includes: a reminder module, configured to periodically send the performance test results and / or performance test links by executing a reminder script; wherein the performance test link is a network link used to obtain the performance test results.

[0146] In some embodiments, the application performance testing apparatus 533 further includes an on-chain module for storing the performance test results in a blockchain network, so that the blockchain network responds to a performance test data request for the application based on the stored performance test results.

[0147] This invention provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the performance testing method for the application provided in this invention. For example, ... Figure 3 , Figure 4 or Figure 5 The application performance testing method shown is used on computers including various computing devices such as smart terminals and servers.

[0148] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0149] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0150] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hypertext Markup Language document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0151] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0152] The following example, using a live streaming app as the test application, illustrates the application performance testing method provided in this embodiment of the invention.

[0153] In related technologies, performance testing of an app can be achieved by modifying its underlying code. However, this approach has the following technical problems: First, while modifying the underlying code can yield performance test data, it cannot guarantee that the modification will not affect the original performance test data. Second, the obtained performance test data cannot be optimally analyzed by testers quickly, requiring significant manpower for secondary analysis. Third, it does not support performance comparisons between similar apps or between different versions of the same app.

[0154] To address the aforementioned issues, this invention proposes a performance testing method for applications. This method does not require modifying the underlying code of the app under test. It simply involves downloading the installation packages of multiple apps and then combining automated testing tools (tools that encapsulate the aforementioned application performance testing functions) and performance testing tools (tools that encapsulate the aforementioned performance test data collection functions) to obtain performance data (i.e., the aforementioned performance test data or data) of the apps under test in various performance testing scenarios. This allows for the creation of performance curve comparisons within the same performance testing scenario. Furthermore, through front-end technology, the performance data can be presented intuitively, enabling testers to quickly analyze the performance differences among the multiple apps under test.

[0155] The specific implementation scheme of this invention is as follows:

[0156] See Figure 7 , Figure 8 and Figure 9 , Figure 7 , Figure 8 and Figure 9This is a schematic diagram illustrating an application scenario of the performance testing method for applications provided in this embodiment of the invention. Figure 7 On the product presentation page, the first item, "Please Select Product," allows users (or testers) to select a business product (i.e., the APP to be tested mentioned above). After the user (or tester) selects a business product, the page will load the latest performance data of that business product and present it in the form of a chart. The page can also display a performance data comparison between the business product and competing products (i.e., competing products of the same type as the business product). The second item, "Version Comparison," allows users (or testers) to view a performance data comparison between different versions of the same business product. The third item, "Lag Analysis," can be used to upload and analyze logs to provide users (or testers) with lag analysis results.

[0157] Figure 8 It is a performance data comparison chart between the business product and competitors (i.e., competing products of the same type as the business product). Figure 8 In the test setup, the "Execution Environment" section provides users (or testers) with basic information such as the device model, version, and processor used by the product. The "Overall Overview" section allows users (or testers) to compare key performance test data, such as frame rate, average memory usage, average CPU usage, and average GPU usage. The "Test Scenarios" section shows the trends of performance test data, including CPU usage, memory usage, and GPU usage, under various performance test scenarios. Here, since the application under test is a live streaming app, the performance test scenarios include app startup test scenarios, homepage swiping test scenarios, and entering the live streaming room test scenarios.

[0158] Figure 9 This is a performance data comparison chart between different versions of the same business product. Figure 9 In the test setup, the "Execution Environment" section provides users (or testers) with basic information such as the device model, version, and processor used by the product. The "Overall Overview" section allows users (or testers) to compare key performance test data, such as frame rate, average memory usage, average CPU usage, and average GPU usage. The "Test Scenarios" section shows the trends of performance test data, including CPU usage, memory usage, and GPU usage, under various performance test scenarios. Here, since the application under test is a live streaming app, the performance test scenarios include app startup test scenarios, homepage swiping test scenarios, and entering the live streaming room test scenarios. Figure 9 This demonstrates the performance changes of the business product during version iteration, using an APP startup test scenario as an example. Figure 9It can quickly compare the changes in CPU usage, memory usage, and GPU usage between the previous and current versions, thereby quickly analyzing the areas that need optimization in the current version and improving the user experience.

[0159] See Figure 10 , Figure 10 This is a schematic diagram of the application performance testing system provided in an embodiment of the present invention. The following is a description of the system in conjunction with... Figure 10 Please provide a detailed explanation.

[0160] (1) Automation terminal (or automation unit)

[0161] The automation module includes three functional logics: ① Launching business products and competitor apps through automated testing tools, and using automated scripts to control the business products and competitor apps to enter the corresponding performance testing scenarios for performance testing; ② Launching performance detection tools through automated testing tools (or directly calling performance detection tools), and using performance detection tools to collect performance test data obtained by business products and competitor apps during performance testing in the corresponding performance testing scenarios to generate raw data files; ③ Using the timer function, periodically launching automated scripts to execute automated tasks (or performance testing tasks), and reporting the raw data files to the server.

[0162] Here, each time the automated script starts, the automated end will automatically pull (i.e. obtain) the latest APP build package (i.e. the installation package mentioned above) of the business product and competitors from the corresponding branch of the application store or application build platform (RDM). If the latest APP build package cannot be obtained, it can be obtained through manual intervention.

[0163] (2) Server-side (or service unit)

[0164] The server uses a World Wide Web (WEB) application framework (such as the Django framework). On the one hand, this framework meets the functional requirements of performance testing, and on the other hand, it can accelerate the development or testing progress.

[0165] The server-side includes three functional logics: ① Processing the data uploaded by the automation end (i.e., the raw data file mentioned above) and writing the analyzed and processed data into a database table; ② Retrieving the corresponding specific data from the database table according to the request from the display end and returning it to the display end for the display end to present performance reports (i.e., the performance test results mentioned above); ③ Periodically pushing the results of the automation task (i.e., the performance test results or performance test links mentioned above) to the enterprise WeChat (group) for all parties to pay attention to.

[0166] (3) Display terminal (or display unit)

[0167] The dashboard allows users to view the runtime environment, overall data, and detailed data of automated tasks. It also supports horizontal comparisons of performance data between business products and competitors (e.g.,...). Figure 8 It also supports vertical comparison of performance data between different versions of the same business product (e.g., Figure 9 ).

[0168] In summary, the embodiments of the present invention have the following beneficial effects:

[0169] (1) It not only supports performance comparison between applications of the same type, but also supports performance comparison between different versions of the same application. It also provides testers with performance comparison charts to quickly analyze the problems in the application, thereby helping testers to further improve the performance of the application.

[0170] (2) By executing the installation package acquisition script, the installation package of the application can be automatically obtained from the application building platform or application store, which speeds up the acquisition time of the application installation package and improves the efficiency of the application testing process.

[0171] (3) Injecting test code corresponding to the performance test scenario into the running application does not modify the underlying implementation code of the application. Unlike related technologies that achieve performance testing of the application by changing the underlying code of the application to be tested, this can solve the technical problem in related technologies that cannot guarantee whether changing the underlying code of the application to be tested will affect the original performance test data of the application to be tested, thereby improving the accuracy of performance testing.

[0172] (4) By sending pseudo-random user event streams to the application through a simulation service, the application runs according to the pseudo-random user event streams, which enables the testing of the application without human intervention, thereby saving human resources and improving the efficiency of performance testing.

[0173] (5) Based on the data collection task, the system automatically collects data generated by the application during performance testing. This solves the technical problems in related technologies where performance testing requires manual triggering by testers, and different indicators need to be tested one by one. This manual testing is time-consuming and inefficient, thus saving human resources and improving the efficiency and accuracy of performance testing. Furthermore, this embodiment of the invention performs performance analysis based on the collected data to obtain performance analysis results and provide solutions. This solves the technical problem in related technologies where the performance test data obtained cannot be quickly analyzed by testers, thereby facilitating testers to quickly identify problems in the application and improve its performance.

[0174] (6) By collecting data generated by the application during the performance test in real time and analyzing and processing the collected data to obtain the performance test results, the efficiency of performance testing can be improved without human intervention.

[0175] (7) By running a script on the server and setting the recipient of the performance test results and / or performance test links in advance in the script, the server can automatically send the performance test results and / or performance test links to the recipient (e.g., testers or users) after the test is completed. This allows the recipient to obtain the performance test results and / or performance test links as soon as possible, thereby improving testing efficiency and human-computer interaction efficiency.

[0176] (8) By storing the performance test results in the blockchain network, the blockchain network can respond to the performance test data request for the application based on the stored performance test results, which can avoid repeatedly performing performance tests on the application to be tested and save server resources.

[0177] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are included within the scope of protection of the present invention.

Claims

1. A performance testing method for an application, characterized in that, The method includes: The application is run based on the application's installation package. By executing a performance test script, test code corresponding to the performance test scenario is injected into the process of the running application. The control tree information in the process of the running application is obtained through the reflection function in the test code. During the operation of the application, a simulated service is created using the test code; Based on the control tree information of the application, a communication channel is established between the simulation service and the application. The simulation service of the application is run through the test code. The simulation service sends a pseudo-random user event stream to the application through the communication channel, so that the pseudo-random user event stream is transmitted in the control tree of the application. The application responds to the pseudo-random user event stream transmitted in the control tree to trigger the application to run according to the pseudo-random user event stream. During the operation of the application based on the pseudo-random user event stream, data generated by the application is collected in real time; the data is analyzed and processed to obtain performance test results.

2. The method according to claim 1, characterized in that, The performance test scenario includes at least one of the following: Startup test scenario; memory usage test scenario; CPU usage test scenario; GPU usage test scenario; smoothness test scenario; Power consumption test scenario.

3. The method according to claim 1, characterized in that, The process of collecting data generated by the application in real time during the application's operation based on the pseudo-random user event stream includes: During the process of the application running according to the pseudo-random user event stream, the application's running log is recorded in real time; Create a data acquisition task for the type of performance test scenario described above; Based on the data acquisition task corresponding to the performance test scenario, extract the relevant data from the running log; The data includes at least one of the following: Startup time information; memory usage information; CPU usage information; GPU usage information; smoothness information; power consumption information.

4. The method according to claim 1, characterized in that, The process of analyzing and processing the data to obtain performance test results includes: Extract the data values ​​corresponding to each time point from the data, and write the data values ​​corresponding to each time point to the database table; Call the interface of the database table to obtain the data values ​​of the application at various points in time; The performance test results of the application are determined in the form of a coordinate system, with time points as the horizontal axis and the data values ​​of the application at each time point as the vertical axis.

5. The method according to any one of claims 1 to 4, characterized in that, Before the application-based installation package runs the application, the method further includes: The installation package is retrieved from the branch of the application in the application building platform by executing the installation package retrieval script. Alternatively, the installation package of the application can be obtained from the application store by executing the installation package retrieval script.

6. The method according to any one of claims 1 to 4, characterized in that, After analyzing and processing the data to obtain performance test results, the method further includes: Call the database table interface to obtain data values ​​from multiple applications at various points in time; Among them, the multiple applications are multiple applications of the same type but different applications, or multiple applications of the same type but different application versions; Based on the data values ​​of the multiple applications at various points in time, a performance comparison chart of the multiple applications is determined and displayed.

7. The method according to any one of claims 1 to 4, characterized in that, After analyzing and processing the data to obtain performance test results, the method further includes: The performance test results and / or performance test links are sent periodically by executing a reminder script. The performance test link is a network link used to obtain the performance test results.

8. The method according to any one of claims 1 to 4, characterized in that, After analyzing and processing the data to obtain performance test results, the method further includes: The performance test results are stored in a blockchain network so that... The blockchain network responds to performance test data requests for the application based on the stored performance test results.

9. A performance testing device for an application, characterized in that, The device includes: The injection module is used to run the application based on the application's installation package, inject test code corresponding to the performance test scenario into the process of the running application by executing a performance test script, obtain control tree information in the process of the running application through the reflection function in the test code, and create a simulated service through the test code during the operation of the application. The sending module is used to establish a communication channel between the simulation service and the application based on the control tree information of the application, run the simulation service of the application through the test code, and send a pseudo-random user event stream to the application through the communication channel via the simulation service, so that the pseudo-random user event stream is transmitted in the control tree of the application, and the application responds to the pseudo-random user event stream transmitted in the control tree to trigger the application to run according to the pseudo-random user event stream; The data acquisition module is used to collect data generated by the application in real time during the process of the application running according to the pseudo-random user event stream; The analysis module is used to analyze and process the data to obtain performance test results.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the performance testing method for the application described in any one of claims 1 to 8.

11. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the performance testing method for the application described in any one of claims 1 to 8.

Images